Compare commits

...
532 Commits
Author SHA1 Message Date
Aayush Rajasekaran 232cc320bd Merge pull request #3747 from filecoin-project/ntwk-0.7.0
Ntwk 0.7.0
2020-09-10 18:07:06 -04:00
Aayush Rajasekaran e7edc46daf Lotus version 0.7.0 2020-09-10 17:56:54 -04:00
Jakub Sztandera 6813ab4e65 Add additional metrics
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
Jakub Sztandera fd714cbceb Remove PackingEfficiency consideration from base fee in an upgrade
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
whyrusleeping db6bd890d9 condition randomness changes on fork height
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
Steven Allen 7a46c5ff31 [WIP] Fix ticket randomness
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
Aayush Rajasekaran 98a9042c07 Use newest network version in genesiss setup stuff 2020-09-10 17:41:55 -04:00
Aayush Rajasekaran 4170e18dea fix testground build 2020-09-10 17:41:55 -04:00
Aayush Rajasekaran 230ef2484d fix tests 2020-09-10 17:41:55 -04:00
Aayush Rajasekaran beba92aed4 Improve network versioning logic 2020-09-10 17:41:55 -04:00
Aayush Rajasekaran cbb693e99e Update docs 2020-09-10 17:41:55 -04:00
Aayush Rajasekaran 774e068436 Update to specs-actors v0.9.8 2020-09-10 17:41:55 -04:00
Aayush Rajasekaran 86ba21029d Update to go state types 001afaca718c 2020-09-10 17:41:55 -04:00
Aayush Rajasekaran 316ac61875 Update to go state types 3ca0d2890090 2020-09-10 17:41:55 -04:00
Jakub Sztandera d5d264d13b Add fork point
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
Jakub Sztandera c71e1adc93 Integrate multiple drand networks with pubsub
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
Jakub Sztandera 564d0ae974 Upgrade drand
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
Jakub Sztandera 9a48ec194e Add DrandSchedule to testground params
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
Jakub Sztandera bb0a7f91cc Fix drand non-test
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
Jakub Sztandera 73e54352cd Fix off by one
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
Jakub Sztandera 64fa6fd9e5 Draw the rest of the owl
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
Jakub Sztandera 2d3f92aeed Introduce beacon Schedule
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-10 17:41:55 -04:00
Łukasz Magiera 9e9856bb45 Merge pull request #3590 from filecoin-project/asr/multisig
Multisig API additions
2020-09-10 17:04:48 +02:00
Łukasz Magiera 890e6280df Merge pull request #3669 from filecoin-project/feat/independent-version-numbers
build: Separate API versions per node type
2020-09-10 17:00:32 +02:00
Łukasz Magiera da7e4b3a09 Merge pull request #3735 from filecoin-project/deps/update-pubsub
update go-libp2p-pubsub@master
2020-09-10 15:35:28 +02:00
Raúl Kripalani 747a7de168 Merge pull request #3733 from filecoin-project/feat/chaos-abort-2 2020-09-10 14:20:34 +01:00
vyzo bfa47b3663 update go-libp2p-pubsub@master 2020-09-10 16:11:23 +03:00
Alan Shaw 0e7f14159a refactor: another rename 2020-09-10 12:06:00 +01:00
Alan Shaw b755ce9528 refactor: renames 2020-09-10 12:03:13 +01:00
Alan Shaw f3cae55bd4 feat: chaos abort 2020-09-10 11:31:34 +01:00
Łukasz Magiera f64bd55bc6 Merge pull request #3720 from filecoin-project/fix/increase-precommit2-retry-counter
Increase the number of times precommit2 is attempted before moving back to precommit1
2020-09-10 11:52:01 +02:00
Łukasz Magiera 4c8bb846ea Merge pull request #3724 from filecoin-project/fix/relax-strictness-push
relax mpool add strictness checks for local pushes
2020-09-10 11:51:02 +02:00
Łukasz Magiera bd14fb90fc Merge pull request #3729 from filecoin-project/fix/statediff-ci
ci: fix statediff build; make optional.
2020-09-10 11:49:04 +02:00
Raúl Kripalani 9e5bce90f4 ci: fix statediff build; make optional. 2020-09-10 10:41:18 +01:00
vyzo 72c7d4c886 relax mpool add strictness checks for local pushes
So that a node can have more than 1k pending messages for its own local addresses.
2020-09-10 09:55:50 +03:00
lanzafame 558b637b07 Increase the number of times precommit2 is attempted before moving back to precommit1 2020-09-10 13:52:21 +10:00
Whyrusleeping 767341b279 Merge pull request #3437 from filecoin-project/feat/mpool-stats-cmd
add a simple command to watch messages sitting in the mempool
2020-09-09 18:29:42 -07:00
whyrusleeping c73916b4e7 fix the linter 2020-09-09 17:37:49 -07:00
whyrusleeping 0361ca1c39 add some bits about windowed post 2020-09-09 17:18:55 -07:00
Aayush Rajasekaran cd4cd22928 Merge pull request #3717 from filecoin-project/asr/checkpoint-msg
Fix IsAncestorOf
2020-09-09 19:49:47 -04:00
Aayush Rajasekaran e1a0cf6ca5 Add a checkpoint test 2020-09-09 19:21:48 -04:00
Łukasz Magiera 73bb9d5afb Merge pull request #3715 from filecoin-project/fix/chainwatch/partition-diff
fix(chainwatch): compare prev miner with cur miner
2020-09-10 01:19:58 +02:00
Aayush Rajasekaran 183df4c135 Fix IsAncestorOf 2020-09-09 19:09:55 -04:00
frrist f2b238f50a fix(chainwatch): compare prev miner with cur miner
- bug was comparing cur miner with cur miner
2020-09-09 15:30:30 -07:00
whyrusleeping 09194aa613 print better error when losing connection 2020-09-09 15:19:54 -07:00
whyrusleeping e58327d4ed rename LE tag to quantile 2020-09-09 15:19:54 -07:00
whyrusleeping 8414aa6e4b more metrics! 2020-09-09 15:19:54 -07:00
whyrusleeping 7d1d690eb7 lets get some metrics 2020-09-09 15:19:54 -07:00
whyrusleeping 47ad7ccb5f some more stats 2020-09-09 15:19:54 -07:00
whyrusleeping 679c4183e7 add a simple command to watch messages sitting in the mempool 2020-09-09 15:19:53 -07:00
Aayush Rajasekaran 8807c75c51 Fix warn message 2020-09-09 18:17:05 -04:00
Łukasz Magiera be9d8f7ce3 Merge pull request #3712 from filecoin-project/asr/devnet
Fix devnets
2020-09-09 23:15:36 +02:00
Aayush Rajasekaran a7a43a1124 Fix devnets 2020-09-09 17:10:35 -04:00
Aayush Rajasekaran 8b2862a1b8 Merge pull request #3706 from filecoin-project/asr/changelog
Lotus version 0.6.2
2020-09-09 16:45:10 -04:00
Aayush Rajasekaran 1d4b5c201e Lotus version 0.6.2 2020-09-09 16:44:37 -04:00
Łukasz Magiera f983145897 Merge pull request #3708 from filecoin-project/feat/warn-random-selection
warn when optimal selection fails to pack a block and we fall back to random selection
2020-09-09 22:42:04 +02:00
vyzo d369f542c5 warn when optimal selection fails to pack a block and we fall back to random selection 2020-09-09 23:17:09 +03:00
Łukasz Magiera 6c8a44b056 Merge pull request #3701 from filecoin-project/feat/gas-perf-cmd
add basic command for printing gas performance of messages in the mpool
2020-09-09 21:39:40 +02:00
Jakub Sztandera ef18f93609 Merge pull request #3698 from filecoin-project/fix/mpool-packing-failure
fix mpool optimal selection packing failure
2020-09-09 21:38:26 +02:00
Łukasz Magiera 540afb1573 Merge pull request #3704 from filecoin-project/misc/kubuxu-codeowners
Add Kubuxu to CODEOWNERS
2020-09-09 21:38:04 +02:00
Jakub Sztandera 2d7a924275 Add Kubuxu to CODEOWNERS
3 pairs of eyes are better than two and getting directly notified on PR is what I want (but GH notifications have no repo+PR filter).
2020-09-09 21:32:03 +02:00
Aayush Rajasekaran 702799290f Merge pull request #3680 from filecoin-project/asr/checkpoint
Allow nodes to mark tipsets as checkpointed
2020-09-09 15:22:07 -04:00
vyzo f7d3d371f7 print reward and actual gas performancec 2020-09-09 22:11:51 +03:00
Łukasz Magiera 311d20eeb2 Merge pull request #3702 from filecoin-project/steb/remove-hard-coded-late-fee
Remove hard-coded late-fee in window PoSt
2020-09-09 20:50:57 +02:00
Steven Allen afa913f2ed Remove hard-coded late-fee in window PoSt
This is no longer necessary, and doesn't really serve any purpose.
2020-09-09 11:34:55 -07:00
Aayush Rajasekaran f0554676dd Explicitly check tipsetkeys when considering checkpoint 2020-09-09 14:34:05 -04:00
Aayush Rajasekaran 3337b9a98f Update docs 2020-09-09 14:34:05 -04:00
Aayush Rajasekaran 22fdb9594d Add CLI to checkpoint tipsets 2020-09-09 14:34:05 -04:00
Aayush Rajasekaran 93a0052219 Add a SyncUnmarkBad CLI 2020-09-09 14:34:05 -04:00
Aayush Rajasekaran 5c852e26c2 Add a SyncUnmarkBad API 2020-09-09 14:34:05 -04:00
Aayush Rajasekaran 7d2118cd97 Fix a typo that's totally related to this PR 2020-09-09 14:34:05 -04:00
Aayush Rajasekaran 5e3c178b9e Add a sync test for checkpointed tipsets 2020-09-09 14:34:04 -04:00
Aayush Rajasekaran 35aa78dad9 Allow marking a certain tipset as checkpointed 2020-09-09 14:34:00 -04:00
whyrusleeping 7747f5a3fa add basic command for printing gas performance of messages in the mempool 2020-09-09 11:23:48 -07:00
Łukasz Magiera a2f1e76fea Merge pull request #3688 from filecoin-project/fix/test-vectors-import-cycle
resolve lotus, test-vectors, statediff dependency cycle
2020-09-09 19:53:54 +02:00
vyzo cf515bc5ca add selection test with real world messages dumped from a live node mpool 2020-09-09 20:46:26 +03:00
vyzo 97cbfa11b7 fix nasty bug in optimal selection
the sort has to account for negative gasPerf and be stable; go figure.
2020-09-09 20:45:24 +03:00
Łukasz Magiera f2b76d8b3b Merge remote-tracking branch 'origin/master' into fix/test-vectors-import-cycle 2020-09-09 19:41:24 +02:00
Whyrusleeping e2a6d02570 Merge pull request #3664 from filecoin-project/add-sleep-when-drand-errors
add niceSleep 1 second when drand errors
2020-09-09 10:37:43 -07:00
Łukasz Magiera 1d78ffc04d Merge pull request #3676 from filecoin-project/feat/mpool-replace-auto
add an auto flag to mpool replace
2020-09-09 19:31:43 +02:00
Łukasz Magiera 42b9d42a8f Merge pull request #3624 from filecoin-project/blocksync-refactor
blocksync: introduce interfaces; rename to ChainExchange (abbrev. chainxchg)
2020-09-09 19:29:20 +02:00
Łukasz Magiera 7dbbe84086 Merge pull request #3690 from filecoin-project/docs/cli-paych-status
paych: add docs on how to use paych status
2020-09-09 19:25:32 +02:00
whyrusleeping e9b85f5acb ensure replaced message can properly RBF 2020-09-09 10:23:41 -07:00
Łukasz Magiera cb3b0ab2bb Merge remote-tracking branch 'origin/master' into blocksync-refactor 2020-09-09 19:19:05 +02:00
Łukasz Magiera de772bf1f9 Merge pull request #3696 from filecoin-project/bugfix/pubsish
typo fix in error message
2020-09-09 18:45:20 +02:00
Łukasz Magiera 91a9bae308 Merge pull request #3691 from filecoin-project/codeowners
initial CODEOWNERS.
2020-09-09 18:43:28 +02:00
Mosh fe7c152c97 typo fix in error message 2020-09-09 12:24:33 -04:00
Raúl Kripalani 333c7ce1c9 initial CODEOWNERS. 2020-09-09 14:40:30 +01:00
Dirk McCormick ef2303bc5d docs: add docs on how to use paych status 2020-09-09 15:37:34 +02:00
Raúl Kripalani 6d2b4ab775 fix lint. 2020-09-09 14:28:56 +01:00
Raúl Kripalani b3dedfedc0 ci: remove unnecessary go get. 2020-09-09 14:20:11 +01:00
Raúl Kripalani 2a3743ff84 invoke the statediff command properly. 2020-09-09 14:08:54 +01:00
Raúl Kripalani a5afd83c7b Merge branch 'master' into fix/test-vectors-import-cycle 2020-09-09 13:04:41 +01:00
Raúl Kripalani e8d1bab914 make the runner call statediff as a binary. 2020-09-09 13:03:43 +01:00
Łukasz Magiera de654de911 Merge pull request #3687 from filecoin-project/misc/v0.6.2-rc1
Bump version to 0.6.2-rc1
2020-09-09 13:08:51 +02:00
Łukasz Magiera 37c0f75499 Bump version to 0.6.2-rc1 2020-09-09 13:07:15 +02:00
Łukasz Magiera 5c5c1c80cf Merge pull request #3686 from filecoin-project/fix/gas-premium-median
gas: Fix median calc
2020-09-09 12:22:10 +02:00
Łukasz Magiera e91ee9f62a Fix lint 2020-09-09 12:18:02 +02:00
Łukasz Magiera 213b4eace8 Merge pull request #3685 from filecoin-project/fix/mpool-selection-full-blocks
adjust optimal selection to always try to fill blocks
2020-09-09 12:15:38 +02:00
Łukasz Magiera f695c0c164 gas: Add tests to median premium math 2020-09-09 12:14:46 +02:00
vyzo 194b6eb9d8 fix issue with gasLimit check for trimming (> instead of >=) 2020-09-09 12:58:51 +03:00
Łukasz Magiera 2aba16e2c9 gas: Fix median calc 2020-09-09 11:41:02 +02:00
vyzo 0b09082450 adjust optimal selection to always try to fill blocks 2020-09-09 12:37:03 +03:00
Łukasz Magiera 29aa972f18 Merge pull request #3546 from filecoin-project/feat/netconnect-miner-addr
net cli: Support miner addresses in net connect
2020-09-09 08:55:51 +02:00
Aayush Rajasekaran 06b814b627 Merge pull request #3673 from filecoin-project/fix/long-syncs-over-upgrade
pass tipset through upgrade logic
2020-09-08 22:28:42 -04:00
whyrusleeping 8c0994e290 add an auto flag to mpool replace 2020-09-08 15:47:40 -07:00
Łukasz Magiera b1d251e950 Merge pull request #3667 from filecoin-project/fix/new-ts
Add peer to peer manager before fetching the tipset
2020-09-08 22:54:51 +02:00
Raúl Kripalani 63cdbef219 temp fix test-vectors import cycle. 2020-09-08 21:50:25 +01:00
whyrusleeping 8a8f0ab3ec pass tipset through upgrade logic 2020-09-08 13:45:44 -07:00
Łukasz Magiera 1ef0359731 Merge pull request #3668 from filecoin-project/feat/sync-fetching-messages-state
Add StageFetchingMessages to sync status
2020-09-08 21:15:05 +02:00
Łukasz Magiera 38863d3025 build: Separate API versions per node type 2020-09-08 21:14:53 +02:00
Jakub Sztandera 44f4372ca3 Add StageFetchingMessages to sync status
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-08 20:42:20 +02:00
Jakub Sztandera 46d3769a44 Add peer to peer manager before fetching the tipset
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-08 20:13:16 +02:00
Łukasz Magiera 65b7d71bdf Merge pull request #3663 from filecoin-project/fix/minor-mpool-bug
fix very minor bug in repub baseFeeLowerBound
2020-09-08 19:49:51 +02:00
Anton Evangelatov 1d67e38f78 add niceSleep 1 second when drand errors 2020-09-08 18:35:10 +02:00
Łukasz Magiera 3697a1af4d Merge pull request #3655 from filecoin-project/fix/stgmgr-undef-unsealed
storage manager: bail out with an error if unsealed cid is undefined (redux)
2020-09-08 18:03:24 +02:00
vyzo 26ff596983 fix very minor bug in repub baseFeeLowerBound
messages will not be accepted if the GasFeeCap is less than minimumBaseFee, but it doesn't hurt
to be correct.
2020-09-08 19:02:05 +03:00
Raúl Kripalani b33db9c1ab Merge branch 'master' into blocksync-refactor 2020-09-08 14:22:43 +01:00
Łukasz Magiera 867469e9b3 Merge pull request #3648 from filecoin-project/fix/dont-use-latency
Don't use latency as initital estimate for blocksync
2020-09-08 14:56:23 +02:00
Łukasz Magiera a4fc5c6975 Merge pull request #3650 from filecoin-project/fix/chain-nearsync-check
fix isChainNearSync check in block validator
2020-09-08 14:52:36 +02:00
Łukasz Magiera e87ccbfa22 Merge pull request #3657 from filecoin-project/fix/stg-sealer-read-piece
storage: return true from Sealer.ReadPiece() on success
2020-09-08 14:44:30 +02:00
Dirk McCormick 17c15a74a2 fix: return true from Sealer.ReadPiece() on success 2020-09-08 13:50:56 +02:00
Dirk McCormick 8bbdf2e7cb fix: storage manager - bail out on undefined unsealed cid 2020-09-08 12:54:01 +02:00
Jakub Sztandera b66058e417 Add 0 guard
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-08 12:19:37 +02:00
Jakub Sztandera 4fce0181ab Ignore the linter
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-08 12:18:48 +02:00
Jakub Sztandera ba9678bd61 Use for to iterate over channel
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-08 12:11:30 +02:00
Jakub Sztandera 74e577610a Forward peers from hello to blocksync
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-08 10:18:51 +02:00
vyzo 37ec1a978e nicer check using Time objects 2020-09-08 10:47:41 +03:00
vyzo 27fa9969f3 fix isChainNearSync check in block validator 2020-09-08 10:26:27 +03:00
Łukasz Magiera 4045ffea3c Merge pull request #3649 from filecoin-project/asr/changelog
Lotus version 0.6.1
2020-09-08 09:14:14 +02:00
Aayush Rajasekaran e9e067eebf Lotus version 0.6.1 2020-09-08 03:10:27 -04:00
Jakub Sztandera 93176c91f4 Track time in relation to request size
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-08 09:06:31 +02:00
Łukasz Magiera 7ee5973400 Merge pull request #3645 from filecoin-project/asr/test-vectors-update
Update test-vectors
2020-09-08 09:05:40 +02:00
Jakub Sztandera 043e62ab63 Don't use latency as initital estimate for blocksync
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-08 08:59:05 +02:00
Aayush Rajasekaran f91f665f3f Update test-vectors 2020-09-08 01:24:30 -04:00
Aayush Rajasekaran f985f42e93 Merge pull request #3643 from filecoin-project/revert-3602-feat/delayed-pubsub-join
Revert "only subscribe to pubsub topics once we are synced"
2020-09-08 00:43:03 -04:00
Aayush Rajasekaran c77f5f6252 Revert "only subscribe to pubsub topics once we are synced" 2020-09-08 00:32:26 -04:00
Aayush Rajasekaran 60cae87bfa Merge pull request #3635 from filecoin-project/ntwk-0.6.0
Network Upgrade: 0.6.0
2020-09-07 19:29:35 -04:00
Aayush Rajasekaran eda33f7d11 Merge pull request #3634 from filecoin-project/fix/proper-epoch-breeze
use defined constant for fork height
2020-09-07 19:20:05 -04:00
whyrusleeping d1cf445327 use defined constant for fork height 2020-09-07 16:18:22 -07:00
Aayush Rajasekaran 8e3dd11f81 Merge pull request #3633 from filecoin-project/asr/changelog
Lotus version v.0.6.0
2020-09-07 19:16:47 -04:00
Aayush Rajasekaran 80a4beace6 Lotus version v.0.6.0 2020-09-07 19:14:17 -04:00
Aayush Rajasekaran 6dc821186e Merge pull request #3632 from filecoin-project/feat/faucet-fixins
Implement faucet funds reallocation logic
2020-09-07 19:13:39 -04:00
whyrusleeping 26ec0716b9 fix lint errors 2020-09-07 16:12:44 -07:00
whyrusleeping 97af576f6a Implement faucet funds reallocation logic
use all committed sectors instead of power

give all miners who had at least a sector on friday a base amount

include genesis miner in redistribution

drop back to 1,000,000
2020-09-07 16:06:13 -07:00
Aayush Rajasekaran 0e986df7e5 Merge pull request #3627 from filecoin-project/asr/specs-update-0.9.7
Update to specs 0.9.7 and markets 0.6.0
2020-09-07 18:00:56 -04:00
Aayush Rajasekaran 45011c0fad Use latest version if no upgrade height is specified 2020-09-07 17:43:14 -04:00
Jakub Sztandera ca919dddd6 Merge pull request #3628 from filecoin-project/fix/fork-treshold
Upgrade to correct fork treshold
2020-09-07 23:17:24 +02:00
Aayush Rajasekaran 6eea5dd109 Update to specs 0.9.7 and markets 0.6.0 2020-09-07 17:11:32 -04:00
Jakub Sztandera 46ee2eeded Upgrade to correct fork treshold
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-07 23:07:33 +02:00
Łukasz Magiera 3741186656 Merge pull request #3625 from filecoin-project/fix/temporary-allow-negative-chains
temporarily allow negative chains
2020-09-07 23:06:15 +02:00
Jakub Sztandera 704db225d1 Update the fork point
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-07 22:45:06 +02:00
Jakub Sztandera ce0e9594df Update to go-state-types
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-07 22:44:12 +02:00
Jakub Sztandera 835b7eb6f7 Make allowNegativeChains a function
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-07 22:44:12 +02:00
vyzo 172ff846fd decouple negative chain selection from base fee lower bound validation
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-07 22:44:12 +02:00
vyzo 97fddc45d9 temporarily allow negative perfoming chains
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-07 22:44:09 +02:00
Jakub Sztandera 8a018ce3ba Merge pull request #3623 from filecoin-project/feat/base-fee-tamping
introduce logic to perform base fee tamping around the upgrade time
2020-09-07 22:43:34 +02:00
Jakub Sztandera baaec9db87 Fix testground build
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-07 22:31:39 +02:00
whyrusleeping 5d5c964d06 set correct upgrade height
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-07 22:31:39 +02:00
whyrusleeping 502d3fb335 set values for test networks
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-07 22:31:39 +02:00
whyrusleeping 5654322f54 introduce logic to perform base fee tamping around the upgrade time
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-07 22:31:36 +02:00
Łukasz Magiera bea31fdcb0 Merge pull request #3603 from filecoin-project/asr/specs-for-0.6.0
Update to specs actors v0.9.6
2020-09-07 21:54:36 +02:00
Aayush Rajasekaran 529002c2b7 Update to markets master 2020-09-07 15:48:42 -04:00
Aayush Rajasekaran 4367d74102 Update statediff and test-vectors 2020-09-07 15:48:42 -04:00
hannahhoward 04e34b8ca7 docs(apidocs): run docsgen 2020-09-07 15:48:42 -04:00
hannahhoward 5ad8e23eae feat(markets): use new api 2020-09-07 15:48:42 -04:00
hannahhoward 2db4b57013 feat(markets): upgrade markets 0.6.0 2020-09-07 15:48:42 -04:00
Aayush Rajasekaran d678fe4bfa Fix tests 2020-09-07 15:48:42 -04:00
Aayush Rajasekaran be7fb9d3df Use custom markets 2020-09-07 15:48:42 -04:00
Aayush Rajasekaran 39755a294a Update to specs v0.9.6 2020-09-07 15:48:41 -04:00
anorth a8dcea1422 Remove use of exitcode.SysErrInvalidParameters 2020-09-07 15:48:41 -04:00
Łukasz Magiera 2080458f0d Merge pull request #3518 from filecoin-project/feat/pubsub-red
integrate pubsub Random Early Drop
2020-09-07 21:42:14 +02:00
Jakub Sztandera 3398426293 Merge pull request #3592 from filecoin-project/feat/improved-publish-logic
Improve publish/republish logic
2020-09-07 21:32:52 +02:00
Raúl Kripalani 878ffafb51 rename other Chain{Exchange=>Bitswap}; fix ChainBlock{s=>S}ervice(). 2020-09-07 20:20:23 +01:00
vyzo 2233751668 use the baseFeeLowerBound for computing repub message chains 2020-09-07 21:53:30 +03:00
Łukasz Magiera b86831356b Merge pull request #3622 from filecoin-project/revert-3615-fix/stormgr-undef-unsealed
Revert "storage manager: bail out with an error if unsealed cid is undefined"
2020-09-07 20:50:04 +02:00
Raúl Kripalani 453e826a0f rename p2p protocol to /fil/chain/xchg/0.0.1 (backwards-compatible); rename more. 2020-09-07 19:45:34 +01:00
Raúl Kripalani 55b1456d45 blocksync: introduce interfaces; rename to chainexchange. 2020-09-07 19:31:43 +01:00
Łukasz Magiera 47c59afea0 Revert "storage manager: bail out with an error if unsealed cid is undefined" 2020-09-07 20:12:29 +02:00
vyzo 41222792ed add docstring for verifyMsgBeforeAdd 2020-09-07 20:26:51 +03:00
vyzo 5659faf7f0 don't immediately publish messages that cannot be included in the next 20 blocks 2020-09-07 20:20:49 +03:00
vyzo 2e75d9c80a be explicit about republish interval, check against timecache duration 2020-09-07 20:20:49 +03:00
vyzo 0fcf8838cc improve republish logic to only republish messages that can be included in the next 20 blocks 2020-09-07 20:20:49 +03:00
Łukasz Magiera 758aae8556 Merge pull request #3586 from filecoin-project/feat/expected-height-metric
metrics: add expected height metric
2020-09-07 19:09:06 +02:00
Łukasz Magiera ffbbd968c2 Merge pull request #3602 from filecoin-project/feat/delayed-pubsub-join
only subscribe to pubsub topics once we are synced
2020-09-07 19:08:21 +02:00
vyzo 0d0dd250ee use faster decay for bootstrappers 2020-09-07 19:54:38 +03:00
vyzo e3aaedbc33 fix tests 2020-09-07 19:46:58 +03:00
vyzo 81e674ad36 subscribe early in bootstrappers and get rid of ugly test flag 2020-09-07 19:46:35 +03:00
Łukasz Magiera f6f5405a2c Fix lint 2020-09-07 18:21:45 +02:00
Łukasz Magiera 551ff6a6fb Merge pull request #3608 from filecoin-project/fix/second-argument-error-message
fix: second argument error message
2020-09-07 18:06:22 +02:00
Łukasz Magiera d3e5092b43 Merge pull request #3611 from filecoin-project/fix/tscache-nil-best
chain events: if cache best() is nil, return chain head
2020-09-07 18:05:34 +02:00
Łukasz Magiera 3fe8cdece3 Merge pull request #3615 from filecoin-project/fix/stormgr-undef-unsealed
storage manager: bail out with an error if unsealed cid is undefined
2020-09-07 18:02:44 +02:00
Łukasz Magiera 69f203a718 Merge pull request #3606 from filecoin-project/rm-chain-validation
decommission chain-validation.
2020-09-07 18:00:08 +02:00
Dirk McCormick a97f978cad fix: storage manager - dont fail on successful read piece 2020-09-07 16:14:19 +02:00
Dirk McCormick 07a4553e6e fix: storage manager - bail out with an error if unsealed cid is undefined 2020-09-07 16:04:12 +02:00
vyzo e33798ff90 rename variable for better semantics 2020-09-07 15:58:05 +03:00
vyzo d9faff19a5 fix tests 2020-09-07 15:55:53 +03:00
vyzo 4f0ab4f226 remove notifee after subscribing 2020-09-07 15:55:53 +03:00
vyzo 05740e0f64 silence linter for unnecessary conversion 2020-09-07 15:55:53 +03:00
vyzo 666dc65b8e only subscribed to pubsub topics once we are synced 2020-09-07 15:55:53 +03:00
Dirk McCormick 76a1b3286b fix: if cache best is nil, return chain head 2020-09-07 14:43:06 +02:00
Alan Shaw 9d9d2a2a98 fix: second argument error message
Second argument should be a pointer, not runtime.
2020-09-07 12:29:40 +01:00
vyzo 219c8b927d go mod tidy 2020-09-07 13:46:23 +03:00
vyzo c616958557 specify topic delivery weights, with priority for blocks 2020-09-07 13:40:48 +03:00
vyzo 453ab95e54 enable pubsub RED 2020-09-07 13:40:48 +03:00
vyzo 7fdab46729 update go-libp2p-pubsub@master 2020-09-07 13:40:42 +03:00
Raúl Kripalani 798061506e decommission chain validation. 2020-09-07 11:34:36 +01:00
Aayush Rajasekaran 0ad0d4ea11 Merge pull request #3579 from filecoin-project/fix/mpool-basefee-check
add defensive check for minimum GasFeeCap for inclusion within the next 20 blocks
2020-09-07 03:03:35 -04:00
Travis Person f7b52d16f5 metrics: add expected height metric 2020-09-07 02:57:12 +00:00
vyzo e24a29b146 fix tests that touch the mpool; mock block must have a ParentBaseFee 2020-09-06 11:55:43 +03:00
vyzo ffb2640736 use faster lookup for base fee 2020-09-06 11:55:43 +03:00
vyzo f9492691a6 don't check baseFee lower bound for local messages 2020-09-06 11:55:43 +03:00
vyzo 1c0e6d76f0 add defensive check for minimum GasFeeCap for inclusion within the next 20 blocks 2020-09-06 11:55:43 +03:00
Aayush Rajasekaran 88c04ce959 Update docs 2020-09-06 03:51:06 -04:00
Aayush Rajasekaran 5ead23a8d4 Msig: Add CLI commands to propose, approve, or cancel adding signers 2020-09-06 03:46:07 -04:00
Aayush Rajasekaran d0ccb54aba Msig: Add RPC endpoints to propose, approve, or cancel adding signers 2020-09-06 01:52:30 -04:00
Aayush Rajasekaran aaad01105e Msig: Introduce an API & CLI to calculate amount that vests between 2 tipsets 2020-09-05 23:42:31 -04:00
Łukasz Magiera 6bdd433570 Merge pull request #3538 from filecoin-project/steb/safe-exports
Robustify state manager against holes in actor method numbers
2020-09-05 21:58:03 +02:00
Łukasz Magiera c767e5ebed Merge pull request #3547 from filecoin-project/feat/paych-status-addr
paych: get available funds by address or by from/to
2020-09-05 21:55:08 +02:00
Łukasz Magiera 4718efab9b docsgen 2020-09-05 21:51:30 +02:00
Łukasz Magiera 145493388a Merge pull request #3553 from filecoin-project/fix/panic-in-expired-slashed
Fix panic in OnDealExpiredSlashed
2020-09-05 21:49:54 +02:00
Łukasz Magiera 16bd0d134e Merge pull request #3556 from filecoin-project/fix/pubsub-reduce-gossipfactor
adjust gossipsub gossip factor
2020-09-05 21:49:32 +02:00
Łukasz Magiera f58cbc2594 Merge pull request #3557 from filecoin-project/fix/mpool-repub-negative
republish messages even if the chains have negative performance
2020-09-05 21:48:39 +02:00
Łukasz Magiera 1e190e462a Merge pull request #3562 from filecoin-project/fix/mpool-log-spam
reduce mpool add failure log spam
2020-09-05 21:47:29 +02:00
Łukasz Magiera 70da2b7b4d Merge pull request #3568 from filecoin-project/feat/lotus-shed-math
lotus-shed: add math command
2020-09-05 21:47:06 +02:00
Łukasz Magiera 2297c58ab6 Merge pull request #3578 from filecoin-project/misc/debug-info-premium
Add additional info about gas premium
2020-09-05 21:45:57 +02:00
Łukasz Magiera 5e97e29f95 Merge pull request #3580 from filecoin-project/fix/mpool-selection-bug
fix selection bug; priority messages were not included if other chains were negative
2020-09-05 21:44:03 +02:00
vyzo 5037282b98 add test for priority selection with negative chains 2020-09-05 22:15:18 +03:00
vyzo 0b3685e529 fix selection bug; priority messages were not included if other's chains were negative 2020-09-05 21:30:02 +03:00
Jakub Sztandera 2adb80729b Add additional info about gas premium
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-05 15:17:55 +02:00
Łukasz Magiera c074c8d0ca Merge pull request #3570 from filecoin-project/feat/lotus-pcr-current-tipset
lotus-pcr: use current tipset during refund
2020-09-05 05:29:59 +02:00
Łukasz Magiera 2b62573b65 Merge pull request #3571 from shepf/master
Batch replacement,update lotus-storage-miner to lotus-miner.
2020-09-05 05:29:28 +02:00
shepf 16fd12024e Batch replacement,update lotus-storage-miner to lotus-miner. 2020-09-05 10:40:53 +08:00
Travis Person bf3466f3cb lotus-pcr: use current tipset during refund
Using the tipset which is being processed might lead to incorrect gas
estimations or balance checking for the wallet.
2020-09-05 02:38:24 +00:00
Travis Person eebd2efcbf lotus-shed: add math command 2020-09-05 00:50:56 +00:00
Jakub Sztandera c573310c77 Merge pull request #3565 from filecoin-project/feat/lotus-pcr-aggregate-tipsets
lotus-pcr: add tipset aggregation
2020-09-05 00:43:27 +02:00
Travis Person d6691fe9d8 lotus-pcr: add tipset aggregation 2020-09-04 22:28:13 +00:00
vyzo 275f071b12 reduce mpool add failure log spam 2020-09-05 00:28:13 +03:00
vyzo ad8d550d73 remove unused ts argument from Trim, replace it with allowNegative specifier 2020-09-04 22:32:30 +03:00
Steven Allen fceeaf42ca Robustify state manager against holes in actor method numbers
Also, don't simply assume that the field order matches the method numbers in
`builtin.Method*` structs.
2020-09-04 12:26:55 -07:00
Steven Allen 30fdff17b5 Skip nil exports
The actors may skip previously used but no longer defined method numbers.
2020-09-04 12:26:55 -07:00
vyzo 6d53862142 republish messages even if the chains have negative performance 2020-09-04 22:16:10 +03:00
vyzo 1c27a0c1cf adjust gossipsub gossip factor 2020-09-04 22:04:48 +03:00
hannahhoward 5ef753bfa8 fix(markets): check for nil tipset
check for nil tipset when check func is called to avoid panic
2020-09-04 10:04:50 -07:00
Łukasz Magiera 52ce80f2b4 Merge pull request #3551 from filecoin-project/fix/paych-cli-tests
paych: fix broken paych tests
2020-09-04 18:32:26 +02:00
Jakub Sztandera d8a790a13c Merge pull request #3552 from filecoin-project/fix/fee-cap-premium
Fix GasPremium capping logic
2020-09-04 18:29:33 +02:00
Jakub Sztandera 20801263ec Fix paych tests
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-04 18:13:09 +02:00
Jakub Sztandera 4cf52ea025 Add 0.1FIL cap too DefaultMessageSendSpec
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-04 15:52:50 +02:00
Dirk McCormick de132d39ec fix: broken paych tests 2020-09-04 15:48:13 +02:00
Jakub Sztandera 114c0ea85c Fix GasPremium capping logic
Cap it just to FeeCap, there is no reason to cap it proportionally

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-04 15:42:12 +02:00
Dirk McCormick 51ef39592f feat: paych - get available funds by address or by from/to 2020-09-04 13:44:09 +02:00
Łukasz Magiera eb3ca82fa0 net cli: Support miner addresses in net connect 2020-09-04 13:37:42 +02:00
Łukasz Magiera a081dc7674 Merge pull request #3520 from filecoin-project/feat/paych-cli-add-funds
paych: rename CLI command from "paych get" to "paych add-funds"
2020-09-04 04:21:54 +02:00
Łukasz Magiera 924fd82d7f Merge pull request #3523 from filecoin-project/feat/paych-cli-status
paych: Create `lotus paych status` command
2020-09-04 04:20:45 +02:00
Łukasz Magiera 2177fd59ef Merge pull request #3521 from filecoin-project/conformance/tipset-actor-address
conformance: various changes
2020-09-04 04:18:41 +02:00
Łukasz Magiera d81feb05d1 Merge pull request #3527 from filecoin-project/feat/watch-option-miner-storage-deals-list
Add watch option to storage-deals list
2020-09-04 04:13:01 +02:00
Łukasz Magiera 0a620514b7 Merge pull request #3526 from filecoin-project/fix/chainwatch/no-panic-during-processing
Stop SyncIncomingBlocks from leaking into chainwatch processing; No panics during processing
2020-09-04 03:55:59 +02:00
Łukasz Magiera e52232d85f Merge pull request #3535 from filecoin-project/steb/robust-method-lookup
[cli/state] Robust actor lookup
2020-09-04 03:52:43 +02:00
Łukasz Magiera 4cdd81ec86 Merge pull request #3534 from filecoin-project/feat/netpeers-agent-version
Add agent flag to net peers
2020-09-04 02:50:04 +02:00
Steven Allen 6e711ed739 [cli/state] Robust actor name lookup 2020-09-03 17:22:18 -07:00
Steven Allen 8778787091 [cli/state] Robust actor lookup
This probably isn't critical, but I noticed it, so I fixed it.
2020-09-03 17:19:05 -07:00
Łukasz Magiera ce8bcf7932 Add agent flag to net peers 2020-09-04 02:00:53 +02:00
Mike Greenberg abaef98fd8 fix(chainwatch): Stop SyncIncomingBlocks from leaking into processing
The SQL query was anchoring data from the `blocks` table, which includes
all blocks seen from SyncIncomingBlocks which isn't always available in
the chainstate via the API. In order to prevent these blocks from
leaking into normal processing (which errors anyway), the join was
changed to allow `blocks_synced` to anchor the set as we originally
intended.
2020-09-03 19:15:25 -04:00
Mike Greenberg ddc5e57d4d fix(chainwatch): Error instead of panic during processing 2020-09-03 19:15:19 -04:00
Raúl Kripalani c723554d2e upgrade test-vectors deps. 2020-09-03 23:44:21 +01:00
Whyrusleeping 021f4a881c Merge pull request #3448 from filecoin-project/feat/paych-get-avail-funds
paych: Add PaychAvailableFunds API method
2020-09-03 15:26:25 -07:00
Alan Shaw ada5e6ae68 refactor: remove puppet actor from conformance tests
The puppet actor has been subsumed into the chaos actor and test-vector tests no longer use it.
2020-09-03 23:19:53 +01:00
Raúl Kripalani 8e7a8d8c97 conformance: tipset-class driver: allow actor msgs, dummy-sign secp msgs. 2020-09-03 22:54:19 +01:00
Łukasz Magiera 9151f8b492 Merge pull request #3533 from filecoin-project/feat/fast-export
Make chain export ~1000x times faster
2020-09-03 22:47:20 +02:00
Łukasz Magiera 86cb865fdd Make chain export ~1000x times faster 2020-09-03 21:56:35 +02:00
Aayush Rajasekaran 38cb2601d7 Merge pull request #3531 from filecoin-project/asr/changelog
Lotus version 0.5.10
2020-09-03 15:03:00 -04:00
Aayush Rajasekaran 94173c273a Lotus version 0.9.10 2020-09-03 14:57:58 -04:00
Łukasz Magiera 01449e486c Merge pull request #3528 from filecoin-project/fix/sele-gas-reward
Fix calculation of GasReward in messagepool
2020-09-03 20:41:28 +02:00
Jakub Sztandera 8111b227c4 Appease the linter
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-03 20:19:01 +02:00
Jakub Sztandera 8bb471bccd Add getGasPerf test
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-03 20:19:01 +02:00
Jakub Sztandera 5074b526d0 Fix calculation of GasReward in messagepool
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-03 19:36:03 +02:00
Ingar Shu 1608cd2d53 Add watch option to "lotus-miner storage-deals list" 2020-09-03 10:14:30 -07:00
Łukasz Magiera a123ef6fce Merge pull request #3525 from filecoin-project/asr/changelog
Lotus version v0.5.9
2020-09-03 19:11:43 +02:00
Aayush Rajasekaran 23ec780a13 Lotus version v0.5.9 2020-09-03 12:48:20 -04:00
Aayush Rajasekaran 5569f96885 Merge pull request #3450 from filecoin-project/feat/mpool-nonce-gaps
track expected nonce in mpool, ignore messages with large nonce gaps
2020-09-03 12:27:32 -04:00
Dirk McCormick 930be3d0e5 feat: paych CLI - lotus paych status <from> <to> 2020-09-03 18:11:11 +02:00
Dirk McCormick f84b1d0de9 feat: paych - rename CLI command from "paych get" to "paych add-funds" 2020-09-03 15:45:53 +02:00
Dirk McCormick 8423325a6e refactor: paych - use channel accessor from/to instead of passing them around as params 2020-09-03 15:31:49 +02:00
Łukasz Magiera 97273655dc Merge pull request #3502 from filecoin-project/area/ux-3388-sync-wait-target-height
Added target height to sync wait
2020-09-03 14:49:30 +02:00
Łukasz Magiera 10acc04ba5 Merge pull request #3497 from filecoin-project/feat/bandwidth-tracking
add apis and command to inspect bandwidth usage
2020-09-03 14:37:54 +02:00
vyzo d20b6adfd2 refuse to add duplicates in the mpool 2020-09-03 15:28:44 +03:00
vyzo a4c650316a fix test 2020-09-03 15:28:44 +03:00
vyzo 28f57667f0 cap MaxNonceGap to 4, add delay between batch messages during republish 2020-09-03 15:28:44 +03:00
vyzo f53d2e3a46 cap MaxNonceGap to 16 2020-09-03 15:28:44 +03:00
vyzo 7b76aa2078 warn when adding a nonce-gapped message to the mpool 2020-09-03 15:28:44 +03:00
vyzo d76a3b87c5 ignore messages with large nonce gaps 2020-09-03 15:28:44 +03:00
vyzo da6d384300 fix tests 2020-09-03 15:28:44 +03:00
vyzo 4c489d18a2 track expected nonce in mpool, refuse messages with large gaps 2020-09-03 15:28:44 +03:00
Łukasz Magiera 9785f14df8 Merge pull request #3516 from filecoin-project/feat/cap-fee
Cap fees to reasonable level by default
2020-09-03 14:10:58 +02:00
Jakub Sztandera d35b273dbf Cap fees to reasonable level by default
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-03 13:49:50 +02:00
Jakub Sztandera 13c828f112 Merge pull request #3514 from filecoin-project/ci/disable-codecov-annotations
Disable codecov annotations
2020-09-03 13:47:33 +02:00
Łukasz Magiera ea6ee8e899 Disable codecov annotations 2020-09-03 13:39:16 +02:00
Dirk McCormick 1d3a21f6c8 docs: paych - correct comments on store fields 2020-09-03 08:22:40 +02:00
Dirk McCormick 3daa0b5e63 fix: paych - clarify behaviour of current available funds for non-existent channel 2020-09-03 08:20:08 +02:00
Aayush Rajasekaran 5426ef0b46 Merge pull request #3501 from filecoin-project/asr/changelog
Lotus version 0.5.8
2020-09-02 20:41:22 -04:00
jennijuju 860a6a231d 3388: Added target height to sync wait 2020-09-02 19:48:24 -04:00
whyrusleeping da7fc8dbb3 further appeasement 2020-09-02 16:42:55 -07:00
Aayush Rajasekaran f03f4775fc Lotus version 0.5.8 2020-09-02 19:42:19 -04:00
whyrusleeping 0e39d62c9d Appease the CI overlords 2020-09-02 16:31:41 -07:00
whyrusleeping d196e4a8df docs, mod tidy 2020-09-02 15:45:57 -07:00
whyrusleeping a59dadccc9 add apis and command to inspect bandwidth usage 2020-09-02 15:25:53 -07:00
Łukasz Magiera 05db79ee35 Merge pull request #3496 from filecoin-project/feat/update-markets-0.5.9
Upgrade markets to v0.5.9
2020-09-02 23:47:18 +02:00
Łukasz Magiera d552803f56 Merge pull request #3494 from filecoin-project/feat/faster-blocksync-serving
load fewer messages from disk when serving blocksync requests
2020-09-02 22:43:12 +02:00
hannahhoward d2a10413a1 feat(markets): upgrade to v0.5.9 2020-09-02 13:19:39 -07:00
Lucas MolasandŁukasz Magiera ca8f5861fe Update chain/blocksync/server.go
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2020-09-02 17:07:39 -03:00
Łukasz Magiera 7536744dcf Merge pull request #3495 from filecoin-project/revert-3464-feat/blst-adx
Revert "adx blst"
2020-09-02 21:16:05 +02:00
Łukasz Magiera 8ba0385b3a Revert "adx blst" 2020-09-02 21:05:38 +02:00
whyrusleeping 9a5542100a load fewer messages from disk when serving blocksync requests 2020-09-02 11:43:05 -07:00
Łukasz Magiera b774563ec3 Merge pull request #3492 from filecoin-project/fix/readpiece-panic
ffiwrapper: Fix ReadPiece panic
2020-09-02 18:59:22 +02:00
Łukasz Magiera ca7aa69597 ffiwrapper: More correct error check on openPartialFile 2020-09-02 18:45:07 +02:00
Łukasz Magiera 5aeabe39b5 Merge pull request #3456 from filecoin-project/misc/account-for-premium
Account for GasPremium in GasEstimateFeeCap
2020-09-02 18:20:32 +02:00
Łukasz Magiera 5f79ff340d Merge pull request #3489 from filecoin-project/fix/sched-deadlocks
sealing sched: Fix deadlock between worker.wndLk / workersLk
2020-09-02 18:03:18 +02:00
Łukasz Magiera 5a2b439773 sched: Fix tests 2020-09-02 17:37:19 +02:00
Łukasz Magiera cfbbcd420f Merge pull request #3463 from filecoin-project/feat/chain-state-export
allow exporting a number of recent chain state trees
2020-09-02 17:21:14 +02:00
Jakub Sztandera 59f765f7be Rename to import-snapshot, require more than finality for state export
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-02 17:15:36 +02:00
Łukasz Magiera 528e39fcb3 docsgen, fix snapshot flag name 2020-09-02 17:15:10 +02:00
whyrusleeping 2b16e69e90 allow snapshot importing 2020-09-02 17:15:10 +02:00
whyrusleeping 09ebd1a557 allow exporting a number of recent chain state trees 2020-09-02 17:15:10 +02:00
Łukasz Magiera edc2a28f1d Merge pull request #3485 from filecoin-project/tipset-vector
tipset-class test vector driver (take 2)
2020-09-02 17:08:26 +02:00
Łukasz Magiera 7fe8580da5 sealing sched: Fix deadlock between worker.wndLk / workersLk 2020-09-02 17:06:48 +02:00
Raúl Kripalani 7cc9c83ab8 upgrade test-vectors. 2020-09-02 14:12:27 +01:00
Raúl Kripalani 0962292f92 conformance: add driver for tipset-class vectors. 2020-09-02 14:09:12 +01:00
Łukasz Magiera b7f2487c4f Merge pull request #3457 from filecoin-project/fix/disable-badger-gc
Disable GC on chain badger datastore
2020-09-02 15:02:52 +02:00
Łukasz Magiera db86647e3a Merge pull request #3458 from austinabell/rand/rmheight
Remove height from chain rand
2020-09-02 14:59:29 +02:00
Łukasz Magiera 7415627d95 Merge pull request #3483 from filecoin-project/revert-3449-tipset-vector
Revert "conformance: add driver for tipset-class vectors."
2020-09-02 14:51:11 +02:00
Łukasz Magiera fa6bae988e Revert "conformance: add driver for tipset-class vectors." 2020-09-02 14:49:55 +02:00
Dirk McCormick 2c98bf0cc7 feat: PaychAvailableFunds API method 2020-09-02 14:31:32 +02:00
Łukasz Magiera 351313523c Merge pull request #3470 from filecoin-project/feat/toggle-message-processing
lotus-pcr: add toggle for message processing
2020-09-02 14:26:40 +02:00
Łukasz Magiera 80ecbeddbb Merge pull request #3464 from filecoin-project/feat/blst-adx
adx blst
2020-09-02 14:25:00 +02:00
Łukasz Magiera 8222f803d5 Merge pull request #3476 from filecoin-project/asr/advance-block
Check error in advance-block
2020-09-02 13:46:34 +02:00
Łukasz Magiera d5b33671c2 Merge pull request #3481 from filecoin-project/misc/consensus-label
Add Impact Consensus label
2020-09-02 13:43:23 +02:00
Łukasz Magiera e86749dfe2 Merge pull request #3449 from filecoin-project/tipset-vector
conformance: add driver for tipset-class vectors.
2020-09-02 13:42:42 +02:00
Raúl Kripalani ec5903ca19 fix lint error. 2020-09-02 12:28:55 +01:00
Jakub Sztandera b5dcb5964a Add Impact Consensus label 2020-09-02 13:07:56 +02:00
Aayush Rajasekaran 8417f516e5 Check error in advance-block 2020-09-02 02:41:12 -04:00
Travis Person 9532441ed4 lotus-pcr: add toggle for message processing 2020-09-02 03:50:16 +00:00
Travis Person 630dfb9b55 adx blst 2020-09-02 01:04:56 +00:00
Łukasz Magiera f31dc791e4 Merge pull request #3455 from filecoin-project/deps/update-pubsub
update go-libp2p-pubsub@master
2020-09-01 22:13:25 +02:00
austinabell ba881eb4cf Remove height from chain rand 2020-09-01 15:48:16 -04:00
Raúl Kripalani ad5793e446 update test-vectors commit. 2020-09-01 20:01:41 +01:00
Jakub Sztandera 997d49b63f Disable GC on chain badger datastore
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-01 20:39:58 +02:00
Jakub Sztandera 72fb6afae2 Account for GasPremium in GasEstimateFeeCap
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-09-01 20:35:29 +02:00
vyzo f9df806f15 update go-libp2p-pubsub@master 2020-09-01 20:46:12 +03:00
Raúl Kripalani 01737ef1cc upgrade test-vectors deps. 2020-09-01 17:00:18 +01:00
Raúl Kripalani 3321f36b51 update test-vectors commit. 2020-09-01 16:35:26 +01:00
Raúl Kripalani 64bdd620ac conformance: add driver for tipset-class vectors. 2020-09-01 15:35:19 +01:00
Star.LI 82f5984de1 fix crash - segment fault when partialFile.Allocated() is invoked.
When openPartialFile is invoked, more errors than "existed error" are
returned. If only existing error is checked, the allocated field of
partialFile may be nil.

Signed-off-by: Star.LI <star@trapdoortech.com>
2020-09-01 11:41:26 +02:00
Łukasz Magiera b8bbbf3ea3 Merge pull request #3436 from mitchwagner/patch-2
fix: referenced filename
2020-09-01 11:09:26 +02:00
Łukasz Magiera 247b5205fb Merge pull request #3442 from filecoin-project/fix/chainwatch/logging-sync-blocks
Some more Chainwatch improvements
2020-09-01 10:57:55 +02:00
Mike Greenberg 2b83089da8 fix(chainwatch): Add important documentation 2020-08-31 22:55:11 -04:00
Mike Greenberg 7290431cb0 fix(chainwatch): Restore IncomingBlockSync listening 2020-08-31 22:46:28 -04:00
Mike Greenberg 95a75daf9f fix(chainwatch): Remove useless logging 2020-08-31 22:32:34 -04:00
Mike Greenberg 80ed6b8c25 fix(chainwatch): Process chain waitout waiting for update via ChainNotify 2020-08-31 22:31:25 -04:00
Aayush Rajasekaran 49d64f7f7e Merge pull request #3438 from filecoin-project/asr/changelog
Lotus version 0.5.7
2020-08-31 20:04:19 -04:00
Aayush Rajasekaran f09f38085e Lotus version 0.5.7 2020-08-31 20:03:38 -04:00
Aayush Rajasekaran 04a9822573 Merge pull request #3428 from filecoin-project/fix/estim-gas-price-one-msg
Fix GasEstimateGasPremium when there is only one message on chain
2020-08-31 20:01:02 -04:00
Mitch Wagner 5848877734 fix: referenced filename 2020-08-31 19:29:43 -04:00
Aayush Rajasekaran 93814cc85a Merge pull request #3392 from filecoin-project/fix/mpool-priority-selection
Tweak priority selection to not select negative perfoming chains
2020-08-31 19:22:12 -04:00
Aayush Rajasekaran 4f30311971 Merge pull request #3430 from filecoin-project/misc/bitswap-close-ctx
Don't use context to close bitswap
2020-08-31 19:21:06 -04:00
Aayush Rajasekaran cfe5134375 Use Sign() to check zero-equality 2020-08-31 18:53:14 -04:00
Łukasz Magiera 219d3c9f18 Merge pull request #3425 from filecoin-project/feat/better-worker-selection
Improve worker selection a bit
2020-08-31 23:11:25 +02:00
Łukasz Magiera 9110776a01 Merge pull request #3424 from filecoin-project/fix/move-unsealed
storage: Correcty move unsealed sectors in FinalizeSector
2020-08-31 23:09:36 +02:00
Jakub Sztandera 625c6951bb Don't use context to close bitswap
Use just the Close

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-08-31 19:24:04 +02:00
Jakub Sztandera 56235aee90 Fix GasEstimateGasPremium when there is only one message on chain
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-08-31 18:12:06 +02:00
Łukasz Magiera e14c80360d sealing sched: Factor worker queues into utilization calc 2020-08-31 13:41:34 +02:00
Łukasz Magiera 98d51d3d80 storage: Correcty move unsealed sectors in FinalizeSector 2020-08-31 12:45:57 +02:00
Łukasz Magiera 4f45c623a5 Merge pull request #3401 from filecoin-project/asr/unsealed-infos
Rebuild unsealed infos on miner restart
2020-08-31 10:35:33 +02:00
Łukasz Magiera c96613a499 Merge pull request #3405 from filecoin-project/feat/worker-storage-cli
worker: Cli to attach storage paths
2020-08-31 10:09:20 +02:00
Łukasz Magiera e1e01524e7 Merge pull request #3421 from IPFSMain-Official/master
remove the unnecessary err check
2020-08-31 10:08:36 +02:00
刘林欣 0926f95979 remove the unnecessary err check 2020-08-31 15:11:45 +08:00
David Dias a4decaa188 docs: add label for markets 2020-08-31 08:04:37 +01:00
Aayush Rajasekaran 27f7d3a658 Rebuild unsealed infos on miner restart 2020-08-30 18:01:07 -04:00
Łukasz Magiera 0e6ff668eb worker: Cli to attach storage paths 2020-08-30 20:33:02 +02:00
vyzo fba3ed7b4d update documentation about priority addresses 2020-08-29 15:56:31 +03:00
vyzo 648130eec5 don't include negative performing chains in priority message selection 2020-08-29 15:55:19 +03:00
Aayush Rajasekaran 606a58bc6b Lotus version 0.5.6 2020-08-29 00:56:24 -04:00
Aayush Rajasekaran 09a645e455 Merge pull request #3387 from filecoin-project/asr/import-fix
Fix chain import
2020-08-29 00:55:54 -04:00
Łukasz Magiera d972a47c84 Bump version to 0.5.6 2020-08-29 06:48:28 +02:00
Łukasz Magiera 494c774303 Merge pull request #3389 from filecoin-project/fix/sched-compact-panic
sched: Fix panic in workerCompactWindows
2020-08-29 06:47:12 +02:00
Łukasz Magiera 28ac2fce61 sched: Fix panic in workerCompactWindows 2020-08-29 06:41:19 +02:00
Aayush Rajasekaran 5095e6f07b Fix chain import 2020-08-28 22:15:43 -04:00
Łukasz Magiera 9a3b2abbab Merge pull request #3383 from filecoin-project/asr/changelog
Lotus version 0.5.5
2020-08-29 03:52:00 +02:00
Aayush Rajasekaran 4e5b11b1d5 Lotus version 0.5.5 2020-08-28 21:49:56 -04:00
Łukasz Magiera 7eb2710e67 Merge pull request #3384 from filecoin-project/feat/markets-update-0.5.8
Update to markets v0.5.8
2020-08-29 03:46:38 +02:00
hannahhoward 0a36db06ed feat(markets): update to markets v0.5.8 2020-08-28 18:03:27 -07:00
Łukasz Magiera e86f90fe8d Merge pull request #3380 from filecoin-project/fix/chainwatch/perf
perf(chainwatch): parallelize miner processing
2020-08-29 02:51:46 +02:00
Łukasz Magiera 953effcc43 Merge pull request #3350 from filecoin-project/fix/some-fsm-issues
Fix some fsm issues
2020-08-29 02:16:59 +02:00
Łukasz Magiera a62e44c4e5 Fix lint 2020-08-29 02:13:38 +02:00
Łukasz Magiera 84a632787c Merge pull request #3313 from filecoin-project/feat/mpool-balance-tracking
Message Pool Rudimentary Spam Protection Measures
2020-08-28 22:35:30 +02:00
Łukasz Magiera 16f0daab90 Merge pull request #3379 from filecoin-project/feat/add-list-deals-watch-option
Add watch option to list-deals
2020-08-28 22:33:48 +02:00
Łukasz Magiera 3149390b3b Merge pull request #3381 from filecoin-project/fix/dynamic-chain-sync-window
dynamic chain sync window
2020-08-28 22:31:21 +02:00
frrist 0185090cbb perf(chainwatch): parallelize miner processing 2020-08-28 13:26:11 -07:00
vyzo 806f7dd15f windowSize should not exceed blocksync.MaxRequestLength 2020-08-28 23:18:06 +03:00
Łukasz Magiera 013ebc617e docsgen 2020-08-28 22:17:35 +02:00
vyzo 63a60cdd75 be more conservative in increasing the window size, check that we actually had more messages 2020-08-28 23:09:48 +03:00
Łukasz Magiera 4bcc71c68f Merge pull request #3197 from filecoin-project/fix/paych-spendable-submit-voucher
paych: fix issue where `voucher spendable` fails for submitted vouchers
2020-08-28 22:05:04 +02:00
Łukasz Magiera 7f1374e058 docsgen 2020-08-28 22:03:09 +02:00
vyzo 3935fd77ac remember syncer windowSize 2020-08-28 22:52:40 +03:00
Łukasz Magiera 9d0c8ae3dd sectorstorage: update sched tests for new logic 2020-08-28 21:38:21 +02:00
vyzo 4e7fd5b6d7 sync: increase window size when successful fetching messages 2020-08-28 22:03:06 +03:00
Ingar Shu a4ded8c2b9 Add watch option to list-deals 2020-08-28 10:40:27 -07:00
Łukasz Magiera 4a75e1e4b4 sectorstorage: Don't require tasks within a window to run in order 2020-08-28 19:38:55 +02:00
Łukasz Magiera 11b11e416b sectorstorage: Compact assigned windows 2020-08-28 18:26:38 +02:00
vyzo 1f660c9e04 sync: half window size and retry when fetching messages fails 2020-08-28 18:45:20 +03:00
Łukasz Magiera 20ffce5c0d Merge remote-tracking branch 'origin/master' into fix/some-fsm-issues 2020-08-28 16:47:25 +02:00
Łukasz Magiera a7ef61279c Merge pull request #3374 from IPFSMain-Official/fix-packing-sector
ignore resend SectorStartPacking event. and fix StartPacking's unsealedInfoMap lock
2020-08-28 16:37:16 +02:00
Łukasz Magiera 5ee85dc263 sectorstorage: Fix tests 2020-08-28 16:33:41 +02:00
刘林欣 93dc33687a ignore resend SectorStartPacking event, and unsealedInfoMap lock 2020-08-28 19:04:23 +08:00
Łukasz Magiera 932ab61c2d Merge pull request #3275 from filecoin-project/fix/chainwatch/sync
fix(chainwatch): use height to determine unsynced blocks and fix deadlock in sector deal table
2020-08-28 12:16:00 +02:00
Łukasz Magiera 1d9b1f4aab Merge pull request #3311 from filecoin-project/daviddias-patch-1
docs: removed non used packages from the list
2020-08-28 12:08:20 +02:00
Łukasz Magiera 7aed16ce4c Merge pull request #3372 from filecoin-project/fix/distinguish-local-msg-errors
distinguish local message validation failures from remote
2020-08-28 12:07:52 +02:00
Łukasz Magiera 9f4506bda5 metrics: gofmt 2020-08-28 11:51:51 +02:00
Łukasz Magiera fa6f9f4ae0 Merge pull request #3367 from filecoin-project/feat/chainwatch/add-power-reward-metrics
feat(chainwatch): Add more complete power and reward metrics
2020-08-28 11:49:22 +02:00
Łukasz Magiera 78ff5e270c Merge pull request #3369 from filecoin-project/asr/advance-block
Use beacon entries in advance-block
2020-08-28 11:46:34 +02:00
Łukasz Magiera 6d1682a27e storagefsm: wire up RecoverDealIDs fully 2020-08-28 11:44:15 +02:00
lanzafame 64768f093d rename local recordFailure func 2020-08-28 17:15:11 +10:00
lanzafame a24db6c584 refactor block validate recordFailure func 2020-08-28 17:11:59 +10:00
lanzafame 050a0ec5e4 further reduce code duplication 2020-08-28 17:01:45 +10:00
lanzafame 0254e4b365 reduce code duplication 2020-08-28 16:53:59 +10:00
lanzafame 3ebe4ebb3d Add views for pubsub metrics 2020-08-28 16:25:50 +10:00
lanzafame 094baf8aa9 distinguish local message validation failures from remote 2020-08-28 16:11:24 +10:00
Mike Greenberg ff5ac51c54 fix(chainwatch): Upsert on chain_economics
This table may get multiple updates depending on how many blocks get
added into a tipset. Each new block affects the tipset state, but the
parent state root will remain the same. This allows the very latest
value to be applied.
2020-08-27 23:21:20 -04:00
Mike Greenberg 9e130af9e7 feat(chainwatch): Add more complete power and reward metrics 2020-08-27 23:00:12 -04:00
Aayush Rajasekaran bed7158212 Use beacon entries in advance-block 2020-08-27 21:38:51 -04:00
Łukasz Magiera 398cbd7f2c deps: Update markets 2020-08-28 01:09:45 +02:00
Łukasz Magiera 5aa90b0478 Merge pull request #3337 from filecoin-project/feat/chainwatch/lotus-tokens
polish(chainwatch): accept lotus api and token
2020-08-28 00:27:27 +02:00
Łukasz Magiera 86c936759b Merge pull request #3357 from filecoin-project/fix/block-validator-power
ignore, don't reject block if we can't determine power
2020-08-28 00:24:53 +02:00
Łukasz Magiera 22bdc43023 Merge pull request #3359 from austinabell/runner/returnval
Add in return value check in test-vector runner
2020-08-28 00:19:32 +02:00
Łukasz Magiera ec2bc0baf6 Merge pull request #3363 from filecoin-project/fix/pubsub-params-tweaking
tweak some pubsub parameters
2020-08-28 00:18:26 +02:00
Łukasz Magiera 5b860261a0 Merge pull request #3366 from austinabell/vm/rmusm
Remove unsigned message pointer from Runtime
2020-08-28 00:14:26 +02:00
Łukasz Magiera 010e32d778 Merge pull request #3360 from filecoin-project/fix/dont-fail-drand-dns
Don't fail if could not resolve drand dnsaddr
2020-08-28 00:13:07 +02:00
Łukasz Magiera 1097d29213 sealing sched: Call trySched less when there are many tasks 2020-08-28 00:03:42 +02:00
Łukasz Magiera a6492b1ed6 storagefsm: Auto-retry failed remove 2020-08-27 23:59:01 +02:00
Łukasz Magiera 59d2034cbb sealing sched: Wait a bit for tasks to come in on restart 2020-08-27 23:58:37 +02:00
austinabell 39ef3a701b Remove unsigned message pointer from Runtime 2020-08-27 17:34:45 -04:00
Łukasz Magiera 7fdffc0340 sealing sched: Give more priority to tasks (re)moving data 2020-08-27 23:29:39 +02:00
Łukasz Magiera f2bd680cc5 gofmt 2020-08-27 23:14:46 +02:00
Łukasz Magiera 59f554b658 sealing sched: Show waiting tasks assigned to workers in sealing jobs cli 2020-08-27 23:14:33 +02:00
frrist 0ad38fe38f lint 2020-08-27 14:09:03 -07:00
frrist 5c4db23da4 polish(chainwatch): accept lotus api and token 2020-08-27 14:05:39 -07:00
vyzo 708a8b6543 flag to allow add to skip limit check
essential for correctness in the revert case
2020-08-28 00:04:21 +03:00
Łukasz Magiera d8e58e67c6 storagefsm: Treat PackingFailed sectors as expired 2020-08-27 22:41:35 +02:00
Łukasz Magiera 6c7874c883 Merge remote-tracking branch 'origin/master' into fix/some-fsm-issues 2020-08-27 21:37:08 +02:00
Łukasz Magiera 15204a9b90 storagefsm: fix tests 2020-08-27 21:34:45 +02:00
Łukasz Magiera 489d5239a5 storagefsm: Attempt to auto-recover from reorged DealIDs 2020-08-27 21:11:11 +02:00
frrist f934ebd429 fix(chainwatch): sync based on height 2020-08-27 11:47:53 -07:00
frrist fccdd70524 polish(chainwatch): relax scheduler refresh 2020-08-27 11:47:53 -07:00
frrist 5e40dda3bc fix(chainwatch): fix deadlock in sector deals 2020-08-27 11:47:53 -07:00
frrist fb8340acb2 polish(chainwatch): clean up logging 2020-08-27 11:47:53 -07:00
vyzo 20fbd66c90 tweak some pubsub parameters
- increase Bootstapp prune backoff to 5min from the default 1min
- reduce opportunistic graft threshold to 3.5 to account for low peer scores
2020-08-27 21:12:38 +03:00
Jakub Sztandera b27ee8241a Add TODO note
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-08-27 19:05:37 +02:00
Łukasz Magiera 7806a9885a storagefsm: Record publish deals message CID 2020-08-27 17:50:50 +02:00
Jakub Sztandera 6e4889d340 Don't fail if could not resolved drand dnsaddr
Resolves https://github.com/filecoin-project/lotus/issues/3335

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-08-27 17:50:16 +02:00
austinabell 679a8e0c24 Add in return value check in test-vector runner 2020-08-27 11:42:01 -04:00
vyzo 6c3d8ec1de ignore, don't reject block if we can't determine power 2020-08-27 16:53:53 +03:00
Łukasz Magiera 29078aaae9 storagefsm: Retry sending messages on out-of-gas errors 2020-08-27 14:02:00 +02:00
Łukasz Magiera 8a45148070 docsgen 2020-08-27 13:53:04 +02:00
Łukasz Magiera df635579c4 storagefsm: Handle sectors with expired deals better 2020-08-27 13:51:38 +02:00
Łukasz Magiera 788c7dbf48 storagefsm: Separate satte for submitting commit message 2020-08-27 12:57:28 +02:00
Łukasz Magiera 99ecef89b8 gofmt 2020-08-27 12:41:24 +02:00
Łukasz Magiera 51ca460f18 miner: Fix sorting of some sector states in miner-info 2020-08-27 12:41:12 +02:00
Łukasz Magiera e236173417 miner: Print (pre)commit message cids in sector status 2020-08-27 12:40:19 +02:00
Łukasz Magiera 15191ff80f storagefsm: Allow removing sectors in all states 2020-08-27 12:39:20 +02:00
Łukasz Magiera efd2129187 Merge pull request #3330 from filecoin-project/feat/slightly-better-log-info
print block 'age' when validating blocks during sync
2020-08-27 11:13:49 +02:00
Łukasz Magiera 73da4b9eed Merge pull request #3336 from filecoin-project/feat/even-less-flaky-tests
Wait until blocks show up full nodes 'head' before claiming block has been mined
2020-08-27 11:07:14 +02:00
whyrusleeping 36ff37ba47 Wait until blocks show up full nodes 'head' before claiming block has been mined 2020-08-26 17:51:16 -07:00
whyrusleeping a33784c434 print block 'age' when validating blocks during sync 2020-08-26 14:48:53 -07:00
Whyrusleeping deb013cecb Merge pull request #3081 from filecoin-project/conformance-tests
driver, runner and CI for interoperable test-vector based testing.
2020-08-26 14:06:39 -07:00
Whyrusleeping cb411637cc Merge pull request #3328 from filecoin-project/fix/log-level-miner
use node type agnostic api getter in log commands
2020-08-26 14:02:29 -07:00
Will be89466367 add back junit report in conformance ci (#3327) 2020-08-26 21:55:06 +01:00
whyrusleeping 111c7cf688 use node type agnostic api getter in log commands 2020-08-26 13:51:32 -07:00
Raúl Kripalani 75e0387be2 fix ffiwrapper import. 2020-08-26 21:49:55 +01:00
Łukasz Magiera c4e32fac1f Merge pull request #3324 from filecoin-project/fix/retrieval-closing-panic
Resolve retrieval close event panic
2020-08-26 22:15:21 +02:00
vyzo 038e83b647 log error if we trip over too many pending messages for actor 2020-08-26 22:55:39 +03:00
vyzo 82ef0529d8 named error for too many pending messages 2020-08-26 22:48:03 +03:00
vyzo 6abfbbddd5 fix tests 2020-08-26 22:44:47 +03:00
vyzo ad889a7240 rate limit messages per actor by enforcing maximum pending message limit 2020-08-26 22:40:10 +03:00
Łukasz Magiera e22ae9f826 Merge pull request #3323 from filecoin-project/steb/fix-post
Fix PoSt with bad sectors
2020-08-26 21:22:21 +02:00
vyzo d3e95d6b77 cast min base fee to uint64 2020-08-26 21:37:32 +03:00
vyzo 8db262c941 check gas fee cap for minimum value 2020-08-26 21:23:28 +03:00
vyzo 793eda986b comment out Value inclusion for required funds tracking
we are not sure whether we want it yet.
2020-08-26 21:10:46 +03:00
hannahhoward 1cd49cec39 fix(client): insure cancelled context avoids lock 2020-08-26 10:59:07 -07:00
hannahhoward 10c1399474 fix(retrieval): resolve retrieval close event panic
Refactor ClientRetrieve command to remove the possibility of a send on close channel race condition
2020-08-26 10:48:23 -07:00
Raúl Kripalani 4709e231b8 drop aurora as dep; upgrade test-vectors. 2020-08-26 18:35:41 +01:00
Steven Allen f8a14184c7 Fix lint issue 2020-08-26 10:05:03 -07:00
Steven Allen 0155a31d1f Fix PoSt with bad sectors
"skipped" sectors must be replaced with a substitute "good" sector, or the
entire partition must be skipped. They should not just be omitted.

This patch also fixes the test to verify the _entire_ proof instead of just
verifying that the proof includes the correct sectors.
2020-08-26 09:56:51 -07:00
vyzo 8848c549a0 move msg serialization outside of lock, consistently use curTs 2020-08-26 19:42:56 +03:00
vyzo 4919a005d3 always verify message min gas 2020-08-26 19:37:27 +03:00
Raúl Kripalani dabe14889d actually apply all messages. 2020-08-26 16:48:47 +01:00
Alan Shaw 6d1e711b2a feat: bleeding edge conformance (#3317) 2020-08-26 16:03:44 +01:00
Raúl Kripalani e19087722f Merge branch 'master' into conformance-tests 2020-08-26 15:13:46 +01:00
Raúl Kripalani d3ac243cea Merge remote-tracking branch 'origin/master' into conformance-tests 2020-08-26 15:12:02 +01:00
Łukasz Magiera 348dadf7ae Merge pull request #3305 from filecoin-project/feat/update-pubsub
update go-libp2p-pubsub
2020-08-26 16:09:48 +02:00
Raúl Kripalani cba5ae81ed call out to filecoin-project/statediff upon state mismatches. 2020-08-26 15:09:33 +01:00
Łukasz Magiera 6d328c0a71 Merge pull request #3298 from filecoin-project/fix/flaky-mining-tests
improve flakiness of a few tests
2020-08-26 16:07:52 +02:00
vyzo 7a7066896e better code for getRequiredFunds 2020-08-26 15:37:42 +03:00
vyzo a2c0c10d2a correctly account for message replacement in required funds tracking 2020-08-26 15:27:09 +03:00
vyzo c473d3c682 rename ErrValidationFailure to ErrSoftValidationFailure
to make it clear to readers that this failure is soft; we might be out of sync.
2020-08-26 15:16:04 +03:00
vyzo bedbdcaf13 give some score weight to the messages topic 2020-08-26 15:14:15 +03:00
vyzo 87e6c0973a include message Value only in soft validation failures 2020-08-26 15:13:50 +03:00
vyzo 78876947a4 ErrBroadcastAnyway is now ErrValidationFailure 2020-08-26 15:03:13 +03:00
vyzo b59f58435c fix tests 2020-08-26 14:58:32 +03:00
vyzo 4adb83eaae check aggregate pending message required funds against balance 2020-08-26 14:58:26 +03:00
vyzo 4ac9828b0c track required funds in message set 2020-08-26 14:38:26 +03:00
vyzo 77f6e0d4f8 refactor balance check, apply it in PushWithNonce 2020-08-26 14:23:34 +03:00
vyzo 835dbfe001 check message before adding in PushWithNonce 2020-08-26 14:13:46 +03:00
David Dias f5eb3fd086 docs: removed non used packages from the list 2020-08-26 12:10:11 +01:00
vyzo da9630c561 remove duplicate check
gas limit is checked against block gas limit in ValidForBlockInclusion
2020-08-26 14:02:34 +03:00
Alan Shaw a89faa72ea feat(conformance): skip incorrect tests (#3310)
Fixes https://github.com/filecoin-project/test-vectors/issues/72
2020-08-26 11:45:34 +01:00
vyzo e042ce21f6 gomod: update go-libp2p-pubsub@v0.3.5 2020-08-26 08:36:58 +03:00
whyrusleeping 91bb559bdf address another spot with the more certain mining fix 2020-08-25 17:06:45 -07:00
whyrusleeping d0585ca7bb improve flakiness of a few tests 2020-08-25 15:07:10 -07:00
Dirk McCormick 340d11be38 fix: check voucher spendable should take into account submitted vouchers 2020-08-25 12:51:29 +02:00
Will a03931daab archive more relevant coverage for conformance tests (#3176) 2020-08-21 16:41:35 +01:00
Raúl Kripalani 6ed1c9e905 register the puppet actor conditionally. 2020-08-21 11:51:50 +01:00
Raúl Kripalani e38f4219c5 update test-vectors commit. 2020-08-19 18:19:03 +01:00
Raúl Kripalani 30b6f38704 update test-vectors commit. 2020-08-19 18:14:19 +01:00
Raúl Kripalani 04e4bab313 register chaos actor only if selector says to do so. 2020-08-19 18:10:56 +01:00
Raúl Kripalani dddf7951ef update test-vectors commit. 2020-08-19 17:33:30 +01:00
Raúl Kripalani b13681df32 update test-vectors commit. 2020-08-19 14:40:55 +01:00
Raúl Kripalani a21234cd54 deploy the chaos actor. 2020-08-19 14:31:34 +01:00
Raúl Kripalani 4f4cebadc7 Merge branch 'master' into conformance-tests 2020-08-19 13:33:15 +01:00
Raúl Kripalani f1c452754f separate conformance suite on CI. 2020-08-16 22:50:00 +01:00
Raúl Kripalani b9d67296ef small adjustments to runner. 2020-08-16 22:15:12 +01:00
Raúl Kripalani 483e82e14b point test-vectors to master. 2020-08-16 22:05:43 +01:00
Raúl Kripalani a0c0d9c98a small fixes. 2020-08-16 00:37:16 +01:00
Raúl Kripalani 322b33197c introduce interoperable vector-based conformance testing.
This commit introduces a new package `conformance` containing:

 1. the test driver to exercise Lotus against interoperable
    test vectors, and
 2. the test runner, which integrates go test with the test vector
    corpus hosted at https://github.com/filecoin-project/conformance-vectors.

The corpus is mounted via a git submodule.

Right now, only message-class test vectors are supported. In the
next week, this support will be extended to tipset-class, chain-class,
and block sequence-class vectors.
2020-08-16 00:15:52 +01:00
338 changed files with 11241 additions and 3532 deletions
+72 -1
View File
@@ -113,7 +113,7 @@ jobs:
test: &test
description: |
Run tests with gotestsum.
parameters:
parameters: &test-params
executor:
type: executor
default: golang
@@ -161,6 +161,7 @@ jobs:
name: go test
environment:
LOTUS_TEST_WINDOW_POST: << parameters.winpost-test >>
SKIP_CONFORMANCE: "1"
command: |
mkdir -p /tmp/test-reports/<< parameters.test-suite-name >>
mkdir -p /tmp/test-artifacts
@@ -191,6 +192,68 @@ jobs:
<<: *test
test-window-post:
<<: *test
test-conformance:
description: |
Run tests using a corpus of interoperable test vectors for Filecoin
implementations to test their correctness and compliance with the Filecoin
specifications.
parameters:
<<: *test-params
vectors-branch:
type: string
default: ""
description: |
Branch on github.com/filecoin-project/test-vectors to checkout and
test with. If empty (the default) the commit defined by the git
submodule is used.
executor: << parameters.executor >>
steps:
- install-deps
- prepare
- run:
command: make deps lotus
no_output_timeout: 30m
- download-params
- when:
condition:
not:
equal: [ "", << parameters.vectors-branch >> ]
steps:
- run:
name: checkout vectors branch
command: |
cd extern/test-vectors
git fetch
git checkout origin/<< parameters.vectors-branch >>
- go/install-gotestsum:
gobin: $HOME/.local/bin
version: 0.5.2
- run:
name: install statediff globally
command: |
## statediff is optional; we succeed even if compilation fails.
mkdir -p /tmp/statediff
git clone https://github.com/filecoin-project/statediff.git /tmp/statediff
cd /tmp/statediff
go install ./cmd/statediff || exit 0
- run:
name: go test
environment:
SKIP_CONFORMANCE: "0"
command: |
mkdir -p /tmp/test-reports
mkdir -p /tmp/test-artifacts
gotestsum \
--format pkgname-and-test-fails \
--junitfile /tmp/test-reports/junit.xml \
-- \
-v -coverpkg ./chain/vm/,github.com/filecoin-project/specs-actors/... -coverprofile=/tmp/conformance.out ./conformance/
go tool cover -html=/tmp/conformance.out -o /tmp/test-artifacts/conformance-coverage.html
no_output_timeout: 30m
- store_test_results:
path: /tmp/test-reports
- store_artifacts:
path: /tmp/test-artifacts/conformance-coverage.html
build-macos:
description: build darwin lotus binary
@@ -356,6 +419,14 @@ workflows:
tags:
only:
- /^v\d+\.\d+\.\d+$/
- test-conformance:
test-suite-name: conformance
packages: "./conformance"
- test-conformance:
name: test-conformance-bleeding-edge
test-suite-name: conformance-bleeding-edge
packages: "./conformance"
vectors-branch: master
- build-debug
- build-all:
requires:
+2
View File
@@ -1,3 +1,5 @@
comment: off
ignore:
- "cbor_gen.go"
github_checks:
annotations: false
+15
View File
@@ -0,0 +1,15 @@
## filecoin-project/lotus CODEOWNERS
## Refer to https://docs.github.com/en/github/creating-cloning-and-archiving-repositories/about-code-owners.
##
## These users or groups will be automatically assigned as reviewers every time
## a PR is submitted that modifies code in the specified locations.
##
## The Lotus repo configuration requires that at least ONE codeowner approves
## the PR before merging.
### Global owners.
* @magik6k @whyrusleeping @Kubuxu
### Conformance testing.
conformance/ @raulk
extern/test-vectors @raulk
+8 -2
View File
@@ -26,11 +26,14 @@
color: 00A4E2
description: "Area: Chain/VM"
- name: area/chain/sync
color: 00A4E2
color: 00A4E4
description: "Area: Chain/Sync"
- name: area/chain/misc
color: 00A4E2
color: 00A4E6
description: "Area: Chain/Misc"
- name: area/markets
color: 00A4E8
description: "Area: Markets"
- name: area/sealing/fsm
color: 0bb1ed
description: "Area: Sealing/FSM"
@@ -149,6 +152,9 @@
- name: impact/test-flakiness
color: DDE1E4
description: "Impact: Test Flakiness"
- name: impact/consensus
color: b20014
description: "Impact: Consensus"
###
### Topics
+3
View File
@@ -5,3 +5,6 @@
[submodule "extern/serialization-vectors"]
path = extern/serialization-vectors
url = https://github.com/filecoin-project/serialization-vectors
[submodule "extern/test-vectors"]
path = extern/test-vectors
url = https://github.com/filecoin-project/test-vectors.git
+264
View File
@@ -1,5 +1,269 @@
# Lotus changelog
# 0.7.0 / 2020-09-10
This consensus-breaking release of Lotus is designed to test a network upgrade on the space race testnet. The changes that break consensus are:
- Upgrading the Drand network used from the test Drand network to the League of Entropy main drand network. This is the same Drand network that will be used in the Filecoin mainnet.
- Upgrading to specs-actors v0.9.8, which adds a new method to the Multisig actor.
## Changes
#### Core Lotus
- Fix IsAncestorOf (https://github.com/filecoin-project/lotus/pull/3717)
- Update to specs-actors v0.9.8 (https://github.com/filecoin-project/lotus/pull/3725)
- Increase chain throughput by 20% (https://github.com/filecoin-project/lotus/pull/3732)
- Updare to go-libp2p-pubsub `master` (https://github.com/filecoin-project/lotus/pull/3735)
- Drand upgrade (https://github.com/filecoin-project/lotus/pull/3670)
- Multisig API additions (https://github.com/filecoin-project/lotus/pull/3590)
#### Storage Miner
- Increase the number of times precommit2 is attempted before moving back to precommit1 (https://github.com/filecoin-project/lotus/pull/3720)
#### Message pool
- Relax mpool add strictness checks for local pushes (https://github.com/filecoin-project/lotus/pull/3724)
#### Maintenance
- Fix devnets (https://github.com/filecoin-project/lotus/pull/3712)
- Fix(chainwatch): compare prev miner with cur miner (https://github.com/filecoin-project/lotus/pull/3715)
- CI: fix statediff build; make optional (https://github.com/filecoin-project/lotus/pull/3729)
- Feat: Chaos abort (https://github.com/filecoin-project/lotus/pull/3733)
## Contributors
The following contributors had commits go into this release.
We are grateful for every contribution!
| Contributor | Commits | Lines ± |
|--------------------|---------|---------------|
| arajasek | 28 | +1144/-239 |
| Kubuxu | 19 | +452/-261 |
| whyrusleeping | 13 | +456/-87 |
| vyzo | 11 | +318/-20 |
| raulk | 10 | +1289/-350 |
| magik6k | 6 | +188/-55 |
| dirkmc | 3 | +31/-8 |
| alanshaw | 3 | +176/-37 |
| Stebalien | 2 | +9/-12 |
| lanzafame | 1 | +1/-1 |
| frrist | 1 | +1/-1 |
| mishmosh | 1 | +1/-1 |
| nonsense | 1 | +1/-0 |
# 0.6.2 / 2020-09-09
This release introduces some critical fixes to message selection and gas estimation logic. It also adds the ability for nodes to mark a certain tipset as checkpointed, as well as various minor improvements and bugfixes.
## Changes
#### Messagepool
- Warn when optimal selection fails to pack a block and we fall back to random selection (https://github.com/filecoin-project/lotus/pull/3708)
- Add basic command for printing gas performance of messages in the mpool (https://github.com/filecoin-project/lotus/pull/3701)
- Adjust optimal selection to always try to fill blocks (https://github.com/filecoin-project/lotus/pull/3685)
- Fix very minor bug in repub baseFeeLowerBound (https://github.com/filecoin-project/lotus/pull/3663)
- Add an auto flag to mpool replace (https://github.com/filecoin-project/lotus/pull/3676)
- Fix mpool optimal selection packing failure (https://github.com/filecoin-project/lotus/pull/3698)
#### Core Lotus
- Don't use latency as initital estimate for blocksync (https://github.com/filecoin-project/lotus/pull/3648)
- Add niceSleep 1 second when drand errors (https://github.com/filecoin-project/lotus/pull/3664)
- Fix isChainNearSync check in block validator (https://github.com/filecoin-project/lotus/pull/3650)
- Add peer to peer manager before fetching the tipset (https://github.com/filecoin-project/lotus/pull/3667)
- Add StageFetchingMessages to sync status (https://github.com/filecoin-project/lotus/pull/3668)
- Pass tipset through upgrade logic (https://github.com/filecoin-project/lotus/pull/3673)
- Allow nodes to mark tipsets as checkpointed (https://github.com/filecoin-project/lotus/pull/3680)
- Remove hard-coded late-fee in window PoSt (https://github.com/filecoin-project/lotus/pull/3702)
- Gas: Fix median calc (https://github.com/filecoin-project/lotus/pull/3686)
#### Storage
- Storage manager: bail out with an error if unsealed cid is undefined (https://github.com/filecoin-project/lotus/pull/3655)
- Storage: return true from Sealer.ReadPiece() on success (https://github.com/filecoin-project/lotus/pull/3657)
#### Maintenance
- Resolve lotus, test-vectors, statediff dependency cycle (https://github.com/filecoin-project/lotus/pull/3688)
- Paych: add docs on how to use paych status (https://github.com/filecoin-project/lotus/pull/3690)
- Initial CODEOWNERS (https://github.com/filecoin-project/lotus/pull/3691)
# 0.6.1 / 2020-09-08
This optional release introduces a minor improvement to the sync process, ensuring nodes don't fall behind and then resync.
## Changes
- Update `test-vectors` (https://github.com/filecoin-project/lotus/pull/3645)
- Revert "only subscribe to pubsub topics once we are synced" (https://github.com/filecoin-project/lotus/pull/3643)
# 0.6.0 / 2020-09-07
This consensus-breaking release of Lotus is designed to test a network upgrade on the space race testnet. The changes that break consensus are:
- Tweaking of some cryptoecon parameters in specs-actors 0.9.7 (https://github.com/filecoin-project/specs-actors/releases/tag/v0.9.7)
- Rebalancing FIL distribution to make testnet FIL scarce, which prevents base fee spikes and sets better expectations for mainnet
This release also introduces many improvements to Lotus! Among them are a new version of go-fil-markets that supports non-blocking retrieval, various spam reduction measures in the messagepool and p2p logic, and UX improvements to payment channels, dealmaking, and state inspection.
## Changes
#### Core Lotus and dependencies
- Implement faucet funds reallocation logic (https://github.com/filecoin-project/lotus/pull/3632)
- Network upgrade: Upgrade to correct fork threshold (https://github.com/filecoin-project/lotus/pull/3628)
- Update to specs 0.9.7 and markets 0.6.0 (https://github.com/filecoin-project/lotus/pull/3627)
- Network upgrade: Perform base fee tamping (https://github.com/filecoin-project/lotus/pull/3623)
- Chain events: if cache best() is nil, return chain head (https://github.com/filecoin-project/lotus/pull/3611)
- Update to specs actors v0.9.6 (https://github.com/filecoin-project/lotus/pull/3603)
#### Messagepool
- Temporarily allow negative chains (https://github.com/filecoin-project/lotus/pull/3625)
- Improve publish/republish logic (https://github.com/filecoin-project/lotus/pull/3592)
- Fix selection bug; priority messages were not included if other chains were negative (https://github.com/filecoin-project/lotus/pull/3580)
- Add defensive check for minimum GasFeeCap for inclusion within the next 20 blocks (https://github.com/filecoin-project/lotus/pull/3579)
- Add additional info about gas premium (https://github.com/filecoin-project/lotus/pull/3578)
- Fix GasPremium capping logic (https://github.com/filecoin-project/lotus/pull/3552)
#### Payment channels
- Get available funds by address or by from/to (https://github.com/filecoin-project/lotus/pull/3547)
- Create `lotus paych status` command (https://github.com/filecoin-project/lotus/pull/3523)
- Rename CLI command from "paych get" to "paych add-funds" (https://github.com/filecoin-project/lotus/pull/3520)
#### Peer-to-peer
- Only subscribe to pubsub topics once we are synced (https://github.com/filecoin-project/lotus/pull/3602)
- Reduce mpool add failure log spam (https://github.com/filecoin-project/lotus/pull/3562)
- Republish messages even if the chains have negative performance(https://github.com/filecoin-project/lotus/pull/3557)
- Adjust gossipsub gossip factor (https://github.com/filecoin-project/lotus/pull/3556)
- Integrate pubsub Random Early Drop (https://github.com/filecoin-project/lotus/pull/3518)
#### Miscellaneous
- Fix panic in OnDealExpiredSlashed (https://github.com/filecoin-project/lotus/pull/3553)
- Robustify state manager against holes in actor method numbers (https://github.com/filecoin-project/lotus/pull/3538)
#### UX
- VM: Fix an error message (https://github.com/filecoin-project/lotus/pull/3608)
- Documentation: Batch replacement,update lotus-storage-miner to lotus-miner (https://github.com/filecoin-project/lotus/pull/3571)
- CLI: Robust actor lookup (https://github.com/filecoin-project/lotus/pull/3535)
- Add agent flag to net peers (https://github.com/filecoin-project/lotus/pull/3534)
- Add watch option to storage-deals list (https://github.com/filecoin-project/lotus/pull/3527)
#### Testing & tooling
- Decommission chain-validation (https://github.com/filecoin-project/lotus/pull/3606)
- Metrics: add expected height metric (https://github.com/filecoin-project/lotus/pull/3586)
- PCR: Use current tipset during refund (https://github.com/filecoin-project/lotus/pull/3570)
- Lotus-shed: Add math command (https://github.com/filecoin-project/lotus/pull/3568)
- PCR: Add tipset aggergation (https://github.com/filecoin-project/lotus/pull/3565)- Fix broken paych tests (https://github.com/filecoin-project/lotus/pull/3551)
- Make chain export ~1000x times faster (https://github.com/filecoin-project/lotus/pull/3533)
- Chainwatch: Stop SyncIncomingBlocks from leaking into chainwatch processing; No panics during processing (https://github.com/filecoin-project/lotus/pull/3526)
- Conformance: various changes (https://github.com/filecoin-project/lotus/pull/3521)
# 0.5.10 / 2020-09-03
This patch includes a crucial fix to the message pool selection logic, strongly disfavouring messages that might cause a miner penalty.
## Changes
- Fix calculation of GasReward in messagepool (https://github.com/filecoin-project/lotus/pull/3528)
# 0.5.9 / 2020-09-03
This patch includes a hotfix to the `GasEstimateFeeCap` method, capping the estimated fee to a reasonable level by default.
## Changes
- Added target height to sync wait (https://github.com/filecoin-project/lotus/pull/3502)
- Disable codecov annotations (https://github.com/filecoin-project/lotus/pull/3514)
- Cap fees to reasonable level by default (https://github.com/filecoin-project/lotus/pull/3516)
- Add APIs and command to inspect bandwidth usage (https://github.com/filecoin-project/lotus/pull/3497)
- Track expected nonce in mpool, ignore messages with large nonce gaps (https://github.com/filecoin-project/lotus/pull/3450)
# 0.5.8 / 2020-09-02
This patch includes some bugfixes to the sector sealing process, and updates go-fil-markets. It also improves the performance of blocksync, adds a method to export chain state trees, and improves chainwatch.
## Changes
- Upgrade markets to v0.5.9 (https://github.com/filecoin-project/lotus/pull/3496)
- Improve blocksync to load fewer messages: (https://github.com/filecoin-project/lotus/pull/3494)
- Fix a panic in the ffi-wrapper's `ReadPiece` (https://github.com/filecoin-project/lotus/pull/3492/files)
- Fix a deadlock in the sealing scheduler (https://github.com/filecoin-project/lotus/pull/3489)
- Add test vectors for tipset tests (https://github.com/filecoin-project/lotus/pull/3485/files)
- Improve the advance-block debug command (https://github.com/filecoin-project/lotus/pull/3476)
- Add toggle for message processing to Lotus PCR (https://github.com/filecoin-project/lotus/pull/3470)
- Allow exporting recent chain state trees (https://github.com/filecoin-project/lotus/pull/3463)
- Remove height from chain rand (https://github.com/filecoin-project/lotus/pull/3458)
- Disable GC on chain badger datastore (https://github.com/filecoin-project/lotus/pull/3457)
- Account for `GasPremium` in `GasEstimateFeeCap` (https://github.com/filecoin-project/lotus/pull/3456)
- Update go-libp2p-pubsub to `master` (https://github.com/filecoin-project/lotus/pull/3455)
- Chainwatch improvements (https://github.com/filecoin-project/lotus/pull/3442)
# 0.5.7 / 2020-08-31
This patch release includes some bugfixes and enhancements to the sector lifecycle and message pool logic.
## Changes
- Rebuild unsealed infos on miner restart (https://github.com/filecoin-project/lotus/pull/3401)
- CLI to attach storage paths to workers (https://github.com/filecoin-project/lotus/pull/3405)
- Do not select negative performing message chains for inclusion (https://github.com/filecoin-project/lotus/pull/3392)
- Remove a redundant error-check (https://github.com/filecoin-project/lotus/pull/3421)
- Correctly move unsealed sectors in `FinalizeSectors` (https://github.com/filecoin-project/lotus/pull/3424)
- Improve worker selection logic (https://github.com/filecoin-project/lotus/pull/3425)
- Don't use context to close bitswap (https://github.com/filecoin-project/lotus/pull/3430)
- Correctly estimate gas premium when there is only one message on chain (https://github.com/filecoin-project/lotus/pull/3428)
# 0.5.6 / 2020-08-29
Hotfix release that fixes a panic in the sealing scheduler (https://github.com/filecoin-project/lotus/pull/3389).
# 0.5.5
This patch release introduces a large number of improvements to the sealing process.
It also updates go-fil-markets to
[version 0.5.8](https://github.com/filecoin-project/go-fil-markets/releases/tag/v0.5.8),
and go-libp2p-pubsub to [v0.3.5](https://github.com/libp2p/go-libp2p-pubsub/releases/tag/v0.3.5).
#### Downstream upgrades
- Upgrades markets to v0.5.8 (https://github.com/filecoin-project/lotus/pull/3384)
- Upgrades go-libp2p-pubsub to v0.3.5 (https://github.com/filecoin-project/lotus/pull/3305)
#### Sector sealing
- The following improvements were introduced in https://github.com/filecoin-project/lotus/pull/3350.
- Allow `lotus-miner sectors remove` to remove a sector in any state.
- Create a separate state in the storage FSM dedicated to submitting the Commit message.
- Recovery for when the Deal IDs of deals in a sector get changed in a reorg.
- Auto-retry sending Precommit and Commit messages if they run out of gas
- Auto-retry sector remove tasks when they fail
- Compact worker windows, and allow their tasks to be executed in any order
- Don't simply skip PoSt for bad sectors (https://github.com/filecoin-project/lotus/pull/3323)
#### Message Pool
- Spam Protection: Track required funds for pending messages (https://github.com/filecoin-project/lotus/pull/3313)
#### Chainwatch
- Add more power and reward metrics (https://github.com/filecoin-project/lotus/pull/3367)
- Fix raciness in sector deal table (https://github.com/filecoin-project/lotus/pull/3275)
- Parallelize miner processing (https://github.com/filecoin-project/lotus/pull/3380)
- Accept Lotus API and token (https://github.com/filecoin-project/lotus/pull/3337)
# 0.5.4
A patch release, containing a few nice bugfixes and improvements:
-3
View File
@@ -40,9 +40,6 @@ All work is tracked via issues. An attempt at keeping an up-to-date view on rema
The lotus Filecoin implementation unfolds into the following packages:
- [This repo](https://github.com/filecoin-project/lotus)
- [storage-fsm](https://github.com/filecoin-project/storage-fsm)
- [sector-storage](https://github.com/filecoin-project/sector-storage)
- [specs-storage](https://github.com/filecoin-project/specs-storage)
- [go-fil-markets](https://github.com/filecoin-project/go-fil-markets) which has its own [kanban work tracker available here](https://app.zenhub.com/workspaces/markets-shared-components-5daa144a7046a60001c6e253/board)
- [spec-actors](https://github.com/filecoin-project/specs-actors) which has its own [kanban work tracker available here](https://app.zenhub.com/workspaces/actors-5ee6f3aa87591f0016c05685/board)
+15
View File
@@ -5,8 +5,10 @@ import (
"fmt"
"github.com/filecoin-project/go-jsonrpc/auth"
metrics "github.com/libp2p/go-libp2p-core/metrics"
"github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/peer"
protocol "github.com/libp2p/go-libp2p-core/protocol"
"github.com/filecoin-project/lotus/build"
)
@@ -28,6 +30,19 @@ type Common interface {
NetFindPeer(context.Context, peer.ID) (peer.AddrInfo, error)
NetPubsubScores(context.Context) ([]PubsubScore, error)
NetAutoNatStatus(context.Context) (NatInfo, error)
NetAgentVersion(ctx context.Context, p peer.ID) (string, error)
// NetBandwidthStats returns statistics about the nodes total bandwidth
// usage and current rate across all peers and protocols.
NetBandwidthStats(ctx context.Context) (metrics.Stats, error)
// NetBandwidthStatsByPeer returns statistics about the nodes bandwidth
// usage and current rate per peer
NetBandwidthStatsByPeer(ctx context.Context) (map[string]metrics.Stats, error)
// NetBandwidthStatsByProtocol returns statistics about the nodes bandwidth
// usage and current rate per protocol
NetBandwidthStatsByProtocol(ctx context.Context) (map[protocol.ID]metrics.Stats, error)
// MethodGroup: Common
+105 -16
View File
@@ -2,23 +2,29 @@ package api
import (
"context"
"fmt"
"time"
"github.com/filecoin-project/go-state-types/dline"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/ipfs/go-cid"
"github.com/libp2p/go-libp2p-core/peer"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-fil-markets/storagemarket"
"github.com/filecoin-project/go-multistore"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/builtin/market"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/paych"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/lotus/chain/types"
marketevents "github.com/filecoin-project/lotus/markets/loggers"
@@ -109,7 +115,10 @@ type FullNode interface {
ChainGetPath(ctx context.Context, from types.TipSetKey, to types.TipSetKey) ([]*HeadChange, error)
// ChainExport returns a stream of bytes with CAR dump of chain data.
ChainExport(context.Context, types.TipSetKey) (<-chan []byte, error)
// The exported chain data includes the header chain from the given tipset
// back to genesis, the entire genesis state, and the most recent 'nroots'
// state trees.
ChainExport(ctx context.Context, nroots abi.ChainEpoch, tsk types.TipSetKey) (<-chan []byte, error)
// MethodGroup: Beacon
// The Beacon method group contains methods for interacting with the random beacon (DRAND)
@@ -150,10 +159,16 @@ type FullNode interface {
// yet synced block headers.
SyncIncomingBlocks(ctx context.Context) (<-chan *types.BlockHeader, error)
// SyncCheckpoint marks a blocks as checkpointed, meaning that it won't ever fork away from it.
SyncCheckpoint(ctx context.Context, tsk types.TipSetKey) error
// SyncMarkBad marks a blocks as bad, meaning that it won't ever by synced.
// Use with extreme caution.
SyncMarkBad(ctx context.Context, bcid cid.Cid) error
// SyncUnmarkBad unmarks a blocks as bad, making it possible to be validated and synced again.
SyncUnmarkBad(ctx context.Context, bcid cid.Cid) error
// SyncCheckBad checks if a block was marked as bad, and if it was, returns
// the reason.
SyncCheckBad(ctx context.Context, bcid cid.Cid) (string, error)
@@ -243,6 +258,8 @@ type FullNode interface {
ClientGetDealInfo(context.Context, cid.Cid) (*DealInfo, error)
// ClientListDeals returns information about the deals made by the local client.
ClientListDeals(ctx context.Context) ([]DealInfo, error)
// ClientGetDealUpdates returns the status of updated deals
ClientGetDealUpdates(ctx context.Context) (<-chan DealInfo, error)
// ClientHasLocal indicates whether a certain CID is locally stored.
ClientHasLocal(ctx context.Context, root cid.Cid) (bool, error)
// ClientFindData identifies peers that have a certain file, and returns QueryOffers (one per peer).
@@ -265,6 +282,9 @@ type FullNode interface {
// ClientListTransfers returns the status of all ongoing transfers of data
ClientListDataTransfers(ctx context.Context) ([]DataTransferChannel, error)
ClientDataTransferUpdates(ctx context.Context) (<-chan DataTransferChannel, error)
// ClientRetrieveTryRestartInsufficientFunds attempts to restart stalled retrievals on a given payment channel
// which are stuck due to insufficient funds
ClientRetrieveTryRestartInsufficientFunds(ctx context.Context, paymentChannel address.Address) error
// ClientUnimport removes references to the specified file from filestore
//ClientUnimport(path string)
@@ -295,12 +315,12 @@ type FullNode interface {
// StateMinerSectors returns info about the given miner's sectors. If the filter bitfield is nil, all sectors are included.
// If the filterOut boolean is set to true, any sectors in the filter are excluded.
// If false, only those sectors in the filter are included.
StateMinerSectors(context.Context, address.Address, *abi.BitField, bool, types.TipSetKey) ([]*ChainSectorInfo, error)
StateMinerSectors(context.Context, address.Address, *bitfield.BitField, bool, types.TipSetKey) ([]*ChainSectorInfo, error)
// StateMinerActiveSectors returns info about sectors that a given miner is actively proving.
StateMinerActiveSectors(context.Context, address.Address, types.TipSetKey) ([]*ChainSectorInfo, error)
// StateMinerProvingDeadline calculates the deadline at some epoch for a proving period
// and returns the deadline-related calculations.
StateMinerProvingDeadline(context.Context, address.Address, types.TipSetKey) (*miner.DeadlineInfo, error)
StateMinerProvingDeadline(context.Context, address.Address, types.TipSetKey) (*dline.Info, error)
// StateMinerPower returns the power of the indicated miner
StateMinerPower(context.Context, address.Address, types.TipSetKey) (*MinerPower, error)
// StateMinerInfo returns info about the indicated miner
@@ -310,11 +330,11 @@ type FullNode interface {
// StateMinerPartitions loads miner partitions for the specified miner/deadline
StateMinerPartitions(context.Context, address.Address, uint64, types.TipSetKey) ([]*miner.Partition, error)
// StateMinerFaults returns a bitfield indicating the faulty sectors of the given miner
StateMinerFaults(context.Context, address.Address, types.TipSetKey) (abi.BitField, error)
StateMinerFaults(context.Context, address.Address, types.TipSetKey) (bitfield.BitField, error)
// StateAllMinerFaults returns all non-expired Faults that occur within lookback epochs of the given tipset
StateAllMinerFaults(ctx context.Context, lookback abi.ChainEpoch, ts types.TipSetKey) ([]*Fault, error)
// StateMinerRecoveries returns a bitfield indicating the recovering sectors of the given miner
StateMinerRecoveries(context.Context, address.Address, types.TipSetKey) (abi.BitField, error)
StateMinerRecoveries(context.Context, address.Address, types.TipSetKey) (bitfield.BitField, error)
// StateMinerInitialPledgeCollateral returns the precommit deposit for the specified miner's sector
StateMinerPreCommitDepositForPower(context.Context, address.Address, miner.SectorPreCommitInfo, types.TipSetKey) (types.BigInt, error)
// StateMinerInitialPledgeCollateral returns the initial pledge collateral for the specified miner's sector
@@ -323,7 +343,7 @@ type FullNode interface {
StateMinerAvailableBalance(context.Context, address.Address, types.TipSetKey) (types.BigInt, error)
// StateSectorPreCommitInfo returns the PreCommit info for the specified miner's sector
StateSectorPreCommitInfo(context.Context, address.Address, abi.SectorNumber, types.TipSetKey) (miner.SectorPreCommitOnChainInfo, error)
// StateSectorGetInfo returns the on-chain info for the specified miner's sector
// StateSectorGetInfo returns the on-chain info for the specified miner's sector. Returns null in case the sector info isn't found
// NOTE: returned info.Expiration may not be accurate in some cases, use StateSectorExpiration to get accurate
// expiration epoch
StateSectorGetInfo(context.Context, address.Address, abi.SectorNumber, types.TipSetKey) (*miner.SectorOnChainInfo, error)
@@ -379,6 +399,9 @@ type FullNode interface {
// MsigGetAvailableBalance returns the portion of a multisig's balance that can be withdrawn or spent
MsigGetAvailableBalance(context.Context, address.Address, types.TipSetKey) (types.BigInt, error)
// MsigGetVested returns the amount of FIL that vested in a multisig in a certain period.
// It takes the following params: <multisig address>, <start epoch>, <end epoch>
MsigGetVested(context.Context, address.Address, types.TipSetKey, types.TipSetKey) (types.BigInt, error)
// MsigCreate creates a multisig wallet
// It takes the following params: <required number of senders>, <approving addresses>, <unlock duration>
//<initial balance>, <sender address of the create msg>, <gas price>
@@ -395,17 +418,29 @@ type FullNode interface {
// It takes the following params: <multisig address>, <proposed message ID>, <recipient address>, <value to transfer>,
// <sender address of the cancel msg>, <method to call in the proposed message>, <params to include in the proposed message>
MsigCancel(context.Context, address.Address, uint64, address.Address, types.BigInt, address.Address, uint64, []byte) (cid.Cid, error)
// MsigAddPropose proposes adding a signer in the multisig
// It takes the following params: <multisig address>, <sender address of the propose msg>,
// <new signer>, <whether the number of required signers should be increased>
MsigAddPropose(context.Context, address.Address, address.Address, address.Address, bool) (cid.Cid, error)
// MsigAddApprove approves a previously proposed AddSigner message
// It takes the following params: <multisig address>, <sender address of the approve msg>, <proposed message ID>,
// <proposer address>, <new signer>, <whether the number of required signers should be increased>
MsigAddApprove(context.Context, address.Address, address.Address, uint64, address.Address, address.Address, bool) (cid.Cid, error)
// MsigAddCancel cancels a previously proposed AddSigner message
// It takes the following params: <multisig address>, <sender address of the cancel msg>, <proposed message ID>,
// <new signer>, <whether the number of required signers should be increased>
MsigAddCancel(context.Context, address.Address, address.Address, uint64, address.Address, bool) (cid.Cid, error)
// MsigSwapPropose proposes swapping 2 signers in the multisig
// It takes the following params: <multisig address>, <sender address of the propose msg>,
// <old signer> <new signer>
// <old signer>, <new signer>
MsigSwapPropose(context.Context, address.Address, address.Address, address.Address, address.Address) (cid.Cid, error)
// MsigSwapApprove approves a previously proposed SwapSigner
// It takes the following params: <multisig address>, <sender address of the approve msg>, <proposed message ID>,
// <proposer address>, <old signer> <new signer>
// <proposer address>, <old signer>, <new signer>
MsigSwapApprove(context.Context, address.Address, address.Address, uint64, address.Address, address.Address, address.Address) (cid.Cid, error)
// MsigSwapCancel cancels a previously proposed SwapSigner message
// It takes the following params: <multisig address>, <sender address of the cancel msg>, <proposed message ID>,
// <old signer> <new signer>
// <old signer>, <new signer>
MsigSwapCancel(context.Context, address.Address, address.Address, uint64, address.Address, address.Address) (cid.Cid, error)
MarketEnsureAvailable(context.Context, address.Address, address.Address, types.BigInt) (cid.Cid, error)
@@ -416,6 +451,8 @@ type FullNode interface {
PaychGet(ctx context.Context, from, to address.Address, amt types.BigInt) (*ChannelInfo, error)
PaychGetWaitReady(context.Context, cid.Cid) (address.Address, error)
PaychAvailableFunds(ch address.Address) (*ChannelAvailableFunds, error)
PaychAvailableFundsByFromTo(from, to address.Address) (*ChannelAvailableFunds, error)
PaychList(context.Context) ([]address.Address, error)
PaychStatus(context.Context, address.Address) (*PaychStatus, error)
PaychSettle(context.Context, address.Address) (cid.Cid, error)
@@ -424,10 +461,10 @@ type FullNode interface {
PaychNewPayment(ctx context.Context, from, to address.Address, vouchers []VoucherSpec) (*PaymentInfo, error)
PaychVoucherCheckValid(context.Context, address.Address, *paych.SignedVoucher) error
PaychVoucherCheckSpendable(context.Context, address.Address, *paych.SignedVoucher, []byte, []byte) (bool, error)
PaychVoucherCreate(context.Context, address.Address, types.BigInt, uint64) (*paych.SignedVoucher, error)
PaychVoucherCreate(context.Context, address.Address, types.BigInt, uint64) (*VoucherCreateResult, error)
PaychVoucherAdd(context.Context, address.Address, *paych.SignedVoucher, []byte, types.BigInt) (types.BigInt, error)
PaychVoucherList(context.Context, address.Address) ([]*paych.SignedVoucher, error)
PaychVoucherSubmit(context.Context, address.Address, *paych.SignedVoucher) (cid.Cid, error)
PaychVoucherSubmit(context.Context, address.Address, *paych.SignedVoucher, []byte, []byte) (cid.Cid, error)
}
type FileRef struct {
@@ -533,6 +570,28 @@ type ChannelInfo struct {
WaitSentinel cid.Cid
}
type ChannelAvailableFunds struct {
// Channel is the address of the channel
Channel *address.Address
// From is the from address of the channel (channel creator)
From address.Address
// To is the to address of the channel
To address.Address
// ConfirmedAmt is the amount of funds that have been confirmed on-chain
// for the channel
ConfirmedAmt types.BigInt
// PendingAmt is the amount of funds that are pending confirmation on-chain
PendingAmt types.BigInt
// PendingWaitSentinel can be used with PaychGetWaitReady to wait for
// confirmation of pending funds
PendingWaitSentinel *cid.Cid
// QueuedAmt is the amount that is queued up behind a pending request
QueuedAmt types.BigInt
// VoucherRedeemedAmt is the amount that is redeemed by vouchers on-chain
// and in the local datastore
VoucherReedeemedAmt types.BigInt
}
type PaymentInfo struct {
Channel address.Address
WaitSentinel cid.Cid
@@ -548,6 +607,16 @@ type VoucherSpec struct {
Extra *paych.ModVerifyParams
}
// VoucherCreateResult is the response to calling PaychVoucherCreate
type VoucherCreateResult struct {
// Voucher that was created, or nil if there was an error or if there
// were insufficient funds in the channel
Voucher *paych.SignedVoucher
// Shortfall is the additional amount that would be needed in the channel
// in order to be able to create the voucher
Shortfall types.BigInt
}
type MinerPower struct {
MinerPower power.Claim
TotalPower power.Claim
@@ -664,8 +733,28 @@ const (
StageMessages
StageSyncComplete
StageSyncErrored
StageFetchingMessages
)
func (v SyncStateStage) String() string {
switch v {
case StageHeaders:
return "header sync"
case StagePersistHeaders:
return "persisting headers"
case StageMessages:
return "message sync"
case StageSyncComplete:
return "complete"
case StageSyncErrored:
return "error"
case StageFetchingMessages:
return "fetching messages"
default:
return fmt.Sprintf("<unknown: %d>", v)
}
}
type MpoolChange int
const (
@@ -699,7 +788,7 @@ type CirculatingSupply struct {
type MiningBaseInfo struct {
MinerPower types.BigInt
NetworkPower types.BigInt
Sectors []abi.SectorInfo
Sectors []proof.SectorInfo
WorkerKey address.Address
SectorSize abi.SectorSize
PrevBeaconEntry types.BeaconEntry
@@ -716,7 +805,7 @@ type BlockTemplate struct {
Messages []*types.SignedMessage
Epoch abi.ChainEpoch
Timestamp uint64
WinningPoStProof []abi.PoStProof
WinningPoStProof []proof.PoStProof
}
type DataSize struct {
+14 -12
View File
@@ -11,11 +11,11 @@ import (
"github.com/filecoin-project/go-fil-markets/piecestore"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-fil-markets/storagemarket"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/extern/sector-storage/fsutil"
"github.com/filecoin-project/lotus/extern/sector-storage/stores"
"github.com/filecoin-project/lotus/extern/sector-storage/storiface"
"github.com/filecoin-project/specs-actors/actors/abi"
)
// StorageMiner is a low-level interface to the Filecoin network storage miner node
@@ -73,7 +73,7 @@ type StorageMiner interface {
MarketImportDealData(ctx context.Context, propcid cid.Cid, path string) error
MarketListDeals(ctx context.Context) ([]storagemarket.StorageDeal, error)
MarketListRetrievalDeals(ctx context.Context) ([]retrievalmarket.ProviderDealState, error)
MarketGetDealUpdates(ctx context.Context, d cid.Cid) (<-chan storagemarket.MinerDeal, error)
MarketGetDealUpdates(ctx context.Context) (<-chan storagemarket.MinerDeal, error)
MarketListIncompleteDeals(ctx context.Context) ([]storagemarket.MinerDeal, error)
MarketSetAsk(ctx context.Context, price types.BigInt, verifiedPrice types.BigInt, duration abi.ChainEpoch, minPieceSize abi.PaddedPieceSize, maxPieceSize abi.PaddedPieceSize) error
MarketGetAsk(ctx context.Context) (*storagemarket.SignedStorageAsk, error)
@@ -120,16 +120,18 @@ type SectorLog struct {
}
type SectorInfo struct {
SectorID abi.SectorNumber
State SectorState
CommD *cid.Cid
CommR *cid.Cid
Proof []byte
Deals []abi.DealID
Ticket SealTicket
Seed SealSeed
Retries uint64
ToUpgrade bool
SectorID abi.SectorNumber
State SectorState
CommD *cid.Cid
CommR *cid.Cid
Proof []byte
Deals []abi.DealID
Ticket SealTicket
Seed SealSeed
PreCommitMsg *cid.Cid
CommitMsg *cid.Cid
Retries uint64
ToUpgrade bool
LastErr string
+4 -2
View File
@@ -6,10 +6,10 @@ import (
"github.com/ipfs/go-cid"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/extern/sector-storage/sealtasks"
"github.com/filecoin-project/lotus/extern/sector-storage/stores"
"github.com/filecoin-project/lotus/extern/sector-storage/storiface"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-storage/storage"
"github.com/filecoin-project/lotus/build"
@@ -27,11 +27,13 @@ type WorkerAPI interface {
storage.Sealer
MoveStorage(ctx context.Context, sector abi.SectorID) error
MoveStorage(ctx context.Context, sector abi.SectorID, types stores.SectorFileType) error
UnsealPiece(context.Context, abi.SectorID, storiface.UnpaddedByteIndex, abi.UnpaddedPieceSize, abi.SealRandomness, cid.Cid) error
ReadPiece(context.Context, io.Writer, abi.SectorID, storiface.UnpaddedByteIndex, abi.UnpaddedPieceSize) (bool, error)
StorageAddLocal(ctx context.Context, path string) error
Fetch(context.Context, abi.SectorID, stores.SectorFileType, stores.PathType, stores.AcquireMode) error
Closing(context.Context) (<-chan struct{}, error)
+143 -63
View File
@@ -5,27 +5,32 @@ import (
"io"
"time"
"github.com/filecoin-project/go-state-types/dline"
"github.com/ipfs/go-cid"
metrics "github.com/libp2p/go-libp2p-core/metrics"
"github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/peer"
protocol "github.com/libp2p/go-libp2p-core/protocol"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
"github.com/filecoin-project/go-fil-markets/piecestore"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-fil-markets/storagemarket"
"github.com/filecoin-project/go-jsonrpc/auth"
"github.com/filecoin-project/go-multistore"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/extern/sector-storage/fsutil"
"github.com/filecoin-project/lotus/extern/sector-storage/sealtasks"
"github.com/filecoin-project/lotus/extern/sector-storage/stores"
"github.com/filecoin-project/lotus/extern/sector-storage/storiface"
marketevents "github.com/filecoin-project/lotus/markets/loggers"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/paych"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-storage/storage"
"github.com/filecoin-project/lotus/api"
@@ -42,14 +47,18 @@ type CommonStruct struct {
AuthVerify func(ctx context.Context, token string) ([]auth.Permission, error) `perm:"read"`
AuthNew func(ctx context.Context, perms []auth.Permission) ([]byte, error) `perm:"admin"`
NetConnectedness func(context.Context, peer.ID) (network.Connectedness, error) `perm:"read"`
NetPeers func(context.Context) ([]peer.AddrInfo, error) `perm:"read"`
NetConnect func(context.Context, peer.AddrInfo) error `perm:"write"`
NetAddrsListen func(context.Context) (peer.AddrInfo, error) `perm:"read"`
NetDisconnect func(context.Context, peer.ID) error `perm:"write"`
NetFindPeer func(context.Context, peer.ID) (peer.AddrInfo, error) `perm:"read"`
NetPubsubScores func(context.Context) ([]api.PubsubScore, error) `perm:"read"`
NetAutoNatStatus func(context.Context) (api.NatInfo, error) `perm:"read"`
NetConnectedness func(context.Context, peer.ID) (network.Connectedness, error) `perm:"read"`
NetPeers func(context.Context) ([]peer.AddrInfo, error) `perm:"read"`
NetConnect func(context.Context, peer.AddrInfo) error `perm:"write"`
NetAddrsListen func(context.Context) (peer.AddrInfo, error) `perm:"read"`
NetDisconnect func(context.Context, peer.ID) error `perm:"write"`
NetFindPeer func(context.Context, peer.ID) (peer.AddrInfo, error) `perm:"read"`
NetPubsubScores func(context.Context) ([]api.PubsubScore, error) `perm:"read"`
NetAutoNatStatus func(context.Context) (api.NatInfo, error) `perm:"read"`
NetBandwidthStats func(ctx context.Context) (metrics.Stats, error) `perm:"read"`
NetBandwidthStatsByPeer func(ctx context.Context) (map[string]metrics.Stats, error) `perm:"read"`
NetBandwidthStatsByProtocol func(ctx context.Context) (map[protocol.ID]metrics.Stats, error) `perm:"read"`
NetAgentVersion func(ctx context.Context, p peer.ID) (string, error) `perm:"read"`
ID func(context.Context) (peer.ID, error) `perm:"read"`
Version func(context.Context) (api.Version, error) `perm:"read"`
@@ -86,7 +95,7 @@ type FullNodeStruct struct {
ChainGetNode func(ctx context.Context, p string) (*api.IpldObject, error) `perm:"read"`
ChainGetMessage func(context.Context, cid.Cid) (*types.Message, error) `perm:"read"`
ChainGetPath func(context.Context, types.TipSetKey, types.TipSetKey) ([]*api.HeadChange, error) `perm:"read"`
ChainExport func(context.Context, types.TipSetKey) (<-chan []byte, error) `perm:"read"`
ChainExport func(context.Context, abi.ChainEpoch, types.TipSetKey) (<-chan []byte, error) `perm:"read"`
BeaconGetEntry func(ctx context.Context, epoch abi.ChainEpoch) (*types.BeaconEntry, error) `perm:"read"`
@@ -98,7 +107,9 @@ type FullNodeStruct struct {
SyncState func(context.Context) (*api.SyncState, error) `perm:"read"`
SyncSubmitBlock func(ctx context.Context, blk *types.BlockMsg) error `perm:"write"`
SyncIncomingBlocks func(ctx context.Context) (<-chan *types.BlockHeader, error) `perm:"read"`
SyncCheckpoint func(ctx context.Context, key types.TipSetKey) error `perm:"admin"`
SyncMarkBad func(ctx context.Context, bcid cid.Cid) error `perm:"admin"`
SyncUnmarkBad func(ctx context.Context, bcid cid.Cid) error `perm:"admin"`
SyncCheckBad func(ctx context.Context, bcid cid.Cid) (string, error) `perm:"read"`
MpoolGetConfig func(context.Context) (*types.MpoolConfig, error) `perm:"read"`
@@ -130,35 +141,37 @@ type FullNodeStruct struct {
WalletImport func(context.Context, *types.KeyInfo) (address.Address, error) `perm:"admin"`
WalletDelete func(context.Context, address.Address) error `perm:"write"`
ClientImport func(ctx context.Context, ref api.FileRef) (*api.ImportRes, error) `perm:"admin"`
ClientListImports func(ctx context.Context) ([]api.Import, error) `perm:"write"`
ClientRemoveImport func(ctx context.Context, importID multistore.StoreID) error `perm:"admin"`
ClientHasLocal func(ctx context.Context, root cid.Cid) (bool, error) `perm:"write"`
ClientFindData func(ctx context.Context, root cid.Cid, piece *cid.Cid) ([]api.QueryOffer, error) `perm:"read"`
ClientMinerQueryOffer func(ctx context.Context, miner address.Address, root cid.Cid, piece *cid.Cid) (api.QueryOffer, error) `perm:"read"`
ClientStartDeal func(ctx context.Context, params *api.StartDealParams) (*cid.Cid, error) `perm:"admin"`
ClientGetDealInfo func(context.Context, cid.Cid) (*api.DealInfo, error) `perm:"read"`
ClientListDeals func(ctx context.Context) ([]api.DealInfo, error) `perm:"write"`
ClientRetrieve func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) error `perm:"admin"`
ClientRetrieveWithEvents func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) (<-chan marketevents.RetrievalEvent, error) `perm:"admin"`
ClientQueryAsk func(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.SignedStorageAsk, error) `perm:"read"`
ClientCalcCommP func(ctx context.Context, inpath string) (*api.CommPRet, error) `perm:"read"`
ClientGenCar func(ctx context.Context, ref api.FileRef, outpath string) error `perm:"write"`
ClientDealSize func(ctx context.Context, root cid.Cid) (api.DataSize, error) `perm:"read"`
ClientListDataTransfers func(ctx context.Context) ([]api.DataTransferChannel, error) `perm:"write"`
ClientDataTransferUpdates func(ctx context.Context) (<-chan api.DataTransferChannel, error) `perm:"write"`
ClientImport func(ctx context.Context, ref api.FileRef) (*api.ImportRes, error) `perm:"admin"`
ClientListImports func(ctx context.Context) ([]api.Import, error) `perm:"write"`
ClientRemoveImport func(ctx context.Context, importID multistore.StoreID) error `perm:"admin"`
ClientHasLocal func(ctx context.Context, root cid.Cid) (bool, error) `perm:"write"`
ClientFindData func(ctx context.Context, root cid.Cid, piece *cid.Cid) ([]api.QueryOffer, error) `perm:"read"`
ClientMinerQueryOffer func(ctx context.Context, miner address.Address, root cid.Cid, piece *cid.Cid) (api.QueryOffer, error) `perm:"read"`
ClientStartDeal func(ctx context.Context, params *api.StartDealParams) (*cid.Cid, error) `perm:"admin"`
ClientGetDealInfo func(context.Context, cid.Cid) (*api.DealInfo, error) `perm:"read"`
ClientListDeals func(ctx context.Context) ([]api.DealInfo, error) `perm:"write"`
ClientGetDealUpdates func(ctx context.Context) (<-chan api.DealInfo, error) `perm:"read"`
ClientRetrieve func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) error `perm:"admin"`
ClientRetrieveWithEvents func(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) (<-chan marketevents.RetrievalEvent, error) `perm:"admin"`
ClientQueryAsk func(ctx context.Context, p peer.ID, miner address.Address) (*storagemarket.SignedStorageAsk, error) `perm:"read"`
ClientCalcCommP func(ctx context.Context, inpath string) (*api.CommPRet, error) `perm:"read"`
ClientGenCar func(ctx context.Context, ref api.FileRef, outpath string) error `perm:"write"`
ClientDealSize func(ctx context.Context, root cid.Cid) (api.DataSize, error) `perm:"read"`
ClientListDataTransfers func(ctx context.Context) ([]api.DataTransferChannel, error) `perm:"write"`
ClientDataTransferUpdates func(ctx context.Context) (<-chan api.DataTransferChannel, error) `perm:"write"`
ClientRetrieveTryRestartInsufficientFunds func(ctx context.Context, paymentChannel address.Address) error `perm:"write"`
StateNetworkName func(context.Context) (dtypes.NetworkName, error) `perm:"read"`
StateMinerSectors func(context.Context, address.Address, *abi.BitField, bool, types.TipSetKey) ([]*api.ChainSectorInfo, error) `perm:"read"`
StateMinerSectors func(context.Context, address.Address, *bitfield.BitField, bool, types.TipSetKey) ([]*api.ChainSectorInfo, error) `perm:"read"`
StateMinerActiveSectors func(context.Context, address.Address, types.TipSetKey) ([]*api.ChainSectorInfo, error) `perm:"read"`
StateMinerProvingDeadline func(context.Context, address.Address, types.TipSetKey) (*miner.DeadlineInfo, error) `perm:"read"`
StateMinerProvingDeadline func(context.Context, address.Address, types.TipSetKey) (*dline.Info, error) `perm:"read"`
StateMinerPower func(context.Context, address.Address, types.TipSetKey) (*api.MinerPower, error) `perm:"read"`
StateMinerInfo func(context.Context, address.Address, types.TipSetKey) (api.MinerInfo, error) `perm:"read"`
StateMinerDeadlines func(context.Context, address.Address, types.TipSetKey) ([]*miner.Deadline, error) `perm:"read"`
StateMinerPartitions func(context.Context, address.Address, uint64, types.TipSetKey) ([]*miner.Partition, error) `perm:"read"`
StateMinerFaults func(context.Context, address.Address, types.TipSetKey) (abi.BitField, error) `perm:"read"`
StateMinerFaults func(context.Context, address.Address, types.TipSetKey) (bitfield.BitField, error) `perm:"read"`
StateAllMinerFaults func(context.Context, abi.ChainEpoch, types.TipSetKey) ([]*api.Fault, error) `perm:"read"`
StateMinerRecoveries func(context.Context, address.Address, types.TipSetKey) (abi.BitField, error) `perm:"read"`
StateMinerRecoveries func(context.Context, address.Address, types.TipSetKey) (bitfield.BitField, error) `perm:"read"`
StateMinerPreCommitDepositForPower func(context.Context, address.Address, miner.SectorPreCommitInfo, types.TipSetKey) (types.BigInt, error) `perm:"read"`
StateMinerInitialPledgeCollateral func(context.Context, address.Address, miner.SectorPreCommitInfo, types.TipSetKey) (types.BigInt, error) `perm:"read"`
StateMinerAvailableBalance func(context.Context, address.Address, types.TipSetKey) (types.BigInt, error) `perm:"read"`
@@ -190,31 +203,37 @@ type FullNodeStruct struct {
StateCirculatingSupply func(context.Context, types.TipSetKey) (api.CirculatingSupply, error) `perm:"read"`
MsigGetAvailableBalance func(context.Context, address.Address, types.TipSetKey) (types.BigInt, error) `perm:"read"`
MsigGetVested func(context.Context, address.Address, types.TipSetKey, types.TipSetKey) (types.BigInt, error) `perm:"read"`
MsigCreate func(context.Context, uint64, []address.Address, abi.ChainEpoch, types.BigInt, address.Address, types.BigInt) (cid.Cid, error) `perm:"sign"`
MsigPropose func(context.Context, address.Address, address.Address, types.BigInt, address.Address, uint64, []byte) (cid.Cid, error) `perm:"sign"`
MsigApprove func(context.Context, address.Address, uint64, address.Address, address.Address, types.BigInt, address.Address, uint64, []byte) (cid.Cid, error) `perm:"sign"`
MsigCancel func(context.Context, address.Address, uint64, address.Address, types.BigInt, address.Address, uint64, []byte) (cid.Cid, error) `perm:"sign"`
MsigAddPropose func(context.Context, address.Address, address.Address, address.Address, bool) (cid.Cid, error) `perm:"sign"`
MsigAddApprove func(context.Context, address.Address, address.Address, uint64, address.Address, address.Address, bool) (cid.Cid, error) `perm:"sign"`
MsigAddCancel func(context.Context, address.Address, address.Address, uint64, address.Address, bool) (cid.Cid, error) `perm:"sign"`
MsigSwapPropose func(context.Context, address.Address, address.Address, address.Address, address.Address) (cid.Cid, error) `perm:"sign"`
MsigSwapApprove func(context.Context, address.Address, address.Address, uint64, address.Address, address.Address, address.Address) (cid.Cid, error) `perm:"sign"`
MsigSwapCancel func(context.Context, address.Address, address.Address, uint64, address.Address, address.Address) (cid.Cid, error) `perm:"sign"`
MarketEnsureAvailable func(context.Context, address.Address, address.Address, types.BigInt) (cid.Cid, error) `perm:"sign"`
PaychGet func(ctx context.Context, from, to address.Address, amt types.BigInt) (*api.ChannelInfo, error) `perm:"sign"`
PaychGetWaitReady func(context.Context, cid.Cid) (address.Address, error) `perm:"sign"`
PaychList func(context.Context) ([]address.Address, error) `perm:"read"`
PaychStatus func(context.Context, address.Address) (*api.PaychStatus, error) `perm:"read"`
PaychSettle func(context.Context, address.Address) (cid.Cid, error) `perm:"sign"`
PaychCollect func(context.Context, address.Address) (cid.Cid, error) `perm:"sign"`
PaychAllocateLane func(context.Context, address.Address) (uint64, error) `perm:"sign"`
PaychNewPayment func(ctx context.Context, from, to address.Address, vouchers []api.VoucherSpec) (*api.PaymentInfo, error) `perm:"sign"`
PaychVoucherCheck func(context.Context, *paych.SignedVoucher) error `perm:"read"`
PaychVoucherCheckValid func(context.Context, address.Address, *paych.SignedVoucher) error `perm:"read"`
PaychVoucherCheckSpendable func(context.Context, address.Address, *paych.SignedVoucher, []byte, []byte) (bool, error) `perm:"read"`
PaychVoucherAdd func(context.Context, address.Address, *paych.SignedVoucher, []byte, types.BigInt) (types.BigInt, error) `perm:"write"`
PaychVoucherCreate func(context.Context, address.Address, big.Int, uint64) (*paych.SignedVoucher, error) `perm:"sign"`
PaychVoucherList func(context.Context, address.Address) ([]*paych.SignedVoucher, error) `perm:"write"`
PaychVoucherSubmit func(context.Context, address.Address, *paych.SignedVoucher) (cid.Cid, error) `perm:"sign"`
PaychGet func(ctx context.Context, from, to address.Address, amt types.BigInt) (*api.ChannelInfo, error) `perm:"sign"`
PaychGetWaitReady func(context.Context, cid.Cid) (address.Address, error) `perm:"sign"`
PaychAvailableFunds func(address.Address) (*api.ChannelAvailableFunds, error) `perm:"sign"`
PaychAvailableFundsByFromTo func(address.Address, address.Address) (*api.ChannelAvailableFunds, error) `perm:"sign"`
PaychList func(context.Context) ([]address.Address, error) `perm:"read"`
PaychStatus func(context.Context, address.Address) (*api.PaychStatus, error) `perm:"read"`
PaychSettle func(context.Context, address.Address) (cid.Cid, error) `perm:"sign"`
PaychCollect func(context.Context, address.Address) (cid.Cid, error) `perm:"sign"`
PaychAllocateLane func(context.Context, address.Address) (uint64, error) `perm:"sign"`
PaychNewPayment func(ctx context.Context, from, to address.Address, vouchers []api.VoucherSpec) (*api.PaymentInfo, error) `perm:"sign"`
PaychVoucherCheck func(context.Context, *paych.SignedVoucher) error `perm:"read"`
PaychVoucherCheckValid func(context.Context, address.Address, *paych.SignedVoucher) error `perm:"read"`
PaychVoucherCheckSpendable func(context.Context, address.Address, *paych.SignedVoucher, []byte, []byte) (bool, error) `perm:"read"`
PaychVoucherAdd func(context.Context, address.Address, *paych.SignedVoucher, []byte, types.BigInt) (types.BigInt, error) `perm:"write"`
PaychVoucherCreate func(context.Context, address.Address, big.Int, uint64) (*api.VoucherCreateResult, error) `perm:"sign"`
PaychVoucherList func(context.Context, address.Address) ([]*paych.SignedVoucher, error) `perm:"write"`
PaychVoucherSubmit func(context.Context, address.Address, *paych.SignedVoucher, []byte, []byte) (cid.Cid, error) `perm:"sign"`
}
}
@@ -234,7 +253,7 @@ type StorageMinerStruct struct {
MarketImportDealData func(context.Context, cid.Cid, string) error `perm:"write"`
MarketListDeals func(ctx context.Context) ([]storagemarket.StorageDeal, error) `perm:"read"`
MarketListRetrievalDeals func(ctx context.Context) ([]retrievalmarket.ProviderDealState, error) `perm:"read"`
MarketGetDealUpdates func(ctx context.Context, d cid.Cid) (<-chan storagemarket.MinerDeal, error) `perm:"read"`
MarketGetDealUpdates func(ctx context.Context) (<-chan storagemarket.MinerDeal, error) `perm:"read"`
MarketListIncompleteDeals func(ctx context.Context) ([]storagemarket.MinerDeal, error) `perm:"read"`
MarketSetAsk func(ctx context.Context, price types.BigInt, verifiedPrice types.BigInt, duration abi.ChainEpoch, minPieceSize abi.PaddedPieceSize, maxPieceSize abi.PaddedPieceSize) error `perm:"admin"`
MarketGetAsk func(ctx context.Context) (*storagemarket.SignedStorageAsk, error) `perm:"read"`
@@ -316,7 +335,8 @@ type WorkerStruct struct {
FinalizeSector func(context.Context, abi.SectorID, []storage.Range) error `perm:"admin"`
ReleaseUnsealed func(ctx context.Context, sector abi.SectorID, safeToFree []storage.Range) error `perm:"admin"`
Remove func(ctx context.Context, sector abi.SectorID) error `perm:"admin"`
MoveStorage func(ctx context.Context, sector abi.SectorID) error `perm:"admin"`
MoveStorage func(ctx context.Context, sector abi.SectorID, types stores.SectorFileType) error `perm:"admin"`
StorageAddLocal func(ctx context.Context, path string) error `perm:"admin"`
UnsealPiece func(context.Context, abi.SectorID, storiface.UnpaddedByteIndex, abi.UnpaddedPieceSize, abi.SealRandomness, cid.Cid) error `perm:"admin"`
ReadPiece func(context.Context, io.Writer, abi.SectorID, storiface.UnpaddedByteIndex, abi.UnpaddedPieceSize) (bool, error) `perm:"admin"`
@@ -369,6 +389,22 @@ func (c *CommonStruct) NetAutoNatStatus(ctx context.Context) (api.NatInfo, error
return c.Internal.NetAutoNatStatus(ctx)
}
func (c *CommonStruct) NetBandwidthStats(ctx context.Context) (metrics.Stats, error) {
return c.Internal.NetBandwidthStats(ctx)
}
func (c *CommonStruct) NetBandwidthStatsByPeer(ctx context.Context) (map[string]metrics.Stats, error) {
return c.Internal.NetBandwidthStatsByPeer(ctx)
}
func (c *CommonStruct) NetBandwidthStatsByProtocol(ctx context.Context) (map[protocol.ID]metrics.Stats, error) {
return c.Internal.NetBandwidthStatsByProtocol(ctx)
}
func (c *CommonStruct) NetAgentVersion(ctx context.Context, p peer.ID) (string, error) {
return c.Internal.NetAgentVersion(ctx, p)
}
// ID implements API.ID
func (c *CommonStruct) ID(ctx context.Context) (peer.ID, error) {
return c.Internal.ID(ctx)
@@ -433,6 +469,10 @@ func (c *FullNodeStruct) ClientListDeals(ctx context.Context) ([]api.DealInfo, e
return c.Internal.ClientListDeals(ctx)
}
func (c *FullNodeStruct) ClientGetDealUpdates(ctx context.Context) (<-chan api.DealInfo, error) {
return c.Internal.ClientGetDealUpdates(ctx)
}
func (c *FullNodeStruct) ClientRetrieve(ctx context.Context, order api.RetrievalOrder, ref *api.FileRef) error {
return c.Internal.ClientRetrieve(ctx, order, ref)
}
@@ -464,6 +504,10 @@ func (c *FullNodeStruct) ClientDataTransferUpdates(ctx context.Context) (<-chan
return c.Internal.ClientDataTransferUpdates(ctx)
}
func (c *FullNodeStruct) ClientRetrieveTryRestartInsufficientFunds(ctx context.Context, paymentChannel address.Address) error {
return c.Internal.ClientRetrieveTryRestartInsufficientFunds(ctx, paymentChannel)
}
func (c *FullNodeStruct) GasEstimateGasPremium(ctx context.Context, nblocksincl uint64, sender address.Address, gaslimit int64, tsk types.TipSetKey) (types.BigInt, error) {
return c.Internal.GasEstimateGasPremium(ctx, nblocksincl, sender, gaslimit, tsk)
}
@@ -648,8 +692,8 @@ func (c *FullNodeStruct) ChainGetPath(ctx context.Context, from types.TipSetKey,
return c.Internal.ChainGetPath(ctx, from, to)
}
func (c *FullNodeStruct) ChainExport(ctx context.Context, tsk types.TipSetKey) (<-chan []byte, error) {
return c.Internal.ChainExport(ctx, tsk)
func (c *FullNodeStruct) ChainExport(ctx context.Context, nroots abi.ChainEpoch, tsk types.TipSetKey) (<-chan []byte, error) {
return c.Internal.ChainExport(ctx, nroots, tsk)
}
func (c *FullNodeStruct) BeaconGetEntry(ctx context.Context, epoch abi.ChainEpoch) (*types.BeaconEntry, error) {
@@ -668,10 +712,18 @@ func (c *FullNodeStruct) SyncIncomingBlocks(ctx context.Context) (<-chan *types.
return c.Internal.SyncIncomingBlocks(ctx)
}
func (c *FullNodeStruct) SyncCheckpoint(ctx context.Context, tsk types.TipSetKey) error {
return c.Internal.SyncCheckpoint(ctx, tsk)
}
func (c *FullNodeStruct) SyncMarkBad(ctx context.Context, bcid cid.Cid) error {
return c.Internal.SyncMarkBad(ctx, bcid)
}
func (c *FullNodeStruct) SyncUnmarkBad(ctx context.Context, bcid cid.Cid) error {
return c.Internal.SyncUnmarkBad(ctx, bcid)
}
func (c *FullNodeStruct) SyncCheckBad(ctx context.Context, bcid cid.Cid) (string, error) {
return c.Internal.SyncCheckBad(ctx, bcid)
}
@@ -680,7 +732,7 @@ func (c *FullNodeStruct) StateNetworkName(ctx context.Context) (dtypes.NetworkNa
return c.Internal.StateNetworkName(ctx)
}
func (c *FullNodeStruct) StateMinerSectors(ctx context.Context, addr address.Address, filter *abi.BitField, filterOut bool, tsk types.TipSetKey) ([]*api.ChainSectorInfo, error) {
func (c *FullNodeStruct) StateMinerSectors(ctx context.Context, addr address.Address, filter *bitfield.BitField, filterOut bool, tsk types.TipSetKey) ([]*api.ChainSectorInfo, error) {
return c.Internal.StateMinerSectors(ctx, addr, filter, filterOut, tsk)
}
@@ -688,7 +740,7 @@ func (c *FullNodeStruct) StateMinerActiveSectors(ctx context.Context, addr addre
return c.Internal.StateMinerActiveSectors(ctx, addr, tsk)
}
func (c *FullNodeStruct) StateMinerProvingDeadline(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*miner.DeadlineInfo, error) {
func (c *FullNodeStruct) StateMinerProvingDeadline(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*dline.Info, error) {
return c.Internal.StateMinerProvingDeadline(ctx, addr, tsk)
}
@@ -708,7 +760,7 @@ func (c *FullNodeStruct) StateMinerPartitions(ctx context.Context, m address.Add
return c.Internal.StateMinerPartitions(ctx, m, dlIdx, tsk)
}
func (c *FullNodeStruct) StateMinerFaults(ctx context.Context, actor address.Address, tsk types.TipSetKey) (abi.BitField, error) {
func (c *FullNodeStruct) StateMinerFaults(ctx context.Context, actor address.Address, tsk types.TipSetKey) (bitfield.BitField, error) {
return c.Internal.StateMinerFaults(ctx, actor, tsk)
}
@@ -716,7 +768,7 @@ func (c *FullNodeStruct) StateAllMinerFaults(ctx context.Context, cutoff abi.Cha
return c.Internal.StateAllMinerFaults(ctx, cutoff, endTsk)
}
func (c *FullNodeStruct) StateMinerRecoveries(ctx context.Context, actor address.Address, tsk types.TipSetKey) (abi.BitField, error) {
func (c *FullNodeStruct) StateMinerRecoveries(ctx context.Context, actor address.Address, tsk types.TipSetKey) (bitfield.BitField, error) {
return c.Internal.StateMinerRecoveries(ctx, actor, tsk)
}
@@ -836,6 +888,10 @@ func (c *FullNodeStruct) MsigGetAvailableBalance(ctx context.Context, a address.
return c.Internal.MsigGetAvailableBalance(ctx, a, tsk)
}
func (c *FullNodeStruct) MsigGetVested(ctx context.Context, a address.Address, sTsk types.TipSetKey, eTsk types.TipSetKey) (types.BigInt, error) {
return c.Internal.MsigGetVested(ctx, a, sTsk, eTsk)
}
func (c *FullNodeStruct) MsigCreate(ctx context.Context, req uint64, addrs []address.Address, duration abi.ChainEpoch, val types.BigInt, src address.Address, gp types.BigInt) (cid.Cid, error) {
return c.Internal.MsigCreate(ctx, req, addrs, duration, val, src, gp)
}
@@ -852,6 +908,18 @@ func (c *FullNodeStruct) MsigCancel(ctx context.Context, msig address.Address, t
return c.Internal.MsigCancel(ctx, msig, txID, to, amt, src, method, params)
}
func (c *FullNodeStruct) MsigAddPropose(ctx context.Context, msig address.Address, src address.Address, newAdd address.Address, inc bool) (cid.Cid, error) {
return c.Internal.MsigAddPropose(ctx, msig, src, newAdd, inc)
}
func (c *FullNodeStruct) MsigAddApprove(ctx context.Context, msig address.Address, src address.Address, txID uint64, proposer address.Address, newAdd address.Address, inc bool) (cid.Cid, error) {
return c.Internal.MsigAddApprove(ctx, msig, src, txID, proposer, newAdd, inc)
}
func (c *FullNodeStruct) MsigAddCancel(ctx context.Context, msig address.Address, src address.Address, txID uint64, newAdd address.Address, inc bool) (cid.Cid, error) {
return c.Internal.MsigAddCancel(ctx, msig, src, txID, newAdd, inc)
}
func (c *FullNodeStruct) MsigSwapPropose(ctx context.Context, msig address.Address, src address.Address, oldAdd address.Address, newAdd address.Address) (cid.Cid, error) {
return c.Internal.MsigSwapPropose(ctx, msig, src, oldAdd, newAdd)
}
@@ -876,6 +944,14 @@ func (c *FullNodeStruct) PaychGetWaitReady(ctx context.Context, sentinel cid.Cid
return c.Internal.PaychGetWaitReady(ctx, sentinel)
}
func (c *FullNodeStruct) PaychAvailableFunds(ch address.Address) (*api.ChannelAvailableFunds, error) {
return c.Internal.PaychAvailableFunds(ch)
}
func (c *FullNodeStruct) PaychAvailableFundsByFromTo(from, to address.Address) (*api.ChannelAvailableFunds, error) {
return c.Internal.PaychAvailableFundsByFromTo(from, to)
}
func (c *FullNodeStruct) PaychList(ctx context.Context) ([]address.Address, error) {
return c.Internal.PaychList(ctx)
}
@@ -896,7 +972,7 @@ func (c *FullNodeStruct) PaychVoucherAdd(ctx context.Context, addr address.Addre
return c.Internal.PaychVoucherAdd(ctx, addr, sv, proof, minDelta)
}
func (c *FullNodeStruct) PaychVoucherCreate(ctx context.Context, pch address.Address, amt types.BigInt, lane uint64) (*paych.SignedVoucher, error) {
func (c *FullNodeStruct) PaychVoucherCreate(ctx context.Context, pch address.Address, amt types.BigInt, lane uint64) (*api.VoucherCreateResult, error) {
return c.Internal.PaychVoucherCreate(ctx, pch, amt, lane)
}
@@ -920,8 +996,8 @@ func (c *FullNodeStruct) PaychNewPayment(ctx context.Context, from, to address.A
return c.Internal.PaychNewPayment(ctx, from, to, vouchers)
}
func (c *FullNodeStruct) PaychVoucherSubmit(ctx context.Context, ch address.Address, sv *paych.SignedVoucher) (cid.Cid, error) {
return c.Internal.PaychVoucherSubmit(ctx, ch, sv)
func (c *FullNodeStruct) PaychVoucherSubmit(ctx context.Context, ch address.Address, sv *paych.SignedVoucher, secret []byte, proof []byte) (cid.Cid, error) {
return c.Internal.PaychVoucherSubmit(ctx, ch, sv, secret, proof)
}
// StorageMinerStruct
@@ -1064,8 +1140,8 @@ func (c *StorageMinerStruct) MarketListRetrievalDeals(ctx context.Context) ([]re
return c.Internal.MarketListRetrievalDeals(ctx)
}
func (c *StorageMinerStruct) MarketGetDealUpdates(ctx context.Context, d cid.Cid) (<-chan storagemarket.MinerDeal, error) {
return c.Internal.MarketGetDealUpdates(ctx, d)
func (c *StorageMinerStruct) MarketGetDealUpdates(ctx context.Context) (<-chan storagemarket.MinerDeal, error) {
return c.Internal.MarketGetDealUpdates(ctx)
}
func (c *StorageMinerStruct) MarketListIncompleteDeals(ctx context.Context) ([]storagemarket.MinerDeal, error) {
@@ -1214,8 +1290,12 @@ func (w *WorkerStruct) Remove(ctx context.Context, sector abi.SectorID) error {
return w.Internal.Remove(ctx, sector)
}
func (w *WorkerStruct) MoveStorage(ctx context.Context, sector abi.SectorID) error {
return w.Internal.MoveStorage(ctx, sector)
func (w *WorkerStruct) MoveStorage(ctx context.Context, sector abi.SectorID, types stores.SectorFileType) error {
return w.Internal.MoveStorage(ctx, sector, types)
}
func (w *WorkerStruct) StorageAddLocal(ctx context.Context, path string) error {
return w.Internal.StorageAddLocal(ctx, path)
}
func (w *WorkerStruct) UnsealPiece(ctx context.Context, id abi.SectorID, index storiface.UnpaddedByteIndex, size abi.UnpaddedPieceSize, randomness abi.SealRandomness, c cid.Cid) error {
+1 -1
View File
@@ -6,7 +6,7 @@ import (
"fmt"
"io"
abi "github.com/filecoin-project/specs-actors/actors/abi"
abi "github.com/filecoin-project/go-state-types/abi"
paych "github.com/filecoin-project/specs-actors/actors/builtin/paych"
cbg "github.com/whyrusleeping/cbor-gen"
xerrors "golang.org/x/xerrors"
+22 -4
View File
@@ -14,8 +14,10 @@ import (
"github.com/ipfs/go-cid"
"github.com/ipfs/go-filestore"
metrics "github.com/libp2p/go-libp2p-core/metrics"
"github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/peer"
protocol "github.com/libp2p/go-libp2p-core/protocol"
pubsub "github.com/libp2p/go-libp2p-pubsub"
"github.com/multiformats/go-multiaddr"
@@ -26,9 +28,9 @@ import (
"github.com/filecoin-project/go-jsonrpc/auth"
"github.com/filecoin-project/go-multistore"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/go-state-types/exitcode"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/api/apistruct"
@@ -103,7 +105,7 @@ func init() {
addExample(network.Connected)
addExample(dtypes.NetworkName("lotus"))
addExample(api.SyncStateStage(1))
addExample(build.APIVersion)
addExample(build.FullAPIVersion)
addExample(api.PCHInbound)
addExample(time.Minute)
addExample(datatransfer.TransferID(3))
@@ -133,6 +135,22 @@ func init() {
InvalidMessageDeliveries: 3,
},
})
addExample(map[string]metrics.Stats{
"12D3KooWSXmXLJmBR1M7i9RW9GQPNUhZSzXKzxDHWtAgNuJAbyEJ": {
RateIn: 100,
RateOut: 50,
TotalIn: 174000,
TotalOut: 12500,
},
})
addExample(map[protocol.ID]metrics.Stats{
"/fil/hello/1.0.0": {
RateIn: 100,
RateOut: 50,
TotalIn: 174000,
TotalOut: 12500,
},
})
maddr, err := multiaddr.NewMultiaddr("/ip4/52.36.61.156/tcp/1347/p2p/12D3KooWFETiESTf1v4PGUvtnxMAcEFMzLZbJGg4tjWfGEimYior")
if err != nil {
+2 -2
View File
@@ -7,8 +7,8 @@ import (
"testing"
"time"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/miner"
"github.com/filecoin-project/specs-actors/actors/abi"
)
type BlockMiner struct {
@@ -41,7 +41,7 @@ func (bm *BlockMiner) MineBlocks() {
nulls := atomic.SwapInt64(&bm.nulls, 0)
if err := bm.miner.MineOne(bm.ctx, miner.MineReq{
InjectNulls: abi.ChainEpoch(nulls),
Done: func(bool, error) {},
Done: func(bool, abi.ChainEpoch, error) {},
}); err != nil {
bm.t.Error(err)
}
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/node/impl"
+16 -13
View File
@@ -20,6 +20,7 @@ import (
"github.com/ipld/go-car"
"github.com/filecoin-project/go-fil-markets/storagemarket"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
sealing "github.com/filecoin-project/lotus/extern/storage-sealing"
@@ -35,7 +36,7 @@ import (
var MineNext = miner.MineReq{
InjectNulls: 0,
Done: func(bool, error) {},
Done: func(bool, abi.ChainEpoch, error) {},
}
func init() {
@@ -333,7 +334,7 @@ loop:
func waitDealPublished(t *testing.T, ctx context.Context, miner TestStorageNode, deal *cid.Cid) {
subCtx, cancel := context.WithCancel(ctx)
defer cancel()
updates, err := miner.MarketGetDealUpdates(subCtx, *deal)
updates, err := miner.MarketGetDealUpdates(subCtx)
if err != nil {
t.Fatal(err)
}
@@ -342,18 +343,20 @@ func waitDealPublished(t *testing.T, ctx context.Context, miner TestStorageNode,
case <-ctx.Done():
t.Fatal("context timeout")
case di := <-updates:
switch di.State {
case storagemarket.StorageDealProposalRejected:
t.Fatal("deal rejected")
case storagemarket.StorageDealFailing:
t.Fatal("deal failed")
case storagemarket.StorageDealError:
t.Fatal("deal errored", di.Message)
case storagemarket.StorageDealFinalizing, storagemarket.StorageDealSealing, storagemarket.StorageDealActive:
fmt.Println("COMPLETE", di)
return
if deal.Equals(di.ProposalCid) {
switch di.State {
case storagemarket.StorageDealProposalRejected:
t.Fatal("deal rejected")
case storagemarket.StorageDealFailing:
t.Fatal("deal failed")
case storagemarket.StorageDealError:
t.Fatal("deal errored", di.Message)
case storagemarket.StorageDealFinalizing, storagemarket.StorageDealSealing, storagemarket.StorageDealActive:
fmt.Println("COMPLETE", di)
return
}
fmt.Println("Deal state: ", storagemarket.DealStates[di.State])
}
fmt.Println("Deal state: ", storagemarket.DealStates[di.State])
}
}
}
+8 -10
View File
@@ -12,7 +12,7 @@ import (
logging "github.com/ipfs/go-log/v2"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/stretchr/testify/require"
"github.com/filecoin-project/lotus/build"
@@ -31,22 +31,20 @@ func (ts *testSuite) testMining(t *testing.T) {
newHeads, err := api.ChainNotify(ctx)
require.NoError(t, err)
initHead := (<-newHeads)[0]
if initHead.Val.Height() != 2 {
<-newHeads
}
baseHeight := initHead.Val.Height()
h1, err := api.ChainHead(ctx)
require.NoError(t, err)
require.Equal(t, abi.ChainEpoch(2), h1.Height())
require.Equal(t, int64(h1.Height()), int64(baseHeight))
err = sn[0].MineOne(ctx, MineNext)
MineUntilBlock(ctx, t, apis[0], sn[0], nil)
require.NoError(t, err)
<-newHeads
h2, err := api.ChainHead(ctx)
require.NoError(t, err)
require.Equal(t, abi.ChainEpoch(3), h2.Height())
require.Greater(t, int64(h2.Height()), int64(h1.Height()))
}
func (ts *testSuite) testMiningReal(t *testing.T) {
@@ -70,7 +68,7 @@ func (ts *testSuite) testMiningReal(t *testing.T) {
require.NoError(t, err)
require.Equal(t, abi.ChainEpoch(2), h1.Height())
err = sn[0].MineOne(ctx, MineNext)
MineUntilBlock(ctx, t, apis[0], sn[0], nil)
require.NoError(t, err)
<-newHeads
@@ -79,7 +77,7 @@ func (ts *testSuite) testMiningReal(t *testing.T) {
require.NoError(t, err)
require.Equal(t, abi.ChainEpoch(3), h2.Height())
err = sn[0].MineOne(ctx, MineNext)
MineUntilBlock(ctx, t, apis[0], sn[0], nil)
require.NoError(t, err)
<-newHeads
@@ -144,7 +142,7 @@ func TestDealMining(t *testing.T, b APIBuilder, blocktime time.Duration, carExpo
complChan := minedTwo
for atomic.LoadInt32(&mine) != 0 {
wait := make(chan int)
mdone := func(mined bool, err error) {
mdone := func(mined bool, _ abi.ChainEpoch, err error) {
n := 0
if mined {
n = 1
+10 -4
View File
@@ -10,8 +10,8 @@ import (
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin/paych"
"github.com/ipfs/go-cid"
@@ -96,18 +96,24 @@ func TestPaymentChannels(t *testing.T, b APIBuilder, blocktime time.Duration) {
if err != nil {
t.Fatal(err)
}
if vouch1.Voucher == nil {
t.Fatal(fmt.Errorf("Not enough funds to create voucher: missing %d", vouch1.Shortfall))
}
vouch2, err := paymentCreator.PaychVoucherCreate(ctx, channel, abi.NewTokenAmount(2000), lane)
if err != nil {
t.Fatal(err)
}
delta1, err := paymentReceiver.PaychVoucherAdd(ctx, channel, vouch1, nil, abi.NewTokenAmount(1000))
if vouch2.Voucher == nil {
t.Fatal(fmt.Errorf("Not enough funds to create voucher: missing %d", vouch2.Shortfall))
}
delta1, err := paymentReceiver.PaychVoucherAdd(ctx, channel, vouch1.Voucher, nil, abi.NewTokenAmount(1000))
if err != nil {
t.Fatal(err)
}
if !delta1.Equals(abi.NewTokenAmount(1000)) {
t.Fatal("voucher didn't have the right amount")
}
delta2, err := paymentReceiver.PaychVoucherAdd(ctx, channel, vouch2, nil, abi.NewTokenAmount(1000))
delta2, err := paymentReceiver.PaychVoucherAdd(ctx, channel, vouch2.Voucher, nil, abi.NewTokenAmount(1000))
if err != nil {
t.Fatal(err)
}
+2
View File
@@ -65,6 +65,8 @@ func TestApis(t *testing.T, b APIBuilder) {
var OneMiner = []StorageMiner{{Full: 0, Preseal: PresealGenesis}}
func (ts *testSuite) testVersion(t *testing.T) {
build.RunningNodeType = build.NodeFull
ctx := context.Background()
apis, _ := ts.makeNodes(t, 1, OneMiner)
api := apis[0]
+51 -1
View File
@@ -3,11 +3,13 @@ package test
import (
"context"
"testing"
"time"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/miner"
)
func SendFunds(ctx context.Context, t *testing.T, sender TestNode, addr address.Address, amount abi.TokenAmount) {
@@ -34,3 +36,51 @@ func SendFunds(ctx context.Context, t *testing.T, sender TestNode, addr address.
t.Fatal("did not successfully send money")
}
}
func MineUntilBlock(ctx context.Context, t *testing.T, fn TestNode, sn TestStorageNode, cb func(abi.ChainEpoch)) {
for i := 0; i < 1000; i++ {
var success bool
var err error
var epoch abi.ChainEpoch
wait := make(chan struct{})
mineErr := sn.MineOne(ctx, miner.MineReq{
Done: func(win bool, ep abi.ChainEpoch, e error) {
success = win
err = e
epoch = ep
wait <- struct{}{}
},
})
if mineErr != nil {
t.Fatal(mineErr)
}
<-wait
if err != nil {
t.Fatal(err)
}
if success {
// Wait until it shows up on the given full nodes ChainHead
nloops := 50
for i := 0; i < nloops; i++ {
ts, err := fn.ChainHead(ctx)
if err != nil {
t.Fatal(err)
}
if ts.Height() == epoch {
break
}
if i == nloops-1 {
t.Fatal("block never managed to sync to node")
}
time.Sleep(time.Millisecond * 10)
}
if cb != nil {
cb(epoch)
}
return
}
t.Log("did not mine block, trying again", i)
}
t.Fatal("failed to mine 1000 times in a row...")
}
+6 -2
View File
@@ -12,9 +12,9 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/extern/sector-storage/mock"
sealing "github.com/filecoin-project/lotus/extern/storage-sealing"
"github.com/filecoin-project/specs-actors/actors/abi"
miner2 "github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/lotus/api"
@@ -53,7 +53,7 @@ func TestPledgeSector(t *testing.T, b APIBuilder, blocktime time.Duration, nSect
defer close(done)
for mine {
build.Clock.Sleep(blocktime)
if err := sn[0].MineOne(ctx, bminer.MineReq{Done: func(bool, error) {
if err := sn[0].MineOne(ctx, bminer.MineReq{Done: func(bool, abi.ChainEpoch, error) {
}}); err != nil {
t.Error(err)
@@ -216,6 +216,10 @@ func TestWindowPost(t *testing.T, b APIBuilder, blocktime time.Duration, nSector
sn, err := parts[0].Sectors.First()
require.NoError(t, err)
all, err := parts[0].Sectors.All(2)
require.NoError(t, err)
fmt.Println("the sectors", all)
s = abi.SectorID{
Miner: abi.ActorID(mid),
Number: abi.SectorNumber(sn),
+4 -3
View File
@@ -6,8 +6,8 @@ import (
"github.com/filecoin-project/go-address"
datatransfer "github.com/filecoin-project/go-data-transfer"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/ipfs/go-cid"
@@ -96,7 +96,8 @@ type MessageSendSpec struct {
}
var DefaultMessageSendSpec = MessageSendSpec{
MaxFee: big.Zero(),
// MaxFee of 0.1FIL
MaxFee: abi.NewTokenAmount(int64(build.FilecoinPrecision) / 10),
}
func (ms *MessageSendSpec) Get() MessageSendSpec {
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"context"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/crypto"
)
type SignFunc = func(context.Context, []byte) (*crypto.Signature, error)
+17 -6
View File
@@ -1,15 +1,26 @@
package build
import "github.com/filecoin-project/lotus/node/modules/dtypes"
import (
"sort"
var DrandNetwork = DrandIncentinet
func DrandConfig() dtypes.DrandConfig {
return DrandConfigs[DrandNetwork]
}
"github.com/filecoin-project/lotus/node/modules/dtypes"
)
type DrandEnum int
func DrandConfigSchedule() dtypes.DrandSchedule {
out := dtypes.DrandSchedule{}
for start, config := range DrandSchedule {
out = append(out, dtypes.DrandPoint{Start: start, Config: DrandConfigs[config]})
}
sort.Slice(out, func(i, j int) bool {
return out[i].Start < out[j].Start
})
return out
}
const (
DrandMainnet DrandEnum = iota + 1
DrandTestnet
+12 -3
View File
@@ -3,13 +3,22 @@
package build
import (
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
)
const UpgradeBreezeHeight = -1
const BreezeGasTampingDuration = 0
const UpgradeSmokeHeight = -1
var DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandMainnet,
}
func init() {
power.ConsensusMinerMinPower = big.NewInt(2048)
miner.SupportedProofTypes = map[abi.RegisteredSealProof]struct{}{
@@ -20,7 +29,7 @@ func init() {
BuildType |= Build2k
}
const BlockDelaySecs = uint64(4)
const BlockDelaySecs = uint64(30)
const PropagationDelaySecs = uint64(1)
+9 -1
View File
@@ -5,7 +5,7 @@ import (
"github.com/libp2p/go-libp2p-core/protocol"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/lotus/node/modules/dtypes"
@@ -36,3 +36,11 @@ func MessagesTopic(netName dtypes.NetworkName) string { return "/fil/msgs/" + st
func DhtProtocolName(netName dtypes.NetworkName) protocol.ID {
return protocol.ID("/fil/kad/" + string(netName))
}
func UseNewestNetwork() bool {
// TODO: Put these in a container we can iterate over
if UpgradeBreezeHeight <= 0 && UpgradeSmokeHeight <= 0 {
return true
}
return false
}
+4 -1
View File
@@ -5,7 +5,9 @@ package build
import (
"math/big"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/network"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
)
@@ -20,6 +22,7 @@ const UnixfsLinksPerLevel = 1024
// Consensus / Network
const AllowableClockDriftSecs = uint64(1)
const NewestNetworkVersion = network.Version2
// Epochs
const ForkLengthThreshold = Finality
+13 -1
View File
@@ -10,7 +10,8 @@ package build
import (
"math/big"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/network"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
)
@@ -69,4 +70,15 @@ var (
PackingEfficiencyNum int64 = 4
PackingEfficiencyDenom int64 = 5
UpgradeBreezeHeight abi.ChainEpoch = -1
BreezeGasTampingDuration abi.ChainEpoch = 0
UpgradeSmokeHeight abi.ChainEpoch = -1
DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandMainnet,
}
NewestNetworkVersion = network.Version2
)
+12 -2
View File
@@ -5,13 +5,23 @@
package build
import (
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
)
var DrandSchedule = map[abi.ChainEpoch]DrandEnum{
0: DrandIncentinet,
UpgradeSmokeHeight: DrandMainnet,
}
const UpgradeBreezeHeight = 41280
const BreezeGasTampingDuration = 120
const UpgradeSmokeHeight = 51000
func init() {
power.ConsensusMinerMinPower = big.NewInt(10 << 40)
miner.SupportedProofTypes = map[abi.RegisteredSealProof]struct{}{
+37 -4
View File
@@ -1,6 +1,10 @@
package build
import "fmt"
import (
"fmt"
"golang.org/x/xerrors"
)
var CurrentCommit string
var BuildType int
@@ -25,7 +29,7 @@ func buildType() string {
}
// BuildVersion is the local build version, set by build system
const BuildVersion = "0.5.4"
const BuildVersion = "0.7.0"
func UserVersion() string {
return BuildVersion + buildType() + CurrentCommit
@@ -52,8 +56,37 @@ func (ve Version) EqMajorMinor(v2 Version) bool {
return ve&minorMask == v2&minorMask
}
// APIVersion is a semver version of the rpc api exposed
var APIVersion Version = newVer(0, 12, 0)
type NodeType int
const (
NodeUnknown NodeType = iota
NodeFull
NodeMiner
NodeWorker
)
var RunningNodeType NodeType
func VersionForType(nodeType NodeType) (Version, error) {
switch nodeType {
case NodeFull:
return FullAPIVersion, nil
case NodeMiner:
return MinerAPIVersion, nil
case NodeWorker:
return WorkerAPIVersion, nil
default:
return Version(0), xerrors.Errorf("unknown node type %d", nodeType)
}
}
// semver versions of the rpc api exposed
var (
FullAPIVersion = newVer(0, 14, 0)
MinerAPIVersion = newVer(0, 14, 0)
WorkerAPIVersion = newVer(0, 14, 0)
)
//nolint:varcheck,deadcode
const (
+1 -1
View File
@@ -3,7 +3,7 @@ package aerrors
import (
"fmt"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/filecoin-project/go-state-types/exitcode"
"golang.org/x/xerrors"
)
+1 -1
View File
@@ -3,8 +3,8 @@ package aerrors_test
import (
"testing"
"github.com/filecoin-project/go-state-types/exitcode"
. "github.com/filecoin-project/lotus/chain/actors/aerrors"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/stretchr/testify/assert"
"golang.org/x/xerrors"
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"errors"
"fmt"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/filecoin-project/go-state-types/exitcode"
cbor "github.com/ipfs/go-ipld-cbor"
"golang.org/x/xerrors"
)
+4
View File
@@ -56,6 +56,10 @@ func (bts *BadBlockCache) Add(c cid.Cid, bbr BadBlockReason) {
bts.badBlocks.Add(c, bbr)
}
func (bts *BadBlockCache) Remove(c cid.Cid) {
bts.badBlocks.Remove(c)
}
func (bts *BadBlockCache) Has(c cid.Cid) (BadBlockReason, bool) {
rval, ok := bts.badBlocks.Get(c)
if !ok {
+72 -7
View File
@@ -3,7 +3,7 @@ package beacon
import (
"context"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
logging "github.com/ipfs/go-log"
"golang.org/x/xerrors"
@@ -18,6 +18,23 @@ type Response struct {
Err error
}
type Schedule []BeaconPoint
func (bs Schedule) BeaconForEpoch(e abi.ChainEpoch) RandomBeacon {
for i := len(bs) - 1; i >= 0; i-- {
bp := bs[i]
if e >= bp.Start {
return bp.Beacon
}
}
return bs[0].Beacon
}
type BeaconPoint struct {
Start abi.ChainEpoch
Beacon RandomBeacon
}
// RandomBeacon represents a system that provides randomness to Lotus.
// Other components interrogate the RandomBeacon to acquire randomness that's
// valid for a specific chain epoch. Also to verify beacon entries that have
@@ -25,11 +42,30 @@ type Response struct {
type RandomBeacon interface {
Entry(context.Context, uint64) <-chan Response
VerifyEntry(types.BeaconEntry, types.BeaconEntry) error
MaxBeaconRoundForEpoch(abi.ChainEpoch, types.BeaconEntry) uint64
MaxBeaconRoundForEpoch(abi.ChainEpoch) uint64
}
func ValidateBlockValues(b RandomBeacon, h *types.BlockHeader, prevEntry types.BeaconEntry) error {
maxRound := b.MaxBeaconRoundForEpoch(h.Height, prevEntry)
func ValidateBlockValues(bSchedule Schedule, h *types.BlockHeader, parentEpoch abi.ChainEpoch,
prevEntry types.BeaconEntry) error {
{
parentBeacon := bSchedule.BeaconForEpoch(parentEpoch)
currBeacon := bSchedule.BeaconForEpoch(h.Height)
if parentBeacon != currBeacon {
if len(h.BeaconEntries) != 2 {
return xerrors.Errorf("expected two beacon entries at beacon fork, got %d", len(h.BeaconEntries))
}
err := currBeacon.VerifyEntry(h.BeaconEntries[1], h.BeaconEntries[0])
if err != nil {
return xerrors.Errorf("beacon at fork point invalid: (%v, %v): %w",
h.BeaconEntries[1], h.BeaconEntries[0], err)
}
return nil
}
}
// TODO: fork logic
b := bSchedule.BeaconForEpoch(h.Height)
maxRound := b.MaxBeaconRoundForEpoch(h.Height)
if maxRound == prevEntry.Round {
if len(h.BeaconEntries) != 0 {
return xerrors.Errorf("expected not to have any beacon entries in this block, got %d", len(h.BeaconEntries))
@@ -37,6 +73,10 @@ func ValidateBlockValues(b RandomBeacon, h *types.BlockHeader, prevEntry types.B
return nil
}
if len(h.BeaconEntries) == 0 {
return xerrors.Errorf("expected to have beacon entries in this block, but didn't find any")
}
last := h.BeaconEntries[len(h.BeaconEntries)-1]
if last.Round != maxRound {
return xerrors.Errorf("expected final beacon entry in block to be at round %d, got %d", maxRound, last.Round)
@@ -52,10 +92,35 @@ func ValidateBlockValues(b RandomBeacon, h *types.BlockHeader, prevEntry types.B
return nil
}
func BeaconEntriesForBlock(ctx context.Context, beacon RandomBeacon, round abi.ChainEpoch, prev types.BeaconEntry) ([]types.BeaconEntry, error) {
func BeaconEntriesForBlock(ctx context.Context, bSchedule Schedule, epoch abi.ChainEpoch, parentEpoch abi.ChainEpoch, prev types.BeaconEntry) ([]types.BeaconEntry, error) {
{
parentBeacon := bSchedule.BeaconForEpoch(parentEpoch)
currBeacon := bSchedule.BeaconForEpoch(epoch)
if parentBeacon != currBeacon {
// Fork logic
round := currBeacon.MaxBeaconRoundForEpoch(epoch)
out := make([]types.BeaconEntry, 2)
rch := currBeacon.Entry(ctx, round-1)
res := <-rch
if res.Err != nil {
return nil, xerrors.Errorf("getting entry %d returned error: %w", round-1, res.Err)
}
out[0] = res.Entry
rch = currBeacon.Entry(ctx, round)
res = <-rch
if res.Err != nil {
return nil, xerrors.Errorf("getting entry %d returned error: %w", round, res.Err)
}
out[1] = res.Entry
return out, nil
}
}
beacon := bSchedule.BeaconForEpoch(epoch)
start := build.Clock.Now()
maxRound := beacon.MaxBeaconRoundForEpoch(round, prev)
maxRound := beacon.MaxBeaconRoundForEpoch(epoch)
if maxRound == prev.Round {
return nil, nil
}
@@ -78,7 +143,7 @@ func BeaconEntriesForBlock(ctx context.Context, beacon RandomBeacon, round abi.C
out = append(out, resp.Entry)
cur = resp.Entry.Round - 1
case <-ctx.Done():
return nil, xerrors.Errorf("context timed out waiting on beacon entry to come back for round %d: %w", round, ctx.Err())
return nil, xerrors.Errorf("context timed out waiting on beacon entry to come back for epoch %d: %w", epoch, ctx.Err())
}
}
+2 -2
View File
@@ -19,7 +19,7 @@ import (
logging "github.com/ipfs/go-log"
pubsub "github.com/libp2p/go-libp2p-pubsub"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/beacon"
@@ -187,7 +187,7 @@ func (db *DrandBeacon) VerifyEntry(curr types.BeaconEntry, prev types.BeaconEntr
return err
}
func (db *DrandBeacon) MaxBeaconRoundForEpoch(filEpoch abi.ChainEpoch, prevEntry types.BeaconEntry) uint64 {
func (db *DrandBeacon) MaxBeaconRoundForEpoch(filEpoch abi.ChainEpoch) uint64 {
// TODO: sometimes the genesis time for filecoin is zero and this goes negative
latestTs := ((uint64(filEpoch) * db.filRoundTime) + db.filGenTime) - db.filRoundTime
dround := (latestTs - db.drandGenTime) / uint64(db.interval.Seconds())
+1 -1
View File
@@ -12,7 +12,7 @@ import (
)
func TestPrintGroupInfo(t *testing.T) {
server := build.DrandConfig().Servers[0]
server := build.DrandConfigs[build.DrandIncentinet].Servers[0]
c, err := hclient.New(server, nil, nil)
assert.NoError(t, err)
cg := c.(interface {
+2 -6
View File
@@ -6,8 +6,8 @@ import (
"encoding/binary"
"time"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/minio/blake2b-simd"
"golang.org/x/xerrors"
)
@@ -53,11 +53,7 @@ func (mb *mockBeacon) VerifyEntry(from types.BeaconEntry, to types.BeaconEntry)
return nil
}
func (mb *mockBeacon) IsEntryForEpoch(e types.BeaconEntry, epoch abi.ChainEpoch, nulls int) (bool, error) {
return int64(e.Round) <= int64(epoch) && int64(epoch)-int64(nulls) >= int64(e.Round), nil
}
func (mb *mockBeacon) MaxBeaconRoundForEpoch(epoch abi.ChainEpoch, prevEntry types.BeaconEntry) uint64 {
func (mb *mockBeacon) MaxBeaconRoundForEpoch(epoch abi.ChainEpoch) uint64 {
return uint64(epoch)
}
+81
View File
@@ -0,0 +1,81 @@
package chain
import (
"encoding/json"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/node/modules/dtypes"
"github.com/ipfs/go-datastore"
"golang.org/x/xerrors"
)
var CheckpointKey = datastore.NewKey("/chain/checks")
func loadCheckpoint(ds dtypes.MetadataDS) (types.TipSetKey, error) {
haveChks, err := ds.Has(CheckpointKey)
if err != nil {
return types.EmptyTSK, err
}
if !haveChks {
return types.EmptyTSK, nil
}
tskBytes, err := ds.Get(CheckpointKey)
if err != nil {
return types.EmptyTSK, err
}
var tsk types.TipSetKey
err = json.Unmarshal(tskBytes, &tsk)
if err != nil {
return types.EmptyTSK, err
}
return tsk, err
}
func (syncer *Syncer) SetCheckpoint(tsk types.TipSetKey) error {
if tsk == types.EmptyTSK {
return xerrors.Errorf("called with empty tsk")
}
syncer.checkptLk.Lock()
defer syncer.checkptLk.Unlock()
ts, err := syncer.ChainStore().LoadTipSet(tsk)
if err != nil {
return xerrors.Errorf("cannot find tipset: %w", err)
}
hts := syncer.ChainStore().GetHeaviestTipSet()
anc, err := syncer.ChainStore().IsAncestorOf(ts, hts)
if err != nil {
return xerrors.Errorf("cannot determine whether checkpoint tipset is in main-chain: %w", err)
}
if !hts.Equals(ts) && !anc {
return xerrors.Errorf("cannot mark tipset as checkpoint, since it isn't in the main-chain: %w", err)
}
tskBytes, err := json.Marshal(tsk)
if err != nil {
return err
}
err = syncer.ds.Put(CheckpointKey, tskBytes)
if err != nil {
return err
}
syncer.checkpt = tsk
return nil
}
func (syncer *Syncer) GetCheckpoint() types.TipSetKey {
syncer.checkptLk.Lock()
defer syncer.checkptLk.Unlock()
return syncer.checkpt
}
+3 -2
View File
@@ -5,7 +5,7 @@ import (
"sync"
"time"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
logging "github.com/ipfs/go-log/v2"
"golang.org/x/xerrors"
@@ -35,6 +35,7 @@ type eventAPI interface {
ChainNotify(context.Context) (<-chan []*api.HeadChange, error)
ChainGetBlockMessages(context.Context, cid.Cid) (*api.BlockMessages, error)
ChainGetTipSetByHeight(context.Context, abi.ChainEpoch, types.TipSetKey) (*types.TipSet, error)
ChainHead(context.Context) (*types.TipSet, error)
StateGetReceipt(context.Context, cid.Cid, types.TipSetKey) (*types.MessageReceipt, error)
ChainGetTipSet(context.Context, types.TipSetKey) (*types.TipSet, error)
@@ -57,7 +58,7 @@ type Events struct {
func NewEvents(ctx context.Context, api eventAPI) *Events {
gcConfidence := 2 * build.ForkLengthThreshold
tsc := newTSCache(gcConfidence, api.ChainGetTipSetByHeight)
tsc := newTSCache(gcConfidence, api)
e := &Events{
api: api,
+5 -2
View File
@@ -5,7 +5,7 @@ import (
"math"
"sync"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
"golang.org/x/xerrors"
@@ -307,7 +307,10 @@ func (e *hcEvents) onHeadChanged(check CheckFunc, hnd EventHandler, rev RevertHa
defer e.lk.Unlock()
// Check if the event has already occurred
ts := e.tsc.best()
ts, err := e.tsc.best()
if err != nil {
return 0, xerrors.Errorf("error getting best tipset: %w", err)
}
done, more, err := check(ts)
if err != nil {
return 0, xerrors.Errorf("called check error (h: %d): %w", ts.Height(), err)
+12 -3
View File
@@ -4,8 +4,9 @@ import (
"context"
"sync"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"go.opencensus.io/trace"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/types"
)
@@ -152,8 +153,12 @@ func (e *heightEvents) ChainAt(hnd HeightHandler, rev RevertHandler, confidence
e.lk.Lock() // Tricky locking, check your locks if you modify this function!
bestH := e.tsc.best().Height()
best, err := e.tsc.best()
if err != nil {
return xerrors.Errorf("error getting best tipset: %w", err)
}
bestH := best.Height()
if bestH >= h+abi.ChainEpoch(confidence) {
ts, err := e.tsc.getNonNull(h)
if err != nil {
@@ -172,7 +177,11 @@ func (e *heightEvents) ChainAt(hnd HeightHandler, rev RevertHandler, confidence
}
e.lk.Lock()
bestH = e.tsc.best().Height()
best, err = e.tsc.best()
if err != nil {
return xerrors.Errorf("error getting best tipset: %w", err)
}
bestH = best.Height()
}
defer e.lk.Unlock()
+16 -5
View File
@@ -11,8 +11,8 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
@@ -46,6 +46,10 @@ type fakeCS struct {
sub func(rev, app []*types.TipSet)
}
func (fcs *fakeCS) ChainHead(ctx context.Context) (*types.TipSet, error) {
panic("implement me")
}
func (fcs *fakeCS) ChainGetTipSet(ctx context.Context, key types.TipSetKey) (*types.TipSet, error) {
return fcs.tipsets[key], nil
}
@@ -110,7 +114,11 @@ func (fcs *fakeCS) makeTs(t *testing.T, parents []cid.Cid, h abi.ChainEpoch, msg
func (fcs *fakeCS) ChainNotify(context.Context) (<-chan []*api.HeadChange, error) {
out := make(chan []*api.HeadChange, 1)
out <- []*api.HeadChange{{Type: store.HCCurrent, Val: fcs.tsc.best()}}
best, err := fcs.tsc.best()
if err != nil {
return nil, err
}
out <- []*api.HeadChange{{Type: store.HCCurrent, Val: best}}
fcs.sub = func(rev, app []*types.TipSet) {
notif := make([]*api.HeadChange, len(rev)+len(app))
@@ -174,7 +182,8 @@ func (fcs *fakeCS) advance(rev, app int, msgs map[int]cid.Cid, nulls ...int) { /
var revs []*types.TipSet
for i := 0; i < rev; i++ {
ts := fcs.tsc.best()
ts, err := fcs.tsc.best()
require.NoError(fcs.t, err)
if _, ok := nullm[int(ts.Height())]; !ok {
revs = append(revs, ts)
@@ -196,7 +205,9 @@ func (fcs *fakeCS) advance(rev, app int, msgs map[int]cid.Cid, nulls ...int) { /
continue
}
ts := fcs.makeTs(fcs.t, fcs.tsc.best().Key().Cids(), fcs.h, mc)
best, err := fcs.tsc.best()
require.NoError(fcs.t, err)
ts := fcs.makeTs(fcs.t, best.Key().Cids(), fcs.h, mc)
require.NoError(fcs.t, fcs.tsc.add(ts))
if hasMsgs {
+2 -2
View File
@@ -5,8 +5,8 @@ import (
"context"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
init_ "github.com/filecoin-project/specs-actors/actors/builtin/init"
"github.com/filecoin-project/specs-actors/actors/builtin/market"
+3 -3
View File
@@ -13,11 +13,11 @@ import (
cbornode "github.com/ipfs/go-ipld-cbor"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/builtin/market"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/util/adt"
tutils "github.com/filecoin-project/specs-actors/support/testing"
+15 -8
View File
@@ -3,13 +3,16 @@ package events
import (
"context"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/types"
)
type tsByHFunc func(context.Context, abi.ChainEpoch, types.TipSetKey) (*types.TipSet, error)
type tsCacheAPI interface {
ChainGetTipSetByHeight(context.Context, abi.ChainEpoch, types.TipSetKey) (*types.TipSet, error)
ChainHead(context.Context) (*types.TipSet, error)
}
// tipSetCache implements a simple ring-buffer cache to keep track of recent
// tipsets
@@ -18,10 +21,10 @@ type tipSetCache struct {
start int
len int
storage tsByHFunc
storage tsCacheAPI
}
func newTSCache(cap abi.ChainEpoch, storage tsByHFunc) *tipSetCache {
func newTSCache(cap abi.ChainEpoch, storage tsCacheAPI) *tipSetCache {
return &tipSetCache{
cache: make([]*types.TipSet, cap),
start: 0,
@@ -94,7 +97,7 @@ func (tsc *tipSetCache) getNonNull(height abi.ChainEpoch) (*types.TipSet, error)
func (tsc *tipSetCache) get(height abi.ChainEpoch) (*types.TipSet, error) {
if tsc.len == 0 {
log.Warnf("tipSetCache.get: cache is empty, requesting from storage (h=%d)", height)
return tsc.storage(context.TODO(), height, types.EmptyTSK)
return tsc.storage.ChainGetTipSetByHeight(context.TODO(), height, types.EmptyTSK)
}
headH := tsc.cache[tsc.start].Height()
@@ -114,14 +117,18 @@ func (tsc *tipSetCache) get(height abi.ChainEpoch) (*types.TipSet, error) {
if height < tail.Height() {
log.Warnf("tipSetCache.get: requested tipset not in cache, requesting from storage (h=%d; tail=%d)", height, tail.Height())
return tsc.storage(context.TODO(), height, tail.Key())
return tsc.storage.ChainGetTipSetByHeight(context.TODO(), height, tail.Key())
}
return tsc.cache[normalModulo(tsc.start-int(headH-height), clen)], nil
}
func (tsc *tipSetCache) best() *types.TipSet {
return tsc.cache[tsc.start]
func (tsc *tipSetCache) best() (*types.TipSet, error) {
best := tsc.cache[tsc.start]
if best == nil {
return tsc.storage.ChainHead(context.TODO())
}
return best, nil
}
func normalModulo(n, m int) int {
+61 -15
View File
@@ -4,8 +4,8 @@ import (
"context"
"testing"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
@@ -13,10 +13,7 @@ import (
)
func TestTsCache(t *testing.T) {
tsc := newTSCache(50, func(context.Context, abi.ChainEpoch, types.TipSetKey) (*types.TipSet, error) {
t.Fatal("storage call")
return &types.TipSet{}, nil
})
tsc := newTSCache(50, &tsCacheAPIFailOnStorageCall{t: t})
h := abi.ChainEpoch(75)
@@ -43,7 +40,12 @@ func TestTsCache(t *testing.T) {
for i := 0; i < 9000; i++ {
if i%90 > 60 {
if err := tsc.revert(tsc.best()); err != nil {
best, err := tsc.best()
if err != nil {
t.Fatal(err, "; i:", i)
return
}
if err := tsc.revert(best); err != nil {
t.Fatal(err, "; i:", i)
return
}
@@ -55,11 +57,21 @@ func TestTsCache(t *testing.T) {
}
type tsCacheAPIFailOnStorageCall struct {
t *testing.T
}
func (tc *tsCacheAPIFailOnStorageCall) ChainGetTipSetByHeight(ctx context.Context, epoch abi.ChainEpoch, key types.TipSetKey) (*types.TipSet, error) {
tc.t.Fatal("storage call")
return &types.TipSet{}, nil
}
func (tc *tsCacheAPIFailOnStorageCall) ChainHead(ctx context.Context) (*types.TipSet, error) {
tc.t.Fatal("storage call")
return &types.TipSet{}, nil
}
func TestTsCacheNulls(t *testing.T) {
tsc := newTSCache(50, func(context.Context, abi.ChainEpoch, types.TipSetKey) (*types.TipSet, error) {
t.Fatal("storage call")
return &types.TipSet{}, nil
})
tsc := newTSCache(50, &tsCacheAPIFailOnStorageCall{t: t})
h := abi.ChainEpoch(75)
@@ -91,7 +103,9 @@ func TestTsCacheNulls(t *testing.T) {
add()
add()
require.Equal(t, h-1, tsc.best().Height())
best, err := tsc.best()
require.NoError(t, err)
require.Equal(t, h-1, best.Height())
ts, err := tsc.get(h - 1)
require.NoError(t, err)
@@ -109,9 +123,17 @@ func TestTsCacheNulls(t *testing.T) {
require.NoError(t, err)
require.Equal(t, h-8, ts.Height())
require.NoError(t, tsc.revert(tsc.best()))
require.NoError(t, tsc.revert(tsc.best()))
require.Equal(t, h-8, tsc.best().Height())
best, err = tsc.best()
require.NoError(t, err)
require.NoError(t, tsc.revert(best))
best, err = tsc.best()
require.NoError(t, err)
require.NoError(t, tsc.revert(best))
best, err = tsc.best()
require.NoError(t, err)
require.Equal(t, h-8, best.Height())
h += 50
add()
@@ -120,3 +142,27 @@ func TestTsCacheNulls(t *testing.T) {
require.NoError(t, err)
require.Equal(t, h-1, ts.Height())
}
type tsCacheAPIStorageCallCounter struct {
t *testing.T
chainGetTipSetByHeight int
chainHead int
}
func (tc *tsCacheAPIStorageCallCounter) ChainGetTipSetByHeight(ctx context.Context, epoch abi.ChainEpoch, key types.TipSetKey) (*types.TipSet, error) {
tc.chainGetTipSetByHeight++
return &types.TipSet{}, nil
}
func (tc *tsCacheAPIStorageCallCounter) ChainHead(ctx context.Context) (*types.TipSet, error) {
tc.chainHead++
return &types.TipSet{}, nil
}
func TestTsCacheEmpty(t *testing.T) {
// Calling best on an empty cache should just call out to the chain API
callCounter := &tsCacheAPIStorageCallCounter{t: t}
tsc := newTSCache(50, callCounter)
_, err := tsc.best()
require.NoError(t, err)
require.Equal(t, 1, callCounter.chainHead)
}
@@ -1,6 +1,6 @@
// Code generated by github.com/whyrusleeping/cbor-gen. DO NOT EDIT.
package blocksync
package exchange
import (
"fmt"
@@ -1,4 +1,4 @@
package blocksync
package exchange
import (
"bufio"
@@ -7,13 +7,16 @@ import (
"math/rand"
"time"
host "github.com/libp2p/go-libp2p-core/host"
inet "github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/host"
"github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/peer"
"go.opencensus.io/trace"
"go.uber.org/fx"
"golang.org/x/xerrors"
cborutil "github.com/filecoin-project/go-cbor-util"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types"
@@ -21,11 +24,9 @@ import (
"github.com/filecoin-project/lotus/lib/peermgr"
)
// Protocol client.
// FIXME: Rename to just `Client`. Not done at the moment to avoid
// disrupting too much of the consumer code, should be done along
// https://github.com/filecoin-project/lotus/issues/2612.
type BlockSync struct {
// client implements exchange.Client, using the libp2p ChainExchange protocol
// as the fetching mechanism.
type client struct {
// Connection manager used to contact the server.
// FIXME: We should have a reduced interface here, initialized
// just with our protocol ID, we shouldn't be able to open *any*
@@ -35,13 +36,14 @@ type BlockSync struct {
peerTracker *bsPeerTracker
}
func NewClient(
host host.Host,
pmgr peermgr.MaybePeerMgr,
) *BlockSync {
return &BlockSync{
var _ Client = (*client)(nil)
// NewClient creates a new libp2p-based exchange.Client that uses the libp2p
// ChainExhange protocol as the fetching mechanism.
func NewClient(lc fx.Lifecycle, host host.Host, pmgr peermgr.MaybePeerMgr) Client {
return &client{
host: host,
peerTracker: newPeerTracker(pmgr.Mgr),
peerTracker: newPeerTracker(lc, host, pmgr.Mgr),
}
}
@@ -62,11 +64,7 @@ func NewClient(
// request options without disrupting external calls. In the future the
// consumers should be forced to use a more standardized service and
// adhere to a single API derived from this function.
func (client *BlockSync) doRequest(
ctx context.Context,
req *Request,
singlePeer *peer.ID,
) (*validatedResponse, error) {
func (c *client) doRequest(ctx context.Context, req *Request, singlePeer *peer.ID) (*validatedResponse, error) {
// Validate request.
if req.Length == 0 {
return nil, xerrors.Errorf("invalid request of length 0")
@@ -86,7 +84,7 @@ func (client *BlockSync) doRequest(
if singlePeer != nil {
peers = []peer.ID{*singlePeer}
} else {
peers = client.getShuffledPeers()
peers = c.getShuffledPeers()
if len(peers) == 0 {
return nil, xerrors.Errorf("no peers available")
}
@@ -107,9 +105,9 @@ func (client *BlockSync) doRequest(
}
// Send request, read response.
res, err := client.sendRequestToPeer(ctx, peer, req)
res, err := c.sendRequestToPeer(ctx, peer, req)
if err != nil {
if !xerrors.Is(err, inet.ErrNoConn) {
if !xerrors.Is(err, network.ErrNoConn) {
log.Warnf("could not connect to peer %s: %s",
peer.String(), err)
}
@@ -117,15 +115,15 @@ func (client *BlockSync) doRequest(
}
// Process and validate response.
validRes, err := client.processResponse(req, res)
validRes, err := c.processResponse(req, res)
if err != nil {
log.Warnf("processing peer %s response failed: %s",
peer.String(), err)
continue
}
client.peerTracker.logGlobalSuccess(build.Clock.Since(globalTime))
client.host.ConnManager().TagPeer(peer, "bsync", SUCCESS_PEER_TAG_VALUE)
c.peerTracker.logGlobalSuccess(build.Clock.Since(globalTime))
c.host.ConnManager().TagPeer(peer, "bsync", SuccessPeerTagValue)
return validRes, nil
}
@@ -144,11 +142,8 @@ func (client *BlockSync) doRequest(
// We are conflating in the single error returned both status and validation
// errors. Peer penalization should happen here then, before returning, so
// we can apply the correct penalties depending on the cause of the error.
func (client *BlockSync) processResponse(
req *Request,
res *Response,
// FIXME: Add the `peer` as argument once we implement penalties.
) (*validatedResponse, error) {
// FIXME: Add the `peer` as argument once we implement penalties.
func (c *client) processResponse(req *Request, res *Response) (*validatedResponse, error) {
err := res.statusToError()
if err != nil {
return nil, xerrors.Errorf("status error: %s", err)
@@ -246,16 +241,8 @@ func (client *BlockSync) processResponse(
return validRes, nil
}
// GetBlocks fetches count blocks from the network, from the provided tipset
// *backwards*, returning as many tipsets as count.
//
// {hint/usage}: This is used by the Syncer during normal chain syncing and when
// resolving forks.
func (client *BlockSync) GetBlocks(
ctx context.Context,
tsk types.TipSetKey,
count int,
) ([]*types.TipSet, error) {
// GetBlocks implements Client.GetBlocks(). Refer to the godocs there.
func (c *client) GetBlocks(ctx context.Context, tsk types.TipSetKey, count int) ([]*types.TipSet, error) {
ctx, span := trace.StartSpan(ctx, "bsync.GetBlocks")
defer span.End()
if span.IsRecordingEvents() {
@@ -271,7 +258,7 @@ func (client *BlockSync) GetBlocks(
Options: Headers,
}
validRes, err := client.doRequest(ctx, req, nil)
validRes, err := c.doRequest(ctx, req, nil)
if err != nil {
return nil, err
}
@@ -279,11 +266,8 @@ func (client *BlockSync) GetBlocks(
return validRes.tipsets, nil
}
func (client *BlockSync) GetFullTipSet(
ctx context.Context,
peer peer.ID,
tsk types.TipSetKey,
) (*store.FullTipSet, error) {
// GetFullTipSet implements Client.GetFullTipSet(). Refer to the godocs there.
func (c *client) GetFullTipSet(ctx context.Context, peer peer.ID, tsk types.TipSetKey) (*store.FullTipSet, error) {
// TODO: round robin through these peers on error
req := &Request{
@@ -292,7 +276,7 @@ func (client *BlockSync) GetFullTipSet(
Options: Headers | Messages,
}
validRes, err := client.doRequest(ctx, req, &peer)
validRes, err := c.doRequest(ctx, req, &peer)
if err != nil {
return nil, err
}
@@ -302,11 +286,8 @@ func (client *BlockSync) GetFullTipSet(
// *one* tipset here, so it's safe to index directly.
}
func (client *BlockSync) GetChainMessages(
ctx context.Context,
head *types.TipSet,
length uint64,
) ([]*CompactedMessages, error) {
// GetChainMessages implements Client.GetChainMessages(). Refer to the godocs there.
func (c *client) GetChainMessages(ctx context.Context, head *types.TipSet, length uint64) ([]*CompactedMessages, error) {
ctx, span := trace.StartSpan(ctx, "GetChainMessages")
if span.IsRecordingEvents() {
span.AddAttributes(
@@ -322,7 +303,7 @@ func (client *BlockSync) GetChainMessages(
Options: Messages,
}
validRes, err := client.doRequest(ctx, req, nil)
validRes, err := c.doRequest(ctx, req, nil)
if err != nil {
return nil, err
}
@@ -333,11 +314,7 @@ func (client *BlockSync) GetChainMessages(
// Send a request to a peer. Write request in the stream and read the
// response back. We do not do any processing of the request/response
// here.
func (client *BlockSync) sendRequestToPeer(
ctx context.Context,
peer peer.ID,
req *Request,
) (_ *Response, err error) {
func (c *client) sendRequestToPeer(ctx context.Context, peer peer.ID, req *Request) (_ *Response, err error) {
// Trace code.
ctx, span := trace.StartSpan(ctx, "sendRequestToPeer")
defer span.End()
@@ -358,34 +335,33 @@ func (client *BlockSync) sendRequestToPeer(
}()
// -- TRACE --
supported, err := client.host.Peerstore().SupportsProtocols(peer, BlockSyncProtocolID)
supported, err := c.host.Peerstore().SupportsProtocols(peer, BlockSyncProtocolID, ChainExchangeProtocolID)
if err != nil {
c.RemovePeer(peer)
return nil, xerrors.Errorf("failed to get protocols for peer: %w", err)
}
if len(supported) == 0 || supported[0] != BlockSyncProtocolID {
return nil, xerrors.Errorf("peer %s does not support protocol %s",
peer, BlockSyncProtocolID)
// FIXME: `ProtoBook` should support a *single* protocol check that returns
// a bool instead of a list.
if len(supported) == 0 || (supported[0] != BlockSyncProtocolID && supported[0] != ChainExchangeProtocolID) {
return nil, xerrors.Errorf("peer %s does not support protocols %s",
peer, []string{BlockSyncProtocolID, ChainExchangeProtocolID})
}
connectionStart := build.Clock.Now()
// Open stream to peer.
stream, err := client.host.NewStream(
inet.WithNoDial(ctx, "should already have connection"),
stream, err := c.host.NewStream(
network.WithNoDial(ctx, "should already have connection"),
peer,
BlockSyncProtocolID)
ChainExchangeProtocolID, BlockSyncProtocolID)
if err != nil {
client.RemovePeer(peer)
c.RemovePeer(peer)
return nil, xerrors.Errorf("failed to open stream to peer: %w", err)
}
// Write request.
_ = stream.SetWriteDeadline(time.Now().Add(WRITE_REQ_DEADLINE))
_ = stream.SetWriteDeadline(time.Now().Add(WriteReqDeadline))
if err := cborutil.WriteCborRPC(stream, req); err != nil {
_ = stream.SetWriteDeadline(time.Time{})
client.peerTracker.logFailure(peer, build.Clock.Since(connectionStart))
c.peerTracker.logFailure(peer, build.Clock.Since(connectionStart), req.Length)
// FIXME: Should we also remove peer here?
return nil, err
}
@@ -395,11 +371,11 @@ func (client *BlockSync) sendRequestToPeer(
// Read response.
var res Response
err = cborutil.ReadCborRPC(
bufio.NewReader(incrt.New(stream, READ_RES_MIN_SPEED, READ_RES_DEADLINE)),
bufio.NewReader(incrt.New(stream, ReadResMinSpeed, ReadResDeadline)),
&res)
if err != nil {
client.peerTracker.logFailure(peer, build.Clock.Since(connectionStart))
return nil, xerrors.Errorf("failed to read blocksync response: %w", err)
c.peerTracker.logFailure(peer, build.Clock.Since(connectionStart), req.Length)
return nil, xerrors.Errorf("failed to read chainxchg response: %w", err)
}
// FIXME: Move all this together at the top using a defer as done elsewhere.
@@ -412,32 +388,34 @@ func (client *BlockSync) sendRequestToPeer(
)
}
client.peerTracker.logSuccess(peer, build.Clock.Since(connectionStart))
c.peerTracker.logSuccess(peer, build.Clock.Since(connectionStart), uint64(len(res.Chain)))
// FIXME: We should really log a success only after we validate the response.
// It might be a bit hard to do.
return &res, nil
}
func (client *BlockSync) AddPeer(p peer.ID) {
client.peerTracker.addPeer(p)
// AddPeer implements Client.AddPeer(). Refer to the godocs there.
func (c *client) AddPeer(p peer.ID) {
c.peerTracker.addPeer(p)
}
func (client *BlockSync) RemovePeer(p peer.ID) {
client.peerTracker.removePeer(p)
// RemovePeer implements Client.RemovePeer(). Refer to the godocs there.
func (c *client) RemovePeer(p peer.ID) {
c.peerTracker.removePeer(p)
}
// getShuffledPeers returns a preference-sorted set of peers (by latency
// and failure counting), shuffling the first few peers so we don't always
// pick the same peer.
// FIXME: Consider merging with `shufflePrefix()s`.
func (client *BlockSync) getShuffledPeers() []peer.ID {
peers := client.peerTracker.prefSortedPeers()
func (c *client) getShuffledPeers() []peer.ID {
peers := c.peerTracker.prefSortedPeers()
shufflePrefix(peers)
return peers
}
func shufflePrefix(peers []peer.ID) {
prefix := SHUFFLE_PEERS_PREFIX
prefix := ShufflePeersPrefix
if len(peers) < prefix {
prefix = len(peers)
}
+19
View File
@@ -0,0 +1,19 @@
// Package exchange contains the ChainExchange server and client components.
//
// ChainExchange is the basic chain synchronization protocol of Filecoin.
// ChainExchange is an RPC-oriented protocol, with a single operation to
// request blocks for now.
//
// A request contains a start anchor block (referred to with a CID), and a
// amount of blocks requested beyond the anchor (including the anchor itself).
//
// A client can also pass options, encoded as a 64-bit bitfield. Lotus supports
// two options at the moment:
//
// - include block contents
// - include block messages
//
// The response will include a status code, an optional message, and the
// response payload in case of success. The payload is a slice of serialized
// tipsets.
package exchange
+51
View File
@@ -0,0 +1,51 @@
package exchange
import (
"context"
"github.com/libp2p/go-libp2p-core/network"
"github.com/libp2p/go-libp2p-core/peer"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types"
)
// Server is the responder side of the ChainExchange protocol. It accepts
// requests from clients and services them by returning the requested
// chain data.
type Server interface {
// HandleStream is the protocol handler to be registered on a libp2p
// protocol router.
//
// In the current version of the protocol, streams are single-use. The
// server will read a single Request, and will respond with a single
// Response. It will dispose of the stream straight after.
HandleStream(stream network.Stream)
}
// Client is the requesting side of the ChainExchange protocol. It acts as
// a proxy for other components to request chain data from peers. It is chiefly
// used by the Syncer.
type Client interface {
// GetBlocks fetches block headers from the network, from the provided
// tipset *backwards*, returning as many tipsets as the count parameter,
// or less.
GetBlocks(ctx context.Context, tsk types.TipSetKey, count int) ([]*types.TipSet, error)
// GetChainMessages fetches messages from the network, from the provided
// tipset *backwards*, returning the messages from as many tipsets as the
// count parameter, or less.
GetChainMessages(ctx context.Context, head *types.TipSet, length uint64) ([]*CompactedMessages, error)
// GetFullTipSet fetches a full tipset from a given peer. If successful,
// the fetched object contains block headers and all messages in full form.
GetFullTipSet(ctx context.Context, peer peer.ID, tsk types.TipSetKey) (*store.FullTipSet, error)
// AddPeer adds a peer to the pool of peers that the Client requests
// data from.
AddPeer(peer peer.ID)
// RemovePeer removes a peer from the pool of peers that the Client
// requests data from.
RemovePeer(peer peer.ID)
}
@@ -1,13 +1,16 @@
package blocksync
package exchange
// FIXME: This needs to be reviewed.
import (
"context"
"sort"
"sync"
"time"
host "github.com/libp2p/go-libp2p-core/host"
"github.com/libp2p/go-libp2p-core/peer"
"go.uber.org/fx"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/lib/peermgr"
@@ -29,11 +32,30 @@ type bsPeerTracker struct {
pmgr *peermgr.PeerMgr
}
func newPeerTracker(pmgr *peermgr.PeerMgr) *bsPeerTracker {
return &bsPeerTracker{
func newPeerTracker(lc fx.Lifecycle, h host.Host, pmgr *peermgr.PeerMgr) *bsPeerTracker {
bsPt := &bsPeerTracker{
peers: make(map[peer.ID]*peerStats),
pmgr: pmgr,
}
sub, err := h.EventBus().Subscribe(new(peermgr.NewFilPeer))
if err != nil {
panic(err)
}
go func() {
for newPeer := range sub.Out() {
bsPt.addPeer(newPeer.(peermgr.NewFilPeer).Id)
}
}()
lc.Append(fx.Hook{
OnStop: func(ctx context.Context) error {
return sub.Close()
},
})
return bsPt
}
func (bpt *bsPeerTracker) addPeer(p peer.ID) {
@@ -72,16 +94,7 @@ func (bpt *bsPeerTracker) prefSortedPeers() []peer.ID {
var costI, costJ float64
getPeerInitLat := func(p peer.ID) float64 {
var res float64
if bpt.pmgr != nil {
if lat, ok := bpt.pmgr.GetPeerLatency(p); ok {
res = float64(lat)
}
}
if res == 0 {
res = float64(bpt.avgGlobalTime)
}
return res * newPeerMul
return float64(bpt.avgGlobalTime) * newPeerMul
}
if pi.successes+pi.failures > 0 {
@@ -107,8 +120,8 @@ func (bpt *bsPeerTracker) prefSortedPeers() []peer.ID {
const (
// xInvAlpha = (N+1)/2
localInvAlpha = 5 // 86% of the value is the last 9
globalInvAlpha = 20 // 86% of the value is the last 39
localInvAlpha = 10 // 86% of the value is the last 19
globalInvAlpha = 25 // 86% of the value is the last 49
)
func (bpt *bsPeerTracker) logGlobalSuccess(dur time.Duration) {
@@ -133,7 +146,7 @@ func logTime(pi *peerStats, dur time.Duration) {
}
func (bpt *bsPeerTracker) logSuccess(p peer.ID, dur time.Duration) {
func (bpt *bsPeerTracker) logSuccess(p peer.ID, dur time.Duration, reqSize uint64) {
bpt.lk.Lock()
defer bpt.lk.Unlock()
@@ -145,10 +158,13 @@ func (bpt *bsPeerTracker) logSuccess(p peer.ID, dur time.Duration) {
}
pi.successes++
logTime(pi, dur)
if reqSize == 0 {
reqSize = 1
}
logTime(pi, dur/time.Duration(reqSize))
}
func (bpt *bsPeerTracker) logFailure(p peer.ID, dur time.Duration) {
func (bpt *bsPeerTracker) logFailure(p peer.ID, dur time.Duration, reqSize uint64) {
bpt.lk.Lock()
defer bpt.lk.Unlock()
@@ -160,7 +176,10 @@ func (bpt *bsPeerTracker) logFailure(p peer.ID, dur time.Duration) {
}
pi.failures++
logTime(pi, dur)
if reqSize == 0 {
reqSize = 1
}
logTime(pi, dur/time.Duration(reqSize))
}
func (bpt *bsPeerTracker) removePeer(p peer.ID) {
@@ -1,4 +1,4 @@
package blocksync
package exchange
import (
"time"
@@ -13,9 +13,17 @@ import (
"github.com/filecoin-project/lotus/chain/types"
)
var log = logging.Logger("blocksync")
var log = logging.Logger("chainxchg")
const BlockSyncProtocolID = "/fil/sync/blk/0.0.1"
const (
// BlockSyncProtocolID is the protocol ID of the former blocksync protocol.
// Deprecated.
BlockSyncProtocolID = "/fil/sync/blk/0.0.1"
// ChainExchangeProtocolID is the protocol ID of the chain exchange
// protocol.
ChainExchangeProtocolID = "/fil/chain/xchg/0.0.1"
)
// FIXME: Bumped from original 800 to this to accommodate `syncFork()`
// use of `GetBlocks()`. It seems the expectation of that API is to
@@ -25,14 +33,16 @@ const BlockSyncProtocolID = "/fil/sync/blk/0.0.1"
// qualifier to avoid "const initializer [...] is not a constant" error.)
var MaxRequestLength = uint64(build.ForkLengthThreshold)
// Extracted constants from the code.
// FIXME: Should be reviewed and confirmed.
const SUCCESS_PEER_TAG_VALUE = 25
const WRITE_REQ_DEADLINE = 5 * time.Second
const READ_RES_DEADLINE = WRITE_REQ_DEADLINE
const READ_RES_MIN_SPEED = 50 << 10
const SHUFFLE_PEERS_PREFIX = 5
const WRITE_RES_DEADLINE = 60 * time.Second
const (
// Extracted constants from the code.
// FIXME: Should be reviewed and confirmed.
SuccessPeerTagValue = 25
WriteReqDeadline = 5 * time.Second
ReadResDeadline = WriteReqDeadline
ReadResMinSpeed = 50 << 10
ShufflePeersPrefix = 5
WriteResDeadline = 60 * time.Second
)
// FIXME: Rename. Make private.
type Request struct {
@@ -117,7 +127,7 @@ func (res *Response) statusToError() error {
case NotFound:
return xerrors.Errorf("not found")
case GoAway:
return xerrors.Errorf("not handling 'go away' blocksync responses yet")
return xerrors.Errorf("not handling 'go away' chainxchg responses yet")
case InternalError:
return xerrors.Errorf("block sync peer errored: %s", res.ErrorMessage)
case BadRequest:
@@ -1,4 +1,4 @@
package blocksync
package exchange
import (
"bufio"
@@ -18,38 +18,25 @@ import (
inet "github.com/libp2p/go-libp2p-core/network"
)
// BlockSyncService is the component that services BlockSync requests from
// peers.
//
// BlockSync is the basic chain synchronization protocol of Filecoin. BlockSync
// is an RPC-oriented protocol, with a single operation to request blocks.
//
// A request contains a start anchor block (referred to with a CID), and a
// amount of blocks requested beyond the anchor (including the anchor itself).
//
// A client can also pass options, encoded as a 64-bit bitfield. Lotus supports
// two options at the moment:
//
// - include block contents
// - include block messages
//
// The response will include a status code, an optional message, and the
// response payload in case of success. The payload is a slice of serialized
// tipsets.
// FIXME: Rename to just `Server` (will be done later, see note on `BlockSync`).
type BlockSyncService struct {
// server implements exchange.Server. It services requests for the
// libp2p ChainExchange protocol.
type server struct {
cs *store.ChainStore
}
func NewBlockSyncService(cs *store.ChainStore) *BlockSyncService {
return &BlockSyncService{
var _ Server = (*server)(nil)
// NewServer creates a new libp2p-based exchange.Server. It services requests
// for the libp2p ChainExchange protocol.
func NewServer(cs *store.ChainStore) Server {
return &server{
cs: cs,
}
}
// Entry point of the service, handles `Request`s.
func (server *BlockSyncService) HandleStream(stream inet.Stream) {
ctx, span := trace.StartSpan(context.Background(), "blocksync.HandleStream")
// HandleStream implements Server.HandleStream. Refer to the godocs there.
func (s *server) HandleStream(stream inet.Stream) {
ctx, span := trace.StartSpan(context.Background(), "chainxchg.HandleStream")
defer span.End()
defer stream.Close() //nolint:errcheck
@@ -62,13 +49,13 @@ func (server *BlockSyncService) HandleStream(stream inet.Stream) {
log.Infow("block sync request",
"start", req.Head, "len", req.Length)
resp, err := server.processRequest(ctx, &req)
resp, err := s.processRequest(ctx, &req)
if err != nil {
log.Warn("failed to process request: ", err)
return
}
_ = stream.SetDeadline(time.Now().Add(WRITE_RES_DEADLINE))
_ = stream.SetDeadline(time.Now().Add(WriteResDeadline))
if err := cborutil.WriteCborRPC(stream, resp); err != nil {
_ = stream.SetDeadline(time.Time{})
log.Warnw("failed to write back response for handle stream",
@@ -80,10 +67,7 @@ func (server *BlockSyncService) HandleStream(stream inet.Stream) {
// Validate and service the request. We return either a protocol
// response or an internal error.
func (server *BlockSyncService) processRequest(
ctx context.Context,
req *Request,
) (*Response, error) {
func (s *server) processRequest(ctx context.Context, req *Request) (*Response, error) {
validReq, errResponse := validateRequest(ctx, req)
if errResponse != nil {
// The request did not pass validation, return the response
@@ -91,17 +75,14 @@ func (server *BlockSyncService) processRequest(
return errResponse, nil
}
return server.serviceRequest(ctx, validReq)
return s.serviceRequest(ctx, validReq)
}
// Validate request. We either return a `validatedRequest`, or an error
// `Response` indicating why we can't process it. We do not return any
// internal errors here, we just signal protocol ones.
func validateRequest(
ctx context.Context,
req *Request,
) (*validatedRequest, *Response) {
_, span := trace.StartSpan(ctx, "blocksync.ValidateRequest")
func validateRequest(ctx context.Context, req *Request) (*validatedRequest, *Response) {
_, span := trace.StartSpan(ctx, "chainxchg.ValidateRequest")
defer span.End()
validReq := validatedRequest{}
@@ -147,14 +128,11 @@ func validateRequest(
return &validReq, nil
}
func (server *BlockSyncService) serviceRequest(
ctx context.Context,
req *validatedRequest,
) (*Response, error) {
_, span := trace.StartSpan(ctx, "blocksync.ServiceRequest")
func (s *server) serviceRequest(ctx context.Context, req *validatedRequest) (*Response, error) {
_, span := trace.StartSpan(ctx, "chainxchg.ServiceRequest")
defer span.End()
chain, err := collectChainSegment(server.cs, req)
chain, err := collectChainSegment(s.cs, req)
if err != nil {
log.Warn("block sync request: collectChainSegment failed: ", err)
return &Response{
@@ -174,10 +152,7 @@ func (server *BlockSyncService) serviceRequest(
}, nil
}
func collectChainSegment(
cs *store.ChainStore,
req *validatedRequest,
) ([]*BSTipSet, error) {
func collectChainSegment(cs *store.ChainStore, req *validatedRequest) ([]*BSTipSet, error) {
var bstips []*BSTipSet
cur := req.head
@@ -221,37 +196,36 @@ func collectChainSegment(
func gatherMessages(cs *store.ChainStore, ts *types.TipSet) ([]*types.Message, [][]uint64, []*types.SignedMessage, [][]uint64, error) {
blsmsgmap := make(map[cid.Cid]uint64)
secpkmsgmap := make(map[cid.Cid]uint64)
var secpkmsgs []*types.SignedMessage
var blsmsgs []*types.Message
var secpkincl, blsincl [][]uint64
var blscids, secpkcids []cid.Cid
for _, block := range ts.Blocks() {
bmsgs, smsgs, err := cs.MessagesForBlock(block)
bc, sc, err := cs.ReadMsgMetaCids(block.Messages)
if err != nil {
return nil, nil, nil, nil, err
}
// FIXME: DRY. Use `chain.Message` interface.
bmi := make([]uint64, 0, len(bmsgs))
for _, m := range bmsgs {
i, ok := blsmsgmap[m.Cid()]
bmi := make([]uint64, 0, len(bc))
for _, m := range bc {
i, ok := blsmsgmap[m]
if !ok {
i = uint64(len(blsmsgs))
blsmsgs = append(blsmsgs, m)
blsmsgmap[m.Cid()] = i
i = uint64(len(blscids))
blscids = append(blscids, m)
blsmsgmap[m] = i
}
bmi = append(bmi, i)
}
blsincl = append(blsincl, bmi)
smi := make([]uint64, 0, len(smsgs))
for _, m := range smsgs {
i, ok := secpkmsgmap[m.Cid()]
smi := make([]uint64, 0, len(sc))
for _, m := range sc {
i, ok := secpkmsgmap[m]
if !ok {
i = uint64(len(secpkmsgs))
secpkmsgs = append(secpkmsgs, m)
secpkmsgmap[m.Cid()] = i
i = uint64(len(secpkcids))
secpkcids = append(secpkcids, m)
secpkmsgmap[m] = i
}
smi = append(smi, i)
@@ -259,5 +233,15 @@ func gatherMessages(cs *store.ChainStore, ts *types.TipSet) ([]*types.Message, [
secpkincl = append(secpkincl, smi)
}
blsmsgs, err := cs.LoadMessagesFromCids(blscids)
if err != nil {
return nil, nil, nil, nil, err
}
secpkmsgs, err := cs.LoadSignedMessagesFromCids(secpkcids)
if err != nil {
return nil, nil, nil, nil, err
}
return blsmsgs, blsincl, secpkmsgs, secpkincl, nil
}
+17 -15
View File
@@ -8,11 +8,13 @@ import (
"sync/atomic"
"time"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
saminer "github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/crypto"
block "github.com/ipfs/go-block-format"
"github.com/ipfs/go-blockservice"
"github.com/ipfs/go-cid"
@@ -46,7 +48,7 @@ const msgsPerBlock = 20
//nolint:deadcode,varcheck
var log = logging.Logger("gen")
var ValidWpostForTesting = []abi.PoStProof{{
var ValidWpostForTesting = []proof.PoStProof{{
ProofBytes: []byte("valid proof"),
}}
@@ -57,7 +59,7 @@ type ChainGen struct {
cs *store.ChainStore
beacon beacon.RandomBeacon
beacon beacon.Schedule
sm *stmgr.StateManager
@@ -250,7 +252,7 @@ func NewGeneratorWithSectors(numSectors int) (*ChainGen, error) {
miners := []address.Address{maddr1, maddr2}
beac := beacon.NewMockBeacon(time.Second)
beac := beacon.Schedule{{Start: 0, Beacon: beacon.NewMockBeacon(time.Second)}}
//beac, err := drand.NewDrandBeacon(tpl.Timestamp, build.BlockDelaySecs)
//if err != nil {
//return nil, xerrors.Errorf("creating drand beacon: %w", err)
@@ -336,7 +338,7 @@ func (cg *ChainGen) nextBlockProof(ctx context.Context, pts *types.TipSet, m add
prev := mbi.PrevBeaconEntry
entries, err := beacon.BeaconEntriesForBlock(ctx, cg.beacon, round, prev)
entries, err := beacon.BeaconEntriesForBlock(ctx, cg.beacon, round, pts.Height(), prev)
if err != nil {
return nil, nil, nil, xerrors.Errorf("get beacon entries for block: %w", err)
}
@@ -356,7 +358,7 @@ func (cg *ChainGen) nextBlockProof(ctx context.Context, pts *types.TipSet, m add
return nil, nil, nil, xerrors.Errorf("failed to cbor marshal address: %w", err)
}
if len(entries) == 0 {
if round > build.UpgradeSmokeHeight {
buf.Write(pts.MinTicket().VRFProof)
}
@@ -457,7 +459,7 @@ func (cg *ChainGen) NextTipSetFromMinersWithMessages(base *types.TipSet, miners
func (cg *ChainGen) makeBlock(parents *types.TipSet, m address.Address, vrfticket *types.Ticket,
eticket *types.ElectionProof, bvals []types.BeaconEntry, height abi.ChainEpoch,
wpost []abi.PoStProof, msgs []*types.SignedMessage) (*types.FullBlock, error) {
wpost []proof.PoStProof, msgs []*types.SignedMessage) (*types.FullBlock, error) {
var ts uint64
if cg.Timestamper != nil {
@@ -557,7 +559,7 @@ type mca struct {
w *wallet.Wallet
sm *stmgr.StateManager
pv ffiwrapper.Verifier
bcn beacon.RandomBeacon
bcn beacon.Schedule
}
func (mca mca) ChainGetRandomnessFromTickets(ctx context.Context, tsk types.TipSetKey, personalization crypto.DomainSeparationTag, randEpoch abi.ChainEpoch, entropy []byte) (abi.Randomness, error) {
@@ -588,7 +590,7 @@ func (mca mca) WalletSign(ctx context.Context, a address.Address, v []byte) (*cr
type WinningPoStProver interface {
GenerateCandidates(context.Context, abi.PoStRandomness, uint64) ([]uint64, error)
ComputeProof(context.Context, []abi.SectorInfo, abi.PoStRandomness) ([]abi.PoStProof, error)
ComputeProof(context.Context, []proof.SectorInfo, abi.PoStRandomness) ([]proof.PoStProof, error)
}
type wppProvider struct{}
@@ -597,7 +599,7 @@ func (wpp *wppProvider) GenerateCandidates(ctx context.Context, _ abi.PoStRandom
return []uint64{0}, nil
}
func (wpp *wppProvider) ComputeProof(context.Context, []abi.SectorInfo, abi.PoStRandomness) ([]abi.PoStProof, error) {
func (wpp *wppProvider) ComputeProof(context.Context, []proof.SectorInfo, abi.PoStRandomness) ([]proof.PoStProof, error) {
return ValidWpostForTesting, nil
}
@@ -664,15 +666,15 @@ type genFakeVerifier struct{}
var _ ffiwrapper.Verifier = (*genFakeVerifier)(nil)
func (m genFakeVerifier) VerifySeal(svi abi.SealVerifyInfo) (bool, error) {
func (m genFakeVerifier) VerifySeal(svi proof.SealVerifyInfo) (bool, error) {
return true, nil
}
func (m genFakeVerifier) VerifyWinningPoSt(ctx context.Context, info abi.WinningPoStVerifyInfo) (bool, error) {
func (m genFakeVerifier) VerifyWinningPoSt(ctx context.Context, info proof.WinningPoStVerifyInfo) (bool, error) {
panic("not supported")
}
func (m genFakeVerifier) VerifyWindowPoSt(ctx context.Context, info abi.WindowPoStVerifyInfo) (bool, error) {
func (m genFakeVerifier) VerifyWindowPoSt(ctx context.Context, info proof.WindowPoStVerifyInfo) (bool, error) {
panic("not supported")
}
+2 -2
View File
@@ -3,8 +3,8 @@ package gen
import (
"testing"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
+10 -3
View File
@@ -6,6 +6,8 @@ import (
"encoding/json"
"fmt"
"github.com/filecoin-project/go-state-types/network"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
cbor "github.com/ipfs/go-ipld-cbor"
@@ -14,13 +16,13 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/account"
"github.com/filecoin-project/specs-actors/actors/builtin/multisig"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/util/adt"
"github.com/filecoin-project/lotus/build"
@@ -404,6 +406,10 @@ func VerifyPreSealedData(ctx context.Context, cs *store.ChainStore, stateroot ci
verifNeeds := make(map[address.Address]abi.PaddedPieceSize)
var sum abi.PaddedPieceSize
nwv := func(context.Context, abi.ChainEpoch) network.Version {
return build.NewestNetworkVersion
}
vmopt := vm.VMOpts{
StateBase: stateroot,
Epoch: 0,
@@ -411,6 +417,7 @@ func VerifyPreSealedData(ctx context.Context, cs *store.ChainStore, stateroot ci
Bstore: cs.Blockstore(),
Syscalls: mkFakedSigSyscalls(cs.VMSys()),
CircSupplyCalc: nil,
NtwkVersion: nwv,
BaseFee: types.NewInt(0),
}
vm, err := vm.NewVM(&vmopt)
+12 -3
View File
@@ -6,6 +6,10 @@ import (
"fmt"
"math/rand"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/go-state-types/network"
"github.com/filecoin-project/lotus/chain/state"
"github.com/ipfs/go-cid"
@@ -14,15 +18,15 @@ import (
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/market"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/reward"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/runtime"
"github.com/filecoin-project/lotus/chain/store"
@@ -61,6 +65,10 @@ func SetupStorageMiners(ctx context.Context, cs *store.ChainStore, sroot cid.Cid
return big.Zero(), nil
}
nwv := func(context.Context, abi.ChainEpoch) network.Version {
return build.NewestNetworkVersion
}
vmopt := &vm.VMOpts{
StateBase: sroot,
Epoch: 0,
@@ -68,6 +76,7 @@ func SetupStorageMiners(ctx context.Context, cs *store.ChainStore, sroot cid.Cid
Bstore: cs.Blockstore(),
Syscalls: mkFakedSigSyscalls(cs.VMSys()),
CircSupplyCalc: csc,
NtwkVersion: nwv,
BaseFee: types.NewInt(0),
}
+1 -1
View File
@@ -3,7 +3,7 @@ package genesis
import (
"context"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/reward"
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"context"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
cbg "github.com/whyrusleeping/cbor-gen"
"golang.org/x/xerrors"
+1 -1
View File
@@ -3,7 +3,7 @@ package gen
import (
"context"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/util/adt"
cid "github.com/ipfs/go-cid"
cbor "github.com/ipfs/go-ipld-cbor"
+1 -1
View File
@@ -9,8 +9,8 @@ import (
ds "github.com/ipfs/go-datastore"
"github.com/ipfs/go-datastore/namespace"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/specs-actors/actors/abi"
)
type SlashFilter struct {
+2 -2
View File
@@ -4,8 +4,8 @@ import (
"context"
"sync"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"go.uber.org/fx"
"github.com/filecoin-project/specs-actors/actors/builtin"
+2 -2
View File
@@ -10,9 +10,9 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/crypto"
tutils "github.com/filecoin-project/specs-actors/support/testing"
"github.com/filecoin-project/lotus/api"
+1 -1
View File
@@ -9,7 +9,7 @@ import (
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
)
+299 -86
View File
@@ -6,12 +6,13 @@ import (
"errors"
"fmt"
"math"
stdbig "math/big"
"sort"
"sync"
"time"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/hashicorp/go-multierror"
lru "github.com/hashicorp/golang-lru"
"github.com/ipfs/go-cid"
@@ -45,7 +46,14 @@ var rbfDenomBig = types.NewInt(RbfDenom)
const RbfDenom = 256
var RepublishInterval = pubsub.TimeCacheDuration + time.Duration(5*build.BlockDelaySecs+build.PropagationDelaySecs)*time.Second
var RepublishInterval = time.Duration(10*build.BlockDelaySecs+build.PropagationDelaySecs) * time.Second
var minimumBaseFee = types.NewInt(uint64(build.MinimumBaseFee))
var baseFeeLowerBoundFactor = types.NewInt(10)
var MaxActorPendingMessages = 1000
var MaxNonceGap = uint64(4)
var (
ErrMessageTooBig = errors.New("message too big")
@@ -54,12 +62,16 @@ var (
ErrNonceTooLow = errors.New("message nonce too low")
ErrGasFeeCapTooLow = errors.New("gas fee cap too low")
ErrNotEnoughFunds = errors.New("not enough funds to execute transaction")
ErrInvalidToAddr = errors.New("message had invalid to address")
ErrBroadcastAnyway = errors.New("broadcasting message despite validation fail")
ErrRBFTooLowPremium = errors.New("replace by fee has too low GasPremium")
ErrSoftValidationFailure = errors.New("validation failure")
ErrRBFTooLowPremium = errors.New("replace by fee has too low GasPremium")
ErrTooManyPendingMessages = errors.New("too many pending messages for actor")
ErrNonceGap = errors.New("unfulfilled nonce gap")
ErrTryAgain = errors.New("state inconsistency while pushing message; please try again")
)
@@ -70,6 +82,18 @@ const (
localUpdates = "update"
)
// this is *temporary* mutilation until we have implemented uncapped miner penalties -- it will go
// away in the next fork.
var strictBaseFeeValidation = false
func init() {
// if the republish interval is too short compared to the pubsub timecache, adjust it
minInterval := pubsub.TimeCacheDuration + time.Duration(build.PropagationDelaySecs)
if RepublishInterval < minInterval {
RepublishInterval = minInterval
}
}
type MessagePool struct {
lk sync.Mutex
@@ -118,27 +142,52 @@ type MessagePool struct {
}
type msgSet struct {
msgs map[uint64]*types.SignedMessage
nextNonce uint64
msgs map[uint64]*types.SignedMessage
nextNonce uint64
requiredFunds *stdbig.Int
}
func newMsgSet() *msgSet {
func newMsgSet(nonce uint64) *msgSet {
return &msgSet{
msgs: make(map[uint64]*types.SignedMessage),
msgs: make(map[uint64]*types.SignedMessage),
nextNonce: nonce,
requiredFunds: stdbig.NewInt(0),
}
}
func (ms *msgSet) add(m *types.SignedMessage, mp *MessagePool) (bool, error) {
if len(ms.msgs) == 0 || m.Message.Nonce >= ms.nextNonce {
ms.nextNonce = m.Message.Nonce + 1
func ComputeMinRBF(curPrem abi.TokenAmount) abi.TokenAmount {
minPrice := types.BigAdd(curPrem, types.BigDiv(types.BigMul(curPrem, rbfNumBig), rbfDenomBig))
return types.BigAdd(minPrice, types.NewInt(1))
}
func (ms *msgSet) add(m *types.SignedMessage, mp *MessagePool, strict bool) (bool, error) {
nextNonce := ms.nextNonce
nonceGap := false
switch {
case m.Message.Nonce == nextNonce:
nextNonce++
// advance if we are filling a gap
for _, fillGap := ms.msgs[nextNonce]; fillGap; _, fillGap = ms.msgs[nextNonce] {
nextNonce++
}
case strict && m.Message.Nonce > nextNonce+MaxNonceGap:
return false, xerrors.Errorf("message nonce has too big a gap from expected nonce (Nonce: %d, nextNonce: %d): %w", m.Message.Nonce, nextNonce, ErrNonceGap)
case m.Message.Nonce > nextNonce:
nonceGap = true
}
exms, has := ms.msgs[m.Message.Nonce]
if has {
// refuse RBF if we have a gap
if strict && nonceGap {
return false, xerrors.Errorf("rejecting replace by fee because of nonce gap (Nonce: %d, nextNonce: %d): %w", m.Message.Nonce, nextNonce, ErrNonceGap)
}
if m.Cid() != exms.Cid() {
// check if RBF passes
minPrice := exms.Message.GasPremium
minPrice = types.BigAdd(minPrice, types.BigDiv(types.BigMul(minPrice, rbfNumBig), rbfDenomBig))
minPrice = types.BigAdd(minPrice, types.NewInt(1))
minPrice := ComputeMinRBF(exms.Message.GasPremium)
if types.BigCmp(m.Message.GasPremium, minPrice) >= 0 {
log.Infow("add with RBF", "oldpremium", exms.Message.GasPremium,
"newpremium", m.Message.GasPremium, "addr", m.Message.From, "nonce", m.Message.Nonce)
@@ -149,22 +198,87 @@ func (ms *msgSet) add(m *types.SignedMessage, mp *MessagePool) (bool, error) {
m.Message.From, m.Message.Nonce, minPrice, m.Message.GasPremium,
ErrRBFTooLowPremium)
}
} else {
return false, xerrors.Errorf("message from %s with nonce %d already in mpool: %w",
m.Message.From, m.Message.Nonce, ErrSoftValidationFailure)
}
ms.requiredFunds.Sub(ms.requiredFunds, exms.Message.RequiredFunds().Int)
//ms.requiredFunds.Sub(ms.requiredFunds, exms.Message.Value.Int)
}
if !has && strict && len(ms.msgs) > MaxActorPendingMessages {
log.Errorf("too many pending messages from actor %s", m.Message.From)
return false, ErrTooManyPendingMessages
}
if strict && nonceGap {
log.Warnf("adding nonce-gapped message from %s (nonce: %d, nextNonce: %d)",
m.Message.From, m.Message.Nonce, nextNonce)
}
ms.nextNonce = nextNonce
ms.msgs[m.Message.Nonce] = m
ms.requiredFunds.Add(ms.requiredFunds, m.Message.RequiredFunds().Int)
//ms.requiredFunds.Add(ms.requiredFunds, m.Message.Value.Int)
return !has, nil
}
func (ms *msgSet) rm(nonce uint64, applied bool) {
m, has := ms.msgs[nonce]
if !has {
if applied && nonce >= ms.nextNonce {
// we removed a message we did not know about because it was applied
// we need to adjust the nonce and check if we filled a gap
ms.nextNonce = nonce + 1
for _, fillGap := ms.msgs[ms.nextNonce]; fillGap; _, fillGap = ms.msgs[ms.nextNonce] {
ms.nextNonce++
}
}
return
}
ms.requiredFunds.Sub(ms.requiredFunds, m.Message.RequiredFunds().Int)
//ms.requiredFunds.Sub(ms.requiredFunds, m.Message.Value.Int)
delete(ms.msgs, nonce)
// adjust next nonce
if applied {
// we removed a (known) message because it was applied in a tipset
// we can't possibly have filled a gap in this case
if nonce >= ms.nextNonce {
ms.nextNonce = nonce + 1
}
return
}
// we removed a message because it was pruned
// we have to adjust the nonce if it creates a gap or rewinds state
if nonce < ms.nextNonce {
ms.nextNonce = nonce
}
}
func (ms *msgSet) getRequiredFunds(nonce uint64) types.BigInt {
requiredFunds := new(stdbig.Int).Set(ms.requiredFunds)
m, has := ms.msgs[nonce]
if has {
requiredFunds.Sub(requiredFunds, m.Message.RequiredFunds().Int)
//requiredFunds.Sub(requiredFunds, m.Message.Value.Int)
}
return types.BigInt{Int: requiredFunds}
}
func New(api Provider, ds dtypes.MetadataDS, netName dtypes.NetworkName) (*MessagePool, error) {
cache, _ := lru.New2Q(build.BlsSignatureCacheSize)
verifcache, _ := lru.New2Q(build.VerifSigCacheSize)
cfg, err := loadConfig(ds)
if err != nil {
if err != nil {
return nil, xerrors.Errorf("error loading mpool config: %w", err)
}
return nil, xerrors.Errorf("error loading mpool config: %w", err)
}
mp := &MessagePool{
@@ -257,13 +371,48 @@ func (mp *MessagePool) addLocal(m *types.SignedMessage, msgb []byte) error {
return nil
}
func (mp *MessagePool) verifyMsgBeforePush(m *types.SignedMessage, epoch abi.ChainEpoch) error {
// verifyMsgBeforeAdd verifies that the message meets the minimum criteria for block inclusio
// and whether the message has enough funds to be included in the next 20 blocks.
// If the message is not valid for block inclusion, it returns an error.
// For local messages, if the message can be included in the next 20 blocks, it returns true to
// signal that it should be immediately published. If the message cannot be included in the next 20
// blocks, it returns false so that the message doesn't immediately get published (and ignored by our
// peers); instead it will be published through the republish loop, once the base fee has fallen
// sufficiently.
// For non local messages, if the message cannot be included in the next 20 blocks it returns
// a (soft) validation error.
func (mp *MessagePool) verifyMsgBeforeAdd(m *types.SignedMessage, curTs *types.TipSet, local bool) (bool, error) {
epoch := curTs.Height()
minGas := vm.PricelistByEpoch(epoch).OnChainMessage(m.ChainLength())
if err := m.VMMessage().ValidForBlockInclusion(minGas.Total()); err != nil {
return xerrors.Errorf("message will not be included in a block: %w", err)
return false, xerrors.Errorf("message will not be included in a block: %w", err)
}
return nil
// this checks if the GasFeeCap is suffisciently high for inclusion in the next 20 blocks
// if the GasFeeCap is too low, we soft reject the message (Ignore in pubsub) and rely
// on republish to push it through later, if the baseFee has fallen.
// this is a defensive check that stops minimum baseFee spam attacks from overloading validation
// queues.
// Note that for local messages, we always add them so that they can be accepted and republished
// automatically.
publish := local
if strictBaseFeeValidation && len(curTs.Blocks()) > 0 {
baseFee := curTs.Blocks()[0].ParentBaseFee
baseFeeLowerBound := types.BigDiv(baseFee, baseFeeLowerBoundFactor)
if m.Message.GasFeeCap.LessThan(baseFeeLowerBound) {
if local {
log.Warnf("local message will not be immediately published because GasFeeCap doesn't meet the lower bound for inclusion in the next 20 blocks (GasFeeCap: %s, baseFeeLowerBound: %s)",
m.Message.GasFeeCap, baseFeeLowerBound)
publish = false
} else {
return false, xerrors.Errorf("GasFeeCap doesn't meet base fee lower bound for inclusion in the next 20 blocks (GasFeeCap: %s, baseFeeLowerBound: %s): %w",
m.Message.GasFeeCap, baseFeeLowerBound, ErrSoftValidationFailure)
}
}
}
return publish, nil
}
func (mp *MessagePool) Push(m *types.SignedMessage) (cid.Cid, error) {
@@ -278,26 +427,14 @@ func (mp *MessagePool) Push(m *types.SignedMessage) (cid.Cid, error) {
<-mp.addSema
}()
mp.curTsLk.Lock()
curTs := mp.curTs
epoch := curTs.Height()
mp.curTsLk.Unlock()
if err := mp.verifyMsgBeforePush(m, epoch); err != nil {
return cid.Undef, err
}
msgb, err := m.Serialize()
if err != nil {
return cid.Undef, err
}
mp.curTsLk.Lock()
if mp.curTs != curTs {
mp.curTsLk.Unlock()
return cid.Undef, ErrTryAgain
}
if err := mp.addTs(m, mp.curTs); err != nil {
publish, err := mp.addTs(m, mp.curTs, true)
if err != nil {
mp.curTsLk.Unlock()
return cid.Undef, err
}
@@ -310,7 +447,11 @@ func (mp *MessagePool) Push(m *types.SignedMessage) (cid.Cid, error) {
}
mp.lk.Unlock()
return m.Cid(), mp.api.PubSubPublish(build.MessagesTopic(mp.netName), msgb)
if publish {
err = mp.api.PubSubPublish(build.MessagesTopic(mp.netName), msgb)
}
return m.Cid(), err
}
func (mp *MessagePool) checkMessage(m *types.SignedMessage) error {
@@ -319,7 +460,7 @@ func (mp *MessagePool) checkMessage(m *types.SignedMessage) error {
return xerrors.Errorf("mpool message too large (%dB): %w", m.Size(), ErrMessageTooBig)
}
// Perform syntaxtic validation, minGas=0 as we check if correctly in select messages
// Perform syntactic validation, minGas=0 as we check the actual mingas before we add it
if err := m.Message.ValidForBlockInclusion(0); err != nil {
return xerrors.Errorf("message not valid for block inclusion: %w", err)
}
@@ -332,8 +473,12 @@ func (mp *MessagePool) checkMessage(m *types.SignedMessage) error {
return ErrMessageValueTooHigh
}
if m.Message.GasFeeCap.LessThan(minimumBaseFee) {
return ErrGasFeeCapTooLow
}
if err := mp.VerifyMsgSig(m); err != nil {
log.Warnf("mpooladd signature verification failed: %s", err)
log.Warnf("signature verification failed: %s", err)
return err
}
@@ -354,7 +499,9 @@ func (mp *MessagePool) Add(m *types.SignedMessage) error {
mp.curTsLk.Lock()
defer mp.curTsLk.Unlock()
return mp.addTs(m, mp.curTs)
_, err = mp.addTs(m, mp.curTs, false)
return err
}
func sigCacheKey(m *types.SignedMessage) (string, error) {
@@ -393,48 +540,107 @@ func (mp *MessagePool) VerifyMsgSig(m *types.SignedMessage) error {
return nil
}
func (mp *MessagePool) addTs(m *types.SignedMessage, curTs *types.TipSet) error {
func (mp *MessagePool) checkBalance(m *types.SignedMessage, curTs *types.TipSet) error {
balance, err := mp.getStateBalance(m.Message.From, curTs)
if err != nil {
return xerrors.Errorf("failed to check sender balance: %s: %w", err, ErrSoftValidationFailure)
}
requiredFunds := m.Message.RequiredFunds()
if balance.LessThan(requiredFunds) {
return xerrors.Errorf("not enough funds (required: %s, balance: %s): %w", types.FIL(requiredFunds), types.FIL(balance), ErrNotEnoughFunds)
}
// add Value for soft failure check
//requiredFunds = types.BigAdd(requiredFunds, m.Message.Value)
mset, ok := mp.pending[m.Message.From]
if ok {
requiredFunds = types.BigAdd(requiredFunds, mset.getRequiredFunds(m.Message.Nonce))
}
if balance.LessThan(requiredFunds) {
// Note: we fail here for ErrSoftValidationFailure to signal a soft failure because we might
// be out of sync.
return xerrors.Errorf("not enough funds including pending messages (required: %s, balance: %s): %w", types.FIL(requiredFunds), types.FIL(balance), ErrSoftValidationFailure)
}
return nil
}
func (mp *MessagePool) addTs(m *types.SignedMessage, curTs *types.TipSet, local bool) (bool, error) {
snonce, err := mp.getStateNonce(m.Message.From, curTs)
if err != nil {
return xerrors.Errorf("failed to look up actor state nonce: %s: %w", err, ErrBroadcastAnyway)
return false, xerrors.Errorf("failed to look up actor state nonce: %s: %w", err, ErrSoftValidationFailure)
}
if snonce > m.Message.Nonce {
return false, xerrors.Errorf("minimum expected nonce is %d: %w", snonce, ErrNonceTooLow)
}
mp.lk.Lock()
defer mp.lk.Unlock()
publish, err := mp.verifyMsgBeforeAdd(m, curTs, local)
if err != nil {
return false, err
}
if err := mp.checkBalance(m, curTs); err != nil {
return false, err
}
return publish, mp.addLocked(m, !local)
}
func (mp *MessagePool) addLoaded(m *types.SignedMessage) error {
err := mp.checkMessage(m)
if err != nil {
return err
}
mp.curTsLk.Lock()
defer mp.curTsLk.Unlock()
curTs := mp.curTs
snonce, err := mp.getStateNonce(m.Message.From, curTs)
if err != nil {
return xerrors.Errorf("failed to look up actor state nonce: %s: %w", err, ErrSoftValidationFailure)
}
if snonce > m.Message.Nonce {
return xerrors.Errorf("minimum expected nonce is %d: %w", snonce, ErrNonceTooLow)
}
balance, err := mp.getStateBalance(m.Message.From, curTs)
if err != nil {
return xerrors.Errorf("failed to check sender balance: %s: %w", err, ErrBroadcastAnyway)
}
if balance.LessThan(m.Message.RequiredFunds()) {
return xerrors.Errorf("not enough funds (required: %s, balance: %s): %w", types.FIL(m.Message.RequiredFunds()), types.FIL(balance), ErrNotEnoughFunds)
}
mp.lk.Lock()
defer mp.lk.Unlock()
return mp.addLocked(m)
_, err = mp.verifyMsgBeforeAdd(m, curTs, true)
if err != nil {
return err
}
if err := mp.checkBalance(m, curTs); err != nil {
return err
}
return mp.addLocked(m, false)
}
func (mp *MessagePool) addSkipChecks(m *types.SignedMessage) error {
mp.lk.Lock()
defer mp.lk.Unlock()
return mp.addLocked(m)
return mp.addLocked(m, false)
}
func (mp *MessagePool) addLocked(m *types.SignedMessage) error {
func (mp *MessagePool) addLocked(m *types.SignedMessage, strict bool) error {
log.Debugf("mpooladd: %s %d", m.Message.From, m.Message.Nonce)
if m.Signature.Type == crypto.SigTypeBLS {
mp.blsSigCache.Add(m.Cid(), m.Signature)
}
if m.Message.GasLimit > build.BlockGasLimit {
return xerrors.Errorf("given message has too high of a gas limit")
}
if _, err := mp.api.PutMessage(m); err != nil {
log.Warnf("mpooladd cs.PutMessage failed: %s", err)
return err
@@ -447,13 +653,18 @@ func (mp *MessagePool) addLocked(m *types.SignedMessage) error {
mset, ok := mp.pending[m.Message.From]
if !ok {
mset = newMsgSet()
nonce, err := mp.getStateNonce(m.Message.From, mp.curTs)
if err != nil {
return xerrors.Errorf("failed to get initial actor nonce: %w", err)
}
mset = newMsgSet(nonce)
mp.pending[m.Message.From] = mset
}
incr, err := mset.add(m, mp)
incr, err := mset.add(m, mp, strict)
if err != nil {
log.Info(err)
log.Debug(err)
return err
}
@@ -562,6 +773,16 @@ func (mp *MessagePool) PushWithNonce(ctx context.Context, addr address.Address,
return nil, err
}
err = mp.checkMessage(msg)
if err != nil {
return nil, err
}
msgb, err := msg.Serialize()
if err != nil {
return nil, err
}
// reacquire the locks and check state for consistency
mp.curTsLk.Lock()
defer mp.curTsLk.Unlock()
@@ -582,33 +803,37 @@ func (mp *MessagePool) PushWithNonce(ctx context.Context, addr address.Address,
return nil, ErrTryAgain
}
if err := mp.verifyMsgBeforePush(msg, mp.curTs.Height()); err != nil {
return nil, err
}
msgb, err := msg.Serialize()
publish, err := mp.verifyMsgBeforeAdd(msg, curTs, true)
if err != nil {
return nil, err
}
if err := mp.addLocked(msg); err != nil {
if err := mp.checkBalance(msg, curTs); err != nil {
return nil, err
}
if err := mp.addLocked(msg, false); err != nil {
return nil, xerrors.Errorf("add locked failed: %w", err)
}
if err := mp.addLocal(msg, msgb); err != nil {
log.Errorf("addLocal failed: %+v", err)
}
return msg, mp.api.PubSubPublish(build.MessagesTopic(mp.netName), msgb)
if publish {
err = mp.api.PubSubPublish(build.MessagesTopic(mp.netName), msgb)
}
return msg, err
}
func (mp *MessagePool) Remove(from address.Address, nonce uint64) {
func (mp *MessagePool) Remove(from address.Address, nonce uint64, applied bool) {
mp.lk.Lock()
defer mp.lk.Unlock()
mp.remove(from, nonce)
mp.remove(from, nonce, applied)
}
func (mp *MessagePool) remove(from address.Address, nonce uint64) {
func (mp *MessagePool) remove(from address.Address, nonce uint64, applied bool) {
mset, ok := mp.pending[from]
if !ok {
return
@@ -625,22 +850,10 @@ func (mp *MessagePool) remove(from address.Address, nonce uint64) {
// NB: This deletes any message with the given nonce. This makes sense
// as two messages with the same sender cannot have the same nonce
delete(mset.msgs, nonce)
mset.rm(nonce, applied)
if len(mset.msgs) == 0 {
delete(mp.pending, from)
} else {
var max uint64
for nonce := range mset.msgs {
if max < nonce {
max = nonce
}
}
if max < nonce {
max = nonce // we could have not seen the removed message before
}
mset.nextNonce = max + 1
}
}
@@ -708,7 +921,7 @@ func (mp *MessagePool) HeadChange(revert []*types.TipSet, apply []*types.TipSet)
rm := func(from address.Address, nonce uint64) {
s, ok := rmsgs[from]
if !ok {
mp.Remove(from, nonce)
mp.Remove(from, nonce, true)
return
}
@@ -717,7 +930,7 @@ func (mp *MessagePool) HeadChange(revert []*types.TipSet, apply []*types.TipSet)
return
}
mp.Remove(from, nonce)
mp.Remove(from, nonce, true)
}
maybeRepub := func(cid cid.Cid) {
@@ -1019,7 +1232,7 @@ func (mp *MessagePool) loadLocal() error {
return xerrors.Errorf("unmarshaling local message: %w", err)
}
if err := mp.Add(&sm); err != nil {
if err := mp.addLoaded(&sm); err != nil {
if xerrors.Is(err, ErrNonceTooLow) {
continue // todo: drop the message from local cache (if above certain confidence threshold)
}
+31 -2
View File
@@ -7,6 +7,8 @@ import (
"testing"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/chain/messagepool/gasguess"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/types/mock"
@@ -14,7 +16,6 @@ import (
_ "github.com/filecoin-project/lotus/lib/sigs/bls"
_ "github.com/filecoin-project/lotus/lib/sigs/secp"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
logging "github.com/ipfs/go-log/v2"
@@ -34,6 +35,8 @@ type testMpoolAPI struct {
tipsets []*types.TipSet
published int
baseFee types.BigInt
}
func newTestMpoolAPI() *testMpoolAPI {
@@ -41,6 +44,7 @@ func newTestMpoolAPI() *testMpoolAPI {
bmsgs: make(map[cid.Cid][]*types.SignedMessage),
statenonce: make(map[address.Address]uint64),
balance: make(map[address.Address]types.BigInt),
baseFee: types.NewInt(100),
}
genesis := mock.MkBlock(nil, 1, 1)
tma.tipsets = append(tma.tipsets, mock.TipSet(genesis))
@@ -53,6 +57,13 @@ func (tma *testMpoolAPI) nextBlock() *types.BlockHeader {
return newBlk
}
func (tma *testMpoolAPI) nextBlockWithHeight(height uint64) *types.BlockHeader {
newBlk := mock.MkBlock(tma.tipsets[len(tma.tipsets)-1], 1, 1)
newBlk.Height = abi.ChainEpoch(height)
tma.tipsets = append(tma.tipsets, mock.TipSet(newBlk))
return newBlk
}
func (tma *testMpoolAPI) applyBlock(t *testing.T, b *types.BlockHeader) {
t.Helper()
if err := tma.cb(nil, []*types.TipSet{mock.TipSet(b)}); err != nil {
@@ -182,7 +193,7 @@ func (tma *testMpoolAPI) LoadTipSet(tsk types.TipSetKey) (*types.TipSet, error)
}
func (tma *testMpoolAPI) ChainComputeBaseFee(ctx context.Context, ts *types.TipSet) (types.BigInt, error) {
return types.NewInt(100), nil
return tma.baseFee, nil
}
func assertNonce(t *testing.T, mp *MessagePool, addr address.Address, val uint64) {
@@ -352,6 +363,12 @@ func TestRevertMessages(t *testing.T) {
}
func TestPruningSimple(t *testing.T) {
oldMaxNonceGap := MaxNonceGap
MaxNonceGap = 1000
defer func() {
MaxNonceGap = oldMaxNonceGap
}()
tma := newTestMpoolAPI()
w, err := wallet.NewWallet(wallet.NewMemKeyStore())
@@ -373,6 +390,7 @@ func TestPruningSimple(t *testing.T) {
if err != nil {
t.Fatal(err)
}
tma.setBalance(sender, 1) // in FIL
target := mock.Address(1001)
for i := 0; i < 5; i++ {
@@ -430,6 +448,8 @@ func TestLoadLocal(t *testing.T) {
t.Fatal(err)
}
tma.setBalance(a1, 1) // in FIL
tma.setBalance(a2, 1) // in FIL
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
msgs := make(map[cid.Cid]struct{})
for i := 0; i < 10; i++ {
@@ -500,6 +520,8 @@ func TestClearAll(t *testing.T) {
t.Fatal(err)
}
tma.setBalance(a1, 1) // in FIL
tma.setBalance(a2, 1) // in FIL
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
for i := 0; i < 10; i++ {
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(i+1))
@@ -552,6 +574,9 @@ func TestClearNonLocal(t *testing.T) {
t.Fatal(err)
}
tma.setBalance(a1, 1) // in FIL
tma.setBalance(a2, 1) // in FIL
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
for i := 0; i < 10; i++ {
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(i+1))
@@ -619,6 +644,10 @@ func TestUpdates(t *testing.T) {
}
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
tma.setBalance(a1, 1) // in FIL
tma.setBalance(a2, 1) // in FIL
for i := 0; i < 10; i++ {
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(i+1))
_, err := mp.Push(m)
+1 -1
View File
@@ -98,7 +98,7 @@ keepLoop:
// and remove all messages that are still in pruneMsgs after processing the chains
log.Infof("Pruning %d messages", len(pruneMsgs))
for _, m := range pruneMsgs {
mp.remove(m.Message.From, m.Message.Nonce)
mp.remove(m.Message.From, m.Message.Nonce, false)
}
return nil
+33 -16
View File
@@ -3,6 +3,7 @@ package messagepool
import (
"context"
"sort"
"time"
"golang.org/x/xerrors"
@@ -15,6 +16,8 @@ import (
const repubMsgLimit = 30
var RepublishBatchDelay = 100 * time.Millisecond
func (mp *MessagePool) republishPendingMessages() error {
mp.curTsLk.Lock()
ts := mp.curTs
@@ -25,6 +28,11 @@ func (mp *MessagePool) republishPendingMessages() error {
return xerrors.Errorf("computing basefee: %w", err)
}
baseFeeLowerBound := types.BigDiv(baseFee, baseFeeLowerBoundFactor)
if baseFeeLowerBoundFactor.LessThan(minimumBaseFee) {
baseFeeLowerBound = minimumBaseFee
}
pending := make(map[address.Address]map[uint64]*types.SignedMessage)
mp.lk.Lock()
mp.republished = nil // clear this to avoid races triggering an early republish
@@ -52,7 +60,11 @@ func (mp *MessagePool) republishPendingMessages() error {
var chains []*msgChain
for actor, mset := range pending {
next := mp.createMessageChains(actor, mset, baseFee, ts)
// We use the baseFee lower bound for createChange so that we optimistically include
// chains that might become profitable in the next 20 blocks.
// We still check the lowerBound condition for individual messages so that we don't send
// messages that will be rejected by the mpool spam protector, so this is safe to do.
next := mp.createMessageChains(actor, mset, baseFeeLowerBound, ts)
chains = append(chains, next...)
}
@@ -64,15 +76,10 @@ func (mp *MessagePool) republishPendingMessages() error {
return chains[i].Before(chains[j])
})
// we don't republish negative performing chains; this is an error that will be screamed
// at the user
if chains[0].gasPerf < 0 {
return xerrors.Errorf("skipping republish: all message chains have negative gas performance; best gas performance: %f", chains[0].gasPerf)
}
gasLimit := int64(build.BlockGasLimit)
minGas := int64(gasguess.MinGas)
var msgs []*types.SignedMessage
loop:
for i := 0; i < len(chains); {
chain := chains[i]
@@ -86,12 +93,6 @@ func (mp *MessagePool) republishPendingMessages() error {
break
}
// we don't republish negative performing chains, as they won't be included in
// a block anyway
if chain.gasPerf < 0 {
break
}
// has the chain been invalidated?
if !chain.valid {
i++
@@ -100,15 +101,25 @@ func (mp *MessagePool) republishPendingMessages() error {
// does it fit in a block?
if chain.gasLimit <= gasLimit {
gasLimit -= chain.gasLimit
msgs = append(msgs, chain.msgs...)
// check the baseFee lower bound -- only republish messages that can be included in the chain
// within the next 20 blocks.
for _, m := range chain.msgs {
if !allowNegativeChains(ts.Height()) && m.Message.GasFeeCap.LessThan(baseFeeLowerBound) {
chain.Invalidate()
continue loop
}
gasLimit -= m.Message.GasLimit
msgs = append(msgs, m)
}
// we processed the whole chain, advance
i++
continue
}
// we can't fit the current chain but there is gas to spare
// trim it and push it down
chain.Trim(gasLimit, mp, baseFee, ts, false)
chain.Trim(gasLimit, mp, baseFee, true)
for j := i; j < len(chains)-1; j++ {
if chains[j].Before(chains[j+1]) {
break
@@ -131,6 +142,12 @@ func (mp *MessagePool) republishPendingMessages() error {
}
count++
if count < len(msgs) {
// this delay is here to encourage the pubsub subsystem to process the messages serially
// and avoid creating nonce gaps because of concurrent validation.
time.Sleep(RepublishBatchDelay)
}
}
// track most recently republished messages
+7 -1
View File
@@ -4,14 +4,20 @@ import (
"testing"
"time"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/chain/messagepool/gasguess"
"github.com/filecoin-project/lotus/chain/wallet"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/ipfs/go-datastore"
)
func TestRepubMessages(t *testing.T) {
oldRepublishBatchDelay := RepublishBatchDelay
RepublishBatchDelay = time.Microsecond
defer func() {
RepublishBatchDelay = oldRepublishBatchDelay
}()
tma := newTestMpoolAPI()
ds := datastore.NewMapDatastore()
+141 -29
View File
@@ -3,21 +3,29 @@ package messagepool
import (
"context"
"math/big"
"math/rand"
"sort"
"time"
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
tbig "github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/messagepool/gasguess"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/vm"
abig "github.com/filecoin-project/specs-actors/actors/abi/big"
)
var bigBlockGasLimit = big.NewInt(build.BlockGasLimit)
// this is *temporary* mutilation until we have implemented uncapped miner penalties -- it will go
// away in the next fork.
func allowNegativeChains(epoch abi.ChainEpoch) bool {
return epoch < build.UpgradeBreezeHeight+5
}
const MaxBlocks = 15
type msgChain struct {
@@ -100,9 +108,9 @@ func (mp *MessagePool) selectMessagesOptimal(curTs, ts *types.TipSet, tq float64
return chains[i].Before(chains[j])
})
if len(chains) != 0 && chains[0].gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && len(chains) != 0 && chains[0].gasPerf < 0 {
log.Warnw("all messages in mpool have non-positive gas performance", "bestGasPerf", chains[0].gasPerf)
return nil, nil
return result, nil
}
// 3. Parition chains into blocks (without trimming)
@@ -153,7 +161,7 @@ func (mp *MessagePool) selectMessagesOptimal(curTs, ts *types.TipSet, tq float64
last := len(chains)
for i, chain := range chains {
// did we run out of performing chains?
if chain.gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && chain.gasPerf < 0 {
break
}
@@ -191,9 +199,11 @@ func (mp *MessagePool) selectMessagesOptimal(curTs, ts *types.TipSet, tq float64
gasLimit -= chainGasLimit
// resort to account for already merged chains and effective performance adjustments
sort.Slice(chains[i+1:], func(i, j int) bool {
// the sort *must* be stable or we end up getting negative gasPerfs pushed up.
sort.SliceStable(chains[i+1:], func(i, j int) bool {
return chains[i].BeforeEffective(chains[j])
})
continue
}
@@ -217,7 +227,7 @@ tailLoop:
for gasLimit >= minGas && last < len(chains) {
// trim if necessary
if chains[last].gasLimit > gasLimit {
chains[last].Trim(gasLimit, mp, baseFee, ts, false)
chains[last].Trim(gasLimit, mp, baseFee, allowNegativeChains(curTs.Height()))
}
// push down if it hasn't been invalidated
@@ -243,7 +253,7 @@ tailLoop:
}
// if gasPerf < 0 we have no more profitable chains
if chain.gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && chain.gasPerf < 0 {
break tailLoop
}
@@ -284,7 +294,7 @@ tailLoop:
}
// dependencies fit, just trim it
chain.Trim(gasLimit-depGasLimit, mp, baseFee, ts, false)
chain.Trim(gasLimit-depGasLimit, mp, baseFee, allowNegativeChains(curTs.Height()))
last += i
continue tailLoop
}
@@ -297,6 +307,79 @@ tailLoop:
log.Infow("pack tail chains done", "took", dt)
}
// if we have gasLimit to spare, pick some random (non-negative) chains to fill the block
// we pick randomly so that we minimize the probability of duplication among all miners
if gasLimit >= minGas {
randomCount := 0
startRandom := time.Now()
shuffleChains(chains)
for _, chain := range chains {
// have we filled the block
if gasLimit < minGas {
break
}
// has it been merged or invalidated?
if chain.merged || !chain.valid {
continue
}
// is it negative?
if !allowNegativeChains(curTs.Height()) && chain.gasPerf < 0 {
continue
}
// compute the dependencies that must be merged and the gas limit including deps
chainGasLimit := chain.gasLimit
depGasLimit := int64(0)
var chainDeps []*msgChain
for curChain := chain.prev; curChain != nil && !curChain.merged; curChain = curChain.prev {
chainDeps = append(chainDeps, curChain)
chainGasLimit += curChain.gasLimit
depGasLimit += curChain.gasLimit
}
// do the deps fit? if the deps won't fit, invalidate the chain
if depGasLimit > gasLimit {
chain.Invalidate()
continue
}
// do they fit as is? if it doesn't, trim to make it fit if possible
if chainGasLimit > gasLimit {
chain.Trim(gasLimit-depGasLimit, mp, baseFee, allowNegativeChains(curTs.Height()))
if !chain.valid {
continue
}
}
// include it together with all dependencies
for i := len(chainDeps) - 1; i >= 0; i-- {
curChain := chainDeps[i]
curChain.merged = true
result = append(result, curChain.msgs...)
randomCount += len(curChain.msgs)
}
chain.merged = true
result = append(result, chain.msgs...)
randomCount += len(chain.msgs)
gasLimit -= chainGasLimit
}
if dt := time.Since(startRandom); dt > time.Millisecond {
log.Infow("pack random tail chains done", "took", dt)
}
if randomCount > 0 {
log.Warnf("optimal selection failed to pack a block; picked %d messages with random selection",
randomCount)
}
}
return result, nil
}
@@ -349,9 +432,9 @@ func (mp *MessagePool) selectMessagesGreedy(curTs, ts *types.TipSet) ([]*types.S
return chains[i].Before(chains[j])
})
if len(chains) != 0 && chains[0].gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && len(chains) != 0 && chains[0].gasPerf < 0 {
log.Warnw("all messages in mpool have non-positive gas performance", "bestGasPerf", chains[0].gasPerf)
return nil, nil
return result, nil
}
// 3. Merge the head chains to produce the list of messages selected for inclusion, subject to
@@ -360,7 +443,7 @@ func (mp *MessagePool) selectMessagesGreedy(curTs, ts *types.TipSet) ([]*types.S
last := len(chains)
for i, chain := range chains {
// did we run out of performing chains?
if chain.gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && chain.gasPerf < 0 {
break
}
@@ -389,7 +472,7 @@ func (mp *MessagePool) selectMessagesGreedy(curTs, ts *types.TipSet) ([]*types.S
tailLoop:
for gasLimit >= minGas && last < len(chains) {
// trim
chains[last].Trim(gasLimit, mp, baseFee, ts, false)
chains[last].Trim(gasLimit, mp, baseFee, allowNegativeChains(curTs.Height()))
// push down if it hasn't been invalidated
if chains[last].valid {
@@ -409,7 +492,7 @@ tailLoop:
}
// if gasPerf < 0 we have no more profitable chains
if chain.gasPerf < 0 {
if !allowNegativeChains(curTs.Height()) && chain.gasPerf < 0 {
break tailLoop
}
@@ -462,15 +545,27 @@ func (mp *MessagePool) selectPriorityMessages(pending map[address.Address]map[ui
}
}
if len(chains) == 0 {
return nil, gasLimit
}
// 2. Sort the chains
sort.Slice(chains, func(i, j int) bool {
return chains[i].Before(chains[j])
})
// 3. Merge chains until the block limit; we are willing to include negative performing chains
// as these are messages from our own miners
if !allowNegativeChains(ts.Height()) && len(chains) != 0 && chains[0].gasPerf < 0 {
log.Warnw("all priority messages in mpool have negative gas performance", "bestGasPerf", chains[0].gasPerf)
return nil, gasLimit
}
// 3. Merge chains until the block limit, as long as they have non-negative gas performance
last := len(chains)
for i, chain := range chains {
if !allowNegativeChains(ts.Height()) && chain.gasPerf < 0 {
break
}
if chain.gasLimit <= gasLimit {
gasLimit -= chain.gasLimit
result = append(result, chain.msgs...)
@@ -484,8 +579,8 @@ func (mp *MessagePool) selectPriorityMessages(pending map[address.Address]map[ui
tailLoop:
for gasLimit >= minGas && last < len(chains) {
// trim, without discarding negative performing messages
chains[last].Trim(gasLimit, mp, baseFee, ts, true)
// trim, discarding negative performing messages
chains[last].Trim(gasLimit, mp, baseFee, allowNegativeChains(ts.Height()))
// push down if it hasn't been invalidated
if chains[last].valid {
@@ -503,6 +598,12 @@ tailLoop:
if !chain.valid {
continue
}
// if gasPerf < 0 we have no more profitable chains
if !allowNegativeChains(ts.Height()) && chain.gasPerf < 0 {
break tailLoop
}
// does it fit in the bock?
if chain.gasLimit <= gasLimit {
gasLimit -= chain.gasLimit
@@ -515,9 +616,9 @@ tailLoop:
continue tailLoop
}
// the merge loop ended after processing all the chains and we probably still have gas to spare
// -- mark the end.
last = len(chains)
// the merge loop ended after processing all the chains and we probably still have gas to spare;
// end the loop
break
}
return result, gasLimit
@@ -567,16 +668,18 @@ func (mp *MessagePool) getPendingMessages(curTs, ts *types.TipSet) (map[address.
return result, nil
}
func (mp *MessagePool) getGasReward(msg *types.SignedMessage, baseFee types.BigInt, ts *types.TipSet) *big.Int {
func (*MessagePool) getGasReward(msg *types.SignedMessage, baseFee types.BigInt) *big.Int {
maxPremium := types.BigSub(msg.Message.GasFeeCap, baseFee)
if types.BigCmp(maxPremium, msg.Message.GasPremium) < 0 {
if types.BigCmp(maxPremium, msg.Message.GasPremium) > 0 {
maxPremium = msg.Message.GasPremium
}
gasReward := abig.Mul(maxPremium, types.NewInt(uint64(msg.Message.GasLimit)))
gasReward := tbig.Mul(maxPremium, types.NewInt(uint64(msg.Message.GasLimit)))
return gasReward.Int
}
func (mp *MessagePool) getGasPerf(gasReward *big.Int, gasLimit int64) float64 {
func (*MessagePool) getGasPerf(gasReward *big.Int, gasLimit int64) float64 {
// gasPerf = gasReward * build.BlockGasLimit / gasLimit
a := new(big.Rat).SetInt(new(big.Int).Mul(gasReward, bigBlockGasLimit))
b := big.NewRat(1, gasLimit)
@@ -654,7 +757,7 @@ func (mp *MessagePool) createMessageChains(actor address.Address, mset map[uint6
balance = new(big.Int).Sub(balance, value)
}
gasReward := mp.getGasReward(m, baseFee, ts)
gasReward := mp.getGasReward(m, baseFee)
rewards = append(rewards, gasReward)
}
@@ -755,10 +858,10 @@ func (mc *msgChain) Before(other *msgChain) bool {
(mc.gasPerf == other.gasPerf && mc.gasReward.Cmp(other.gasReward) > 0)
}
func (mc *msgChain) Trim(gasLimit int64, mp *MessagePool, baseFee types.BigInt, ts *types.TipSet, priority bool) {
func (mc *msgChain) Trim(gasLimit int64, mp *MessagePool, baseFee types.BigInt, allowNegative bool) {
i := len(mc.msgs) - 1
for i >= 0 && (mc.gasLimit > gasLimit || (!priority && mc.gasPerf < 0)) {
gasReward := mp.getGasReward(mc.msgs[i], baseFee, ts)
for i >= 0 && (mc.gasLimit > gasLimit || (!allowNegative && mc.gasPerf < 0)) {
gasReward := mp.getGasReward(mc.msgs[i], baseFee)
mc.gasReward = new(big.Int).Sub(mc.gasReward, gasReward)
mc.gasLimit -= mc.msgs[i].Message.GasLimit
if mc.gasLimit > 0 {
@@ -821,7 +924,16 @@ func (mc *msgChain) SetNullEffectivePerf() {
func (mc *msgChain) BeforeEffective(other *msgChain) bool {
// move merged chains to the front so we can discard them earlier
return (mc.merged && !other.merged) || mc.effPerf > other.effPerf ||
return (mc.merged && !other.merged) ||
(mc.gasPerf >= 0 && other.gasPerf < 0) ||
mc.effPerf > other.effPerf ||
(mc.effPerf == other.effPerf && mc.gasPerf > other.gasPerf) ||
(mc.effPerf == other.effPerf && mc.gasPerf == other.gasPerf && mc.gasReward.Cmp(other.gasReward) > 0)
}
func shuffleChains(lst []*msgChain) {
for i := range lst {
j := rand.Intn(i + 1)
lst[i], lst[j] = lst[j], lst[i]
}
}
+323 -4
View File
@@ -1,20 +1,26 @@
package messagepool
import (
"compress/gzip"
"context"
"encoding/json"
"fmt"
"io"
"math"
"math/big"
"math/rand"
"os"
"sort"
"testing"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/messagepool/gasguess"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/types/mock"
"github.com/filecoin-project/lotus/chain/wallet"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
@@ -23,6 +29,11 @@ import (
logging "github.com/ipfs/go-log"
)
func init() {
// bump this for the selection tests
MaxActorPendingMessages = 1000000
}
func makeTestMessage(w *wallet.Wallet, from, to address.Address, nonce uint64, gasLimit int64, gasPrice uint64) *types.SignedMessage {
msg := &types.Message{
From: from,
@@ -364,6 +375,12 @@ func TestMessageChainSkipping(t *testing.T) {
}
func TestBasicMessageSelection(t *testing.T) {
oldMaxNonceGap := MaxNonceGap
MaxNonceGap = 1000
defer func() {
MaxNonceGap = oldMaxNonceGap
}()
mp, tma := makeTestMpool()
// the actors
@@ -716,6 +733,102 @@ func TestPriorityMessageSelection2(t *testing.T) {
}
}
func TestPriorityMessageSelection3(t *testing.T) {
t.Skip("reenable after removing allow negative")
mp, tma := makeTestMpool()
// the actors
w1, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a1, err := w1.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
w2, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a2, err := w2.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
block := tma.nextBlock()
ts := mock.TipSet(block)
tma.applyBlock(t, block)
gasLimit := gasguess.Costs[gasguess.CostKey{Code: builtin.StorageMarketActorCodeID, M: 2}]
tma.setBalance(a1, 1) // in FIL
tma.setBalance(a2, 1) // in FIL
mp.cfg.PriorityAddrs = []address.Address{a1}
tma.baseFee = types.NewInt(1000)
nMessages := 10
for i := 0; i < nMessages; i++ {
bias := (nMessages - i) / 3
m := makeTestMessage(w1, a1, a2, uint64(i), gasLimit, uint64(1000+i%3+bias))
mustAdd(t, mp, m)
// messages from a2 have negative performance
m = makeTestMessage(w2, a2, a1, uint64(i), gasLimit, 100)
mustAdd(t, mp, m)
}
// test greedy selection
msgs, err := mp.SelectMessages(ts, 1.0)
if err != nil {
t.Fatal(err)
}
expectedMsgs := 10
if len(msgs) != expectedMsgs {
t.Fatalf("expected %d messages but got %d", expectedMsgs, len(msgs))
}
// all messages must be from a1
nextNonce := uint64(0)
for _, m := range msgs {
if m.Message.From != a1 {
t.Fatal("expected messages from a1 before messages from a2")
}
if m.Message.Nonce != nextNonce {
t.Fatalf("expected nonce %d but got %d", nextNonce, m.Message.Nonce)
}
nextNonce++
}
// test optimal selection
msgs, err = mp.SelectMessages(ts, 0.1)
if err != nil {
t.Fatal(err)
}
expectedMsgs = 10
if len(msgs) != expectedMsgs {
t.Fatalf("expected %d messages but got %d", expectedMsgs, len(msgs))
}
// all messages must be from a1
nextNonce = uint64(0)
for _, m := range msgs {
if m.Message.From != a1 {
t.Fatal("expected messages from a1 before messages from a2")
}
if m.Message.Nonce != nextNonce {
t.Fatalf("expected nonce %d but got %d", nextNonce, m.Message.Nonce)
}
nextNonce++
}
}
func TestOptimalMessageSelection1(t *testing.T) {
// this test uses just a single actor sending messages with a low tq
// the chain depenent merging algorithm should pick messages from the actor
@@ -1050,17 +1163,17 @@ func testCompetitiveMessageSelection(t *testing.T, rng *rand.Rand, getPremium fu
greedyReward := big.NewInt(0)
for _, m := range greedyMsgs {
greedyReward.Add(greedyReward, mp.getGasReward(m, baseFee, ts))
greedyReward.Add(greedyReward, mp.getGasReward(m, baseFee))
}
optReward := big.NewInt(0)
for _, m := range optMsgs {
optReward.Add(optReward, mp.getGasReward(m, baseFee, ts))
optReward.Add(optReward, mp.getGasReward(m, baseFee))
}
bestTqReward := big.NewInt(0)
for _, m := range bestMsgs {
bestTqReward.Add(bestTqReward, mp.getGasReward(m, baseFee, ts))
bestTqReward.Add(bestTqReward, mp.getGasReward(m, baseFee))
}
totalBestTQReward += float64(bestTqReward.Uint64())
@@ -1141,3 +1254,209 @@ func TestCompetitiveMessageSelectionZipf(t *testing.T) {
t.Logf("Average reward boost across all seeds: %f", rewardBoost)
t.Logf("Average reward of best ticket across all seeds: %f", tqReward)
}
func TestGasReward(t *testing.T) {
tests := []struct {
Premium uint64
FeeCap uint64
BaseFee uint64
GasReward int64
}{
{Premium: 100, FeeCap: 200, BaseFee: 100, GasReward: 100},
{Premium: 100, FeeCap: 200, BaseFee: 210, GasReward: -10},
{Premium: 200, FeeCap: 250, BaseFee: 210, GasReward: 40},
{Premium: 200, FeeCap: 250, BaseFee: 2000, GasReward: -1750},
}
mp := new(MessagePool)
for _, test := range tests {
test := test
t.Run(fmt.Sprintf("%v", test), func(t *testing.T) {
msg := &types.SignedMessage{
Message: types.Message{
GasLimit: 10,
GasFeeCap: types.NewInt(test.FeeCap),
GasPremium: types.NewInt(test.Premium),
},
}
rew := mp.getGasReward(msg, types.NewInt(test.BaseFee))
if rew.Cmp(big.NewInt(test.GasReward*10)) != 0 {
t.Errorf("bad reward: expected %d, got %s", test.GasReward*10, rew)
}
})
}
}
func TestRealWorldSelection(t *testing.T) {
// load test-messages.json.gz and rewrite the messages so that
// 1) we map each real actor to a test actor so that we can sign the messages
// 2) adjust the nonces so that they start from 0
file, err := os.Open("test-messages.json.gz")
if err != nil {
t.Fatal(err)
}
gzr, err := gzip.NewReader(file)
if err != nil {
t.Fatal(err)
}
dec := json.NewDecoder(gzr)
var msgs []*types.SignedMessage
baseNonces := make(map[address.Address]uint64)
readLoop:
for {
m := new(types.SignedMessage)
err := dec.Decode(m)
switch err {
case nil:
msgs = append(msgs, m)
nonce, ok := baseNonces[m.Message.From]
if !ok || m.Message.Nonce < nonce {
baseNonces[m.Message.From] = m.Message.Nonce
}
case io.EOF:
break readLoop
default:
t.Fatal(err)
}
}
actorMap := make(map[address.Address]address.Address)
actorWallets := make(map[address.Address]*wallet.Wallet)
for _, m := range msgs {
baseNonce := baseNonces[m.Message.From]
localActor, ok := actorMap[m.Message.From]
if !ok {
w, err := wallet.NewWallet(wallet.NewMemKeyStore())
if err != nil {
t.Fatal(err)
}
a, err := w.GenerateKey(crypto.SigTypeSecp256k1)
if err != nil {
t.Fatal(err)
}
actorMap[m.Message.From] = a
actorWallets[a] = w
localActor = a
}
w, ok := actorWallets[localActor]
if !ok {
t.Fatalf("failed to lookup wallet for actor %s", localActor)
}
m.Message.From = localActor
m.Message.Nonce -= baseNonce
sig, err := w.Sign(context.TODO(), localActor, m.Message.Cid().Bytes())
if err != nil {
t.Fatal(err)
}
m.Signature = *sig
}
mp, tma := makeTestMpool()
block := tma.nextBlockWithHeight(build.UpgradeBreezeHeight + 10)
ts := mock.TipSet(block)
tma.applyBlock(t, block)
for _, a := range actorMap {
tma.setBalance(a, 1000000)
}
tma.baseFee = types.NewInt(800_000_000)
sort.Slice(msgs, func(i, j int) bool {
return msgs[i].Message.Nonce < msgs[j].Message.Nonce
})
// add the messages
for _, m := range msgs {
mustAdd(t, mp, m)
}
// do message selection and check block packing
minGasLimit := int64(0.9 * float64(build.BlockGasLimit))
// greedy first
selected, err := mp.SelectMessages(ts, 1.0)
if err != nil {
t.Fatal(err)
}
gasLimit := int64(0)
for _, m := range selected {
gasLimit += m.Message.GasLimit
}
if gasLimit < minGasLimit {
t.Fatalf("failed to pack with tq=1.0; packed %d, minimum packing: %d", gasLimit, minGasLimit)
}
// high quality ticket
selected, err = mp.SelectMessages(ts, .8)
if err != nil {
t.Fatal(err)
}
gasLimit = int64(0)
for _, m := range selected {
gasLimit += m.Message.GasLimit
}
if gasLimit < minGasLimit {
t.Fatalf("failed to pack with tq=0.8; packed %d, minimum packing: %d", gasLimit, minGasLimit)
}
// mid quality ticket
selected, err = mp.SelectMessages(ts, .4)
if err != nil {
t.Fatal(err)
}
gasLimit = int64(0)
for _, m := range selected {
gasLimit += m.Message.GasLimit
}
if gasLimit < minGasLimit {
t.Fatalf("failed to pack with tq=0.4; packed %d, minimum packing: %d", gasLimit, minGasLimit)
}
// low quality ticket
selected, err = mp.SelectMessages(ts, .1)
if err != nil {
t.Fatal(err)
}
gasLimit = int64(0)
for _, m := range selected {
gasLimit += m.Message.GasLimit
}
if gasLimit < minGasLimit {
t.Fatalf("failed to pack with tq=0.1; packed %d, minimum packing: %d", gasLimit, minGasLimit)
}
// very low quality ticket
selected, err = mp.SelectMessages(ts, .01)
if err != nil {
t.Fatal(err)
}
gasLimit = int64(0)
for _, m := range selected {
gasLimit += m.Message.GasLimit
}
if gasLimit < minGasLimit {
t.Fatalf("failed to pack with tq=0.01; packed %d, minimum packing: %d", gasLimit, minGasLimit)
}
}
Binary file not shown.
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"context"
"encoding/json"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/ipfs/go-cid"
logging "github.com/ipfs/go-log/v2"
pubsub "github.com/libp2p/go-libp2p-pubsub"
+6 -4
View File
@@ -5,8 +5,8 @@ import (
"fmt"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/ipfs/go-cid"
"go.opencensus.io/trace"
"golang.org/x/xerrors"
@@ -29,6 +29,7 @@ func (sm *StateManager) CallRaw(ctx context.Context, msg *types.Message, bstate
Bstore: sm.cs.Blockstore(),
Syscalls: sm.cs.VMSys(),
CircSupplyCalc: sm.GetCirculatingSupply,
NtwkVersion: sm.GetNtwkVersion,
BaseFee: types.NewInt(0),
}
@@ -95,7 +96,7 @@ func (sm *StateManager) Call(ctx context.Context, msg *types.Message, ts *types.
state := ts.ParentState()
r := store.NewChainRand(sm.cs, ts.Cids(), ts.Height())
r := store.NewChainRand(sm.cs, ts.Cids())
return sm.CallRaw(ctx, msg, state, r, ts.Height())
}
@@ -113,7 +114,7 @@ func (sm *StateManager) CallWithGas(ctx context.Context, msg *types.Message, pri
return nil, xerrors.Errorf("computing tipset state: %w", err)
}
r := store.NewChainRand(sm.cs, ts.Cids(), ts.Height())
r := store.NewChainRand(sm.cs, ts.Cids())
if span.IsRecordingEvents() {
span.AddAttributes(
@@ -130,6 +131,7 @@ func (sm *StateManager) CallWithGas(ctx context.Context, msg *types.Message, pri
Bstore: sm.cs.Blockstore(),
Syscalls: sm.cs.VMSys(),
CircSupplyCalc: sm.GetCirculatingSupply,
NtwkVersion: sm.GetNtwkVersion,
BaseFee: ts.Blocks()[0].ParentBaseFee,
}
vmi, err := vm.NewVM(vmopt)
+304 -4
View File
@@ -3,16 +3,28 @@ package stmgr
import (
"context"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/util/adt"
cbor "github.com/ipfs/go-ipld-cbor"
"golang.org/x/xerrors"
)
var ForksAtHeight = map[abi.ChainEpoch]func(context.Context, *StateManager, types.StateTree) error{}
var ForksAtHeight = map[abi.ChainEpoch]func(context.Context, *StateManager, types.StateTree, *types.TipSet) error{
build.UpgradeBreezeHeight: UpgradeFaucetBurnRecovery,
}
func (sm *StateManager) handleStateForks(ctx context.Context, st types.StateTree, height abi.ChainEpoch) (err error) {
func (sm *StateManager) handleStateForks(ctx context.Context, st types.StateTree, height abi.ChainEpoch, ts *types.TipSet) (err error) {
f, ok := ForksAtHeight[height]
if ok {
err := f(ctx, sm, st)
err := f(ctx, sm, st, ts)
if err != nil {
return err
}
@@ -20,3 +32,291 @@ func (sm *StateManager) handleStateForks(ctx context.Context, st types.StateTree
return nil
}
type forEachTree interface {
ForEach(func(address.Address, *types.Actor) error) error
}
func doTransfer(tree types.StateTree, from, to address.Address, amt abi.TokenAmount) error {
fromAct, err := tree.GetActor(from)
if err != nil {
return xerrors.Errorf("failed to get 'from' actor for transfer: %w", err)
}
fromAct.Balance = types.BigSub(fromAct.Balance, amt)
if fromAct.Balance.Sign() < 0 {
return xerrors.Errorf("(sanity) deducted more funds from target account than it had (%s, %s)", from, types.FIL(amt))
}
if err := tree.SetActor(from, fromAct); err != nil {
return xerrors.Errorf("failed to persist from actor: %w", err)
}
toAct, err := tree.GetActor(to)
if err != nil {
return xerrors.Errorf("failed to get 'to' actor for transfer: %w", err)
}
toAct.Balance = types.BigAdd(toAct.Balance, amt)
if err := tree.SetActor(to, toAct); err != nil {
return xerrors.Errorf("failed to persist to actor: %w", err)
}
return nil
}
func UpgradeFaucetBurnRecovery(ctx context.Context, sm *StateManager, tree types.StateTree, ts *types.TipSet) error {
// Some initial parameters
FundsForMiners := types.FromFil(1_000_000)
LookbackEpoch := abi.ChainEpoch(32000)
AccountCap := types.FromFil(0)
BaseMinerBalance := types.FromFil(20)
DesiredReimbursementBalance := types.FromFil(5_000_000)
isSystemAccount := func(addr address.Address) (bool, error) {
id, err := address.IDFromAddress(addr)
if err != nil {
return false, xerrors.Errorf("id address: %w", err)
}
if id < 1000 {
return true, nil
}
return false, nil
}
minerFundsAlloc := func(pow, tpow abi.StoragePower) abi.TokenAmount {
return types.BigDiv(types.BigMul(pow, FundsForMiners), tpow)
}
// Grab lookback state for account checks
lbts, err := sm.ChainStore().GetTipsetByHeight(ctx, LookbackEpoch, ts, false)
if err != nil {
return xerrors.Errorf("failed to get tipset at lookback height: %w", err)
}
var lbtree *state.StateTree
if err = sm.WithStateTree(lbts.ParentState(), func(state *state.StateTree) error {
lbtree = state
return nil
}); err != nil {
return xerrors.Errorf("loading state tree failed: %w", err)
}
ReserveAddress, err := address.NewFromString("t090")
if err != nil {
return xerrors.Errorf("failed to parse reserve address: %w", err)
}
fetree, ok := tree.(forEachTree)
if !ok {
return xerrors.Errorf("fork transition state tree doesnt support ForEach (%T)", tree)
}
type transfer struct {
From address.Address
To address.Address
Amt abi.TokenAmount
}
var transfers []transfer
// Take all excess funds away, put them into the reserve account
err = fetree.ForEach(func(addr address.Address, act *types.Actor) error {
switch act.Code {
case builtin.AccountActorCodeID, builtin.MultisigActorCodeID, builtin.PaymentChannelActorCodeID:
sysAcc, err := isSystemAccount(addr)
if err != nil {
return xerrors.Errorf("checking system account: %w", err)
}
if !sysAcc {
transfers = append(transfers, transfer{
From: addr,
To: ReserveAddress,
Amt: act.Balance,
})
}
case builtin.StorageMinerActorCodeID:
var st miner.State
if err := sm.WithActorState(ctx, &st)(act); err != nil {
return xerrors.Errorf("failed to load miner state: %w", err)
}
var available abi.TokenAmount
{
defer func() {
if err := recover(); err != nil {
log.Warnf("Get available balance failed (%s, %s, %s): %s", addr, act.Head, act.Balance, err)
}
available = abi.NewTokenAmount(0)
}()
// this panics if the miner doesnt have enough funds to cover their locked pledge
available = st.GetAvailableBalance(act.Balance)
}
transfers = append(transfers, transfer{
From: addr,
To: ReserveAddress,
Amt: available,
})
}
return nil
})
if err != nil {
return xerrors.Errorf("foreach over state tree failed: %w", err)
}
// Execute transfers from previous step
for _, t := range transfers {
if err := doTransfer(tree, t.From, t.To, t.Amt); err != nil {
return xerrors.Errorf("transfer %s %s->%s failed: %w", t.Amt, t.From, t.To, err)
}
}
// pull up power table to give miners back some funds proportional to their power
var ps power.State
powAct, err := tree.GetActor(builtin.StoragePowerActorAddr)
if err != nil {
return xerrors.Errorf("failed to load power actor: %w", err)
}
cst := cbor.NewCborStore(sm.ChainStore().Blockstore())
if err := cst.Get(ctx, powAct.Head, &ps); err != nil {
return xerrors.Errorf("failed to get power actor state: %w", err)
}
totalPower := ps.TotalBytesCommitted
var transfersBack []transfer
// Now, we return some funds to places where they are needed
err = fetree.ForEach(func(addr address.Address, act *types.Actor) error {
lbact, err := lbtree.GetActor(addr)
if err != nil {
if !xerrors.Is(err, types.ErrActorNotFound) {
return xerrors.Errorf("failed to get actor in lookback state")
}
}
prevBalance := abi.NewTokenAmount(0)
if lbact != nil {
prevBalance = lbact.Balance
}
switch act.Code {
case builtin.AccountActorCodeID, builtin.MultisigActorCodeID, builtin.PaymentChannelActorCodeID:
nbalance := big.Min(prevBalance, AccountCap)
if nbalance.Sign() != 0 {
transfersBack = append(transfersBack, transfer{
From: ReserveAddress,
To: addr,
Amt: nbalance,
})
}
case builtin.StorageMinerActorCodeID:
var st miner.State
if err := sm.WithActorState(ctx, &st)(act); err != nil {
return xerrors.Errorf("failed to load miner state: %w", err)
}
var minfo miner.MinerInfo
if err := cst.Get(ctx, st.Info, &minfo); err != nil {
return xerrors.Errorf("failed to get miner info: %w", err)
}
sectorsArr, err := adt.AsArray(sm.ChainStore().Store(ctx), st.Sectors)
if err != nil {
return xerrors.Errorf("failed to load sectors array: %w", err)
}
slen := sectorsArr.Length()
power := types.BigMul(types.NewInt(slen), types.NewInt(uint64(minfo.SectorSize)))
mfunds := minerFundsAlloc(power, totalPower)
transfersBack = append(transfersBack, transfer{
From: ReserveAddress,
To: minfo.Worker,
Amt: mfunds,
})
// Now make sure to give each miner who had power at the lookback some FIL
lbact, err := lbtree.GetActor(addr)
if err == nil {
var lbst miner.State
if err := sm.WithActorState(ctx, &lbst)(lbact); err != nil {
return xerrors.Errorf("failed to load miner state: %w", err)
}
lbsectors, err := adt.AsArray(sm.ChainStore().Store(ctx), lbst.Sectors)
if err != nil {
return xerrors.Errorf("failed to load lb sectors array: %w", err)
}
if lbsectors.Length() > 0 {
transfersBack = append(transfersBack, transfer{
From: ReserveAddress,
To: minfo.Worker,
Amt: BaseMinerBalance,
})
}
} else {
log.Warnf("failed to get miner in lookback state: %s", err)
}
}
return nil
})
if err != nil {
return xerrors.Errorf("foreach over state tree failed: %w", err)
}
for _, t := range transfersBack {
if err := doTransfer(tree, t.From, t.To, t.Amt); err != nil {
return xerrors.Errorf("transfer %s %s->%s failed: %w", t.Amt, t.From, t.To, err)
}
}
// transfer all burnt funds back to the reserve account
burntAct, err := tree.GetActor(builtin.BurntFundsActorAddr)
if err != nil {
return xerrors.Errorf("failed to load burnt funds actor: %w", err)
}
if err := doTransfer(tree, builtin.BurntFundsActorAddr, ReserveAddress, burntAct.Balance); err != nil {
return xerrors.Errorf("failed to unburn funds: %w", err)
}
// Top up the reimbursement service
reimbAddr, err := address.NewFromString("t0111")
if err != nil {
return xerrors.Errorf("failed to parse reimbursement service address")
}
reimb, err := tree.GetActor(reimbAddr)
if err != nil {
return xerrors.Errorf("failed to load reimbursement account actor: %w", err)
}
difference := types.BigSub(DesiredReimbursementBalance, reimb.Balance)
if err := doTransfer(tree, ReserveAddress, reimbAddr, difference); err != nil {
return xerrors.Errorf("failed to top up reimbursement account: %w", err)
}
// Now, a final sanity check to make sure the balances all check out
total := abi.NewTokenAmount(0)
err = fetree.ForEach(func(addr address.Address, act *types.Actor) error {
total = types.BigAdd(total, act.Balance)
return nil
})
if err != nil {
return xerrors.Errorf("checking final state balance failed: %w", err)
}
exp := types.FromFil(build.FilBase)
if !exp.Equals(total) {
return xerrors.Errorf("resultant state tree account balance was not correct: %s", total)
}
return nil
}
+7 -8
View File
@@ -7,15 +7,14 @@ import (
"testing"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
init_ "github.com/filecoin-project/specs-actors/actors/builtin/init"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/runtime"
"github.com/filecoin-project/specs-actors/actors/util/adt"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/chain/actors"
@@ -73,15 +72,15 @@ func (ta *testActor) Exports() []interface{} {
}
}
func (ta *testActor) Constructor(rt runtime.Runtime, params *adt.EmptyValue) *adt.EmptyValue {
func (ta *testActor) Constructor(rt runtime.Runtime, params *abi.EmptyValue) *abi.EmptyValue {
rt.ValidateImmediateCallerAcceptAny()
rt.State().Create(&testActorState{11})
fmt.Println("NEW ACTOR ADDRESS IS: ", rt.Message().Receiver())
return adt.Empty
return abi.Empty
}
func (ta *testActor) TestMethod(rt runtime.Runtime, params *adt.EmptyValue) *adt.EmptyValue {
func (ta *testActor) TestMethod(rt runtime.Runtime, params *abi.EmptyValue) *abi.EmptyValue {
rt.ValidateImmediateCallerAcceptAny()
var st testActorState
rt.State().Readonly(&st)
@@ -96,7 +95,7 @@ func (ta *testActor) TestMethod(rt runtime.Runtime, params *adt.EmptyValue) *adt
}
}
return adt.Empty
return abi.Empty
}
func TestForkHeightTriggers(t *testing.T) {
@@ -119,7 +118,7 @@ func TestForkHeightTriggers(t *testing.T) {
t.Fatal(err)
}
stmgr.ForksAtHeight[testForkHeight] = func(ctx context.Context, sm *StateManager, st types.StateTree) error {
stmgr.ForksAtHeight[testForkHeight] = func(ctx context.Context, sm *StateManager, st types.StateTree, ts *types.TipSet) error {
cst := cbor.NewCborStore(sm.ChainStore().Blockstore())
act, err := st.GetActor(taddr)
+25 -6
View File
@@ -5,6 +5,8 @@ import (
"fmt"
"sync"
"github.com/filecoin-project/go-state-types/network"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/multisig"
@@ -18,8 +20,8 @@ import (
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/vm"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/market"
"github.com/filecoin-project/specs-actors/actors/builtin/reward"
@@ -145,7 +147,7 @@ func (sm *StateManager) ExecutionTrace(ctx context.Context, ts *types.TipSet) (c
type ExecCallback func(cid.Cid, *types.Message, *vm.ApplyRet) error
func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEpoch, pstate cid.Cid, bms []store.BlockMessages, epoch abi.ChainEpoch, r vm.Rand, cb ExecCallback, baseFee abi.TokenAmount) (cid.Cid, cid.Cid, error) {
func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEpoch, pstate cid.Cid, bms []store.BlockMessages, epoch abi.ChainEpoch, r vm.Rand, cb ExecCallback, baseFee abi.TokenAmount, ts *types.TipSet) (cid.Cid, cid.Cid, error) {
vmopt := &vm.VMOpts{
StateBase: pstate,
@@ -154,6 +156,7 @@ func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEp
Bstore: sm.cs.Blockstore(),
Syscalls: sm.cs.VMSys(),
CircSupplyCalc: sm.GetCirculatingSupply,
NtwkVersion: sm.GetNtwkVersion,
BaseFee: baseFee,
}
@@ -198,7 +201,7 @@ func (sm *StateManager) ApplyBlocks(ctx context.Context, parentEpoch abi.ChainEp
for i := parentEpoch; i < epoch; i++ {
// handle state forks
err = sm.handleStateForks(ctx, vmi.StateTree(), i)
err = sm.handleStateForks(ctx, vmi.StateTree(), i, ts)
if err != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("error handling state forks: %w", err)
}
@@ -338,7 +341,7 @@ func (sm *StateManager) computeTipSetState(ctx context.Context, ts *types.TipSet
cids[i] = v.Cid()
}
r := store.NewChainRand(sm.cs, cids, blks[0].Height)
r := store.NewChainRand(sm.cs, cids)
blkmsgs, err := sm.cs.BlockMsgsForTipset(ts)
if err != nil {
@@ -347,7 +350,7 @@ func (sm *StateManager) computeTipSetState(ctx context.Context, ts *types.TipSet
baseFee := blks[0].ParentBaseFee
return sm.ApplyBlocks(ctx, parentEpoch, pstate, blkmsgs, blks[0].Height, r, cb, baseFee)
return sm.ApplyBlocks(ctx, parentEpoch, pstate, blkmsgs, blks[0].Height, r, cb, baseFee, ts)
}
func (sm *StateManager) parentState(ts *types.TipSet) cid.Cid {
@@ -1120,3 +1123,19 @@ func (sm *StateManager) GetCirculatingSupply(ctx context.Context, height abi.Cha
return csi.FilCirculating, nil
}
func (sm *StateManager) GetNtwkVersion(ctx context.Context, height abi.ChainEpoch) network.Version {
if build.UseNewestNetwork() {
return build.NewestNetworkVersion
}
if height <= build.UpgradeBreezeHeight {
return network.Version0
}
if height <= build.UpgradeSmokeHeight {
return network.Version1
}
return build.NewestNetworkVersion
}
+82 -32
View File
@@ -3,8 +3,14 @@ package stmgr
import (
"bytes"
"context"
"fmt"
"os"
"reflect"
"runtime"
"strings"
saruntime "github.com/filecoin-project/specs-actors/actors/runtime"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
cid "github.com/ipfs/go-cid"
cbor "github.com/ipfs/go-ipld-cbor"
@@ -13,8 +19,9 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/account"
"github.com/filecoin-project/specs-actors/actors/builtin/cron"
@@ -26,7 +33,6 @@ import (
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/reward"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/util/adt"
"github.com/filecoin-project/lotus/api"
@@ -157,13 +163,13 @@ func MinerSectorInfo(ctx context.Context, sm *StateManager, maddr address.Addres
return nil, err
}
if !ok {
return nil, xerrors.New("sector not found")
return nil, nil
}
return sectorInfo, nil
}
func GetMinerSectorSet(ctx context.Context, sm *StateManager, ts *types.TipSet, maddr address.Address, filter *abi.BitField, filterOut bool) ([]*api.ChainSectorInfo, error) {
func GetMinerSectorSet(ctx context.Context, sm *StateManager, ts *types.TipSet, maddr address.Address, filter *bitfield.BitField, filterOut bool) ([]*api.ChainSectorInfo, error) {
var mas miner.State
_, err := sm.LoadActorState(ctx, maddr, &mas, ts)
if err != nil {
@@ -173,8 +179,8 @@ func GetMinerSectorSet(ctx context.Context, sm *StateManager, ts *types.TipSet,
return LoadSectorsFromSet(ctx, sm.ChainStore().Blockstore(), mas.Sectors, filter, filterOut)
}
func GetSectorsForWinningPoSt(ctx context.Context, pv ffiwrapper.Verifier, sm *StateManager, st cid.Cid, maddr address.Address, rand abi.PoStRandomness) ([]abi.SectorInfo, error) {
var partsProving []abi.BitField
func GetSectorsForWinningPoSt(ctx context.Context, pv ffiwrapper.Verifier, sm *StateManager, st cid.Cid, maddr address.Address, rand abi.PoStRandomness) ([]proof.SectorInfo, error) {
var partsProving []bitfield.BitField
var mas *miner.State
var info *miner.MinerInfo
@@ -261,7 +267,7 @@ func GetSectorsForWinningPoSt(ctx context.Context, pv ffiwrapper.Verifier, sm *S
return nil, xerrors.Errorf("failed to load sectors amt: %w", err)
}
out := make([]abi.SectorInfo, len(ids))
out := make([]proof.SectorInfo, len(ids))
for i, n := range ids {
sid := sectors[n]
@@ -272,7 +278,7 @@ func GetSectorsForWinningPoSt(ctx context.Context, pv ffiwrapper.Verifier, sm *S
return nil, xerrors.Errorf("failed to find sector %d", sid)
}
out[i] = abi.SectorInfo{
out[i] = proof.SectorInfo{
SealProof: spt,
SectorNumber: sinfo.SectorNumber,
SealedCID: sinfo.SealedCID,
@@ -387,7 +393,7 @@ func ListMinerActors(ctx context.Context, sm *StateManager, ts *types.TipSet) ([
return miners, nil
}
func LoadSectorsFromSet(ctx context.Context, bs blockstore.Blockstore, ssc cid.Cid, filter *abi.BitField, filterOut bool) ([]*api.ChainSectorInfo, error) {
func LoadSectorsFromSet(ctx context.Context, bs blockstore.Blockstore, ssc cid.Cid, filter *bitfield.BitField, filterOut bool) ([]*api.ChainSectorInfo, error) {
a, err := adt.AsArray(store.ActorStore(ctx, bs), ssc)
if err != nil {
return nil, err
@@ -432,7 +438,7 @@ func ComputeState(ctx context.Context, sm *StateManager, height abi.ChainEpoch,
return cid.Undef, nil, err
}
r := store.NewChainRand(sm.cs, ts.Cids(), height)
r := store.NewChainRand(sm.cs, ts.Cids())
vmopt := &vm.VMOpts{
StateBase: base,
Epoch: height,
@@ -440,6 +446,7 @@ func ComputeState(ctx context.Context, sm *StateManager, height abi.ChainEpoch,
Bstore: sm.cs.Blockstore(),
Syscalls: sm.cs.VMSys(),
CircSupplyCalc: sm.GetCirculatingSupply,
NtwkVersion: sm.GetNtwkVersion,
BaseFee: ts.Blocks()[0].ParentBaseFee,
}
vmi, err := vm.NewVM(vmopt)
@@ -449,7 +456,7 @@ func ComputeState(ctx context.Context, sm *StateManager, height abi.ChainEpoch,
for i := ts.Height(); i < height; i++ {
// handle state forks
err = sm.handleStateForks(ctx, vmi.StateTree(), i)
err = sm.handleStateForks(ctx, vmi.StateTree(), i, ts)
if err != nil {
return cid.Undef, nil, xerrors.Errorf("error handling state forks: %w", err)
}
@@ -495,7 +502,7 @@ func GetLookbackTipSetForRound(ctx context.Context, sm *StateManager, ts *types.
return lbts, nil
}
func MinerGetBaseInfo(ctx context.Context, sm *StateManager, bcn beacon.RandomBeacon, tsk types.TipSetKey, round abi.ChainEpoch, maddr address.Address, pv ffiwrapper.Verifier) (*api.MiningBaseInfo, error) {
func MinerGetBaseInfo(ctx context.Context, sm *StateManager, bcs beacon.Schedule, tsk types.TipSetKey, round abi.ChainEpoch, maddr address.Address, pv ffiwrapper.Verifier) (*api.MiningBaseInfo, error) {
ts, err := sm.ChainStore().LoadTipSet(tsk)
if err != nil {
return nil, xerrors.Errorf("failed to load tipset for mining base: %w", err)
@@ -510,7 +517,7 @@ func MinerGetBaseInfo(ctx context.Context, sm *StateManager, bcn beacon.RandomBe
prev = &types.BeaconEntry{}
}
entries, err := beacon.BeaconEntriesForBlock(ctx, bcn, round, *prev)
entries, err := beacon.BeaconEntriesForBlock(ctx, bcs, round, ts.Height(), *prev)
if err != nil {
return nil, err
}
@@ -586,14 +593,14 @@ func MinerGetBaseInfo(ctx context.Context, sm *StateManager, bcn beacon.RandomBe
}, nil
}
type methodMeta struct {
type MethodMeta struct {
Name string
Params reflect.Type
Ret reflect.Type
}
var MethodsMap = map[cid.Cid][]methodMeta{}
var MethodsMap = map[cid.Cid]map[abi.MethodNum]MethodMeta{}
func init() {
cidToMethods := map[cid.Cid][2]interface{}{
@@ -611,25 +618,65 @@ func init() {
}
for c, m := range cidToMethods {
rt := reflect.TypeOf(m[0])
nf := rt.NumField()
exports := m[1].(saruntime.Invokee).Exports()
methods := make(map[abi.MethodNum]MethodMeta, len(exports))
MethodsMap[c] = append(MethodsMap[c], methodMeta{
// Explicitly add send, it's special.
methods[builtin.MethodSend] = MethodMeta{
Name: "Send",
Params: reflect.TypeOf(new(adt.EmptyValue)),
Ret: reflect.TypeOf(new(adt.EmptyValue)),
})
exports := m[1].(abi.Invokee).Exports()
for i := 0; i < nf; i++ {
export := reflect.TypeOf(exports[i+1])
MethodsMap[c] = append(MethodsMap[c], methodMeta{
Name: rt.Field(i).Name,
Params: export.In(1),
Ret: export.Out(0),
})
Params: reflect.TypeOf(new(abi.EmptyValue)),
Ret: reflect.TypeOf(new(abi.EmptyValue)),
}
// Learn method names from the builtin.Methods* structs.
rv := reflect.ValueOf(m[0])
rt := rv.Type()
nf := rt.NumField()
methodToName := make([]string, len(exports))
for i := 0; i < nf; i++ {
name := rt.Field(i).Name
number := rv.Field(i).Interface().(abi.MethodNum)
methodToName[number] = name
}
// Iterate over exported methods. Some of these _may_ be nil and
// must be skipped.
for number, export := range exports {
if export == nil {
continue
}
ev := reflect.ValueOf(export)
et := ev.Type()
// Make sure the method name is correct.
// This is just a nice sanity check.
fnName := runtime.FuncForPC(ev.Pointer()).Name()
fnName = strings.TrimSuffix(fnName[strings.LastIndexByte(fnName, '.')+1:], "-fm")
mName := methodToName[number]
if mName != fnName {
panic(fmt.Sprintf(
"actor method name is %s but exported method name is %s",
fnName, mName,
))
}
switch abi.MethodNum(number) {
case builtin.MethodSend:
panic("method 0 is reserved for Send")
case builtin.MethodConstructor:
if fnName != "Constructor" {
panic("method 1 is reserved for Constructor")
}
}
methods[abi.MethodNum(number)] = MethodMeta{
Name: fnName,
Params: et.In(1),
Ret: et.Out(0),
}
}
MethodsMap[c] = methods
}
}
@@ -639,7 +686,10 @@ func GetReturnType(ctx context.Context, sm *StateManager, to address.Address, me
return nil, xerrors.Errorf("getting actor: %w", err)
}
m := MethodsMap[act.Code][method]
m, found := MethodsMap[act.Code][method]
if !found {
return nil, fmt.Errorf("unknown method %d for actor %s", method, act.Code)
}
return reflect.New(m.Ret.Elem()).Interface().(cbg.CBORUnmarshaler), nil
}
+18 -8
View File
@@ -3,22 +3,28 @@ package store
import (
"context"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/ipfs/go-cid"
"golang.org/x/xerrors"
)
func computeNextBaseFee(baseFee types.BigInt, gasLimitUsed int64, noOfBlocks int) types.BigInt {
// deta := 1/PackingEfficiency * gasLimitUsed/noOfBlocks - build.BlockGasTarget
// change := baseFee * deta / BlockGasTarget / BaseFeeMaxChangeDenom
func ComputeNextBaseFee(baseFee types.BigInt, gasLimitUsed int64, noOfBlocks int, epoch abi.ChainEpoch) types.BigInt {
// deta := gasLimitUsed/noOfBlocks - build.BlockGasTarget
// change := baseFee * deta / BlockGasTarget
// nextBaseFee = baseFee + change
// nextBaseFee = max(nextBaseFee, build.MinimumBaseFee)
delta := build.PackingEfficiencyDenom * gasLimitUsed / (int64(noOfBlocks) * build.PackingEfficiencyNum)
delta -= build.BlockGasTarget
var delta int64
if epoch > build.UpgradeSmokeHeight {
delta = gasLimitUsed / int64(noOfBlocks)
delta -= build.BlockGasTarget
} else {
delta = build.PackingEfficiencyDenom * gasLimitUsed / (int64(noOfBlocks) * build.PackingEfficiencyNum)
delta -= build.BlockGasTarget
}
// cap change at 12.5% (BaseFeeMaxChangeDenom) by capping delta
if delta > build.BlockGasTarget {
@@ -40,6 +46,10 @@ func computeNextBaseFee(baseFee types.BigInt, gasLimitUsed int64, noOfBlocks int
}
func (cs *ChainStore) ComputeBaseFee(ctx context.Context, ts *types.TipSet) (abi.TokenAmount, error) {
if ts.Height() > build.UpgradeBreezeHeight && ts.Height() < build.UpgradeBreezeHeight+build.BreezeGasTampingDuration {
return abi.NewTokenAmount(100), nil
}
zero := abi.NewTokenAmount(0)
// totalLimit is sum of GasLimits of unique messages in a tipset
@@ -69,5 +79,5 @@ func (cs *ChainStore) ComputeBaseFee(ctx context.Context, ts *types.TipSet) (abi
}
parentBaseFee := ts.Blocks()[0].ParentBaseFee
return computeNextBaseFee(parentBaseFee, totalLimit, len(ts.Blocks())), nil
return ComputeNextBaseFee(parentBaseFee, totalLimit, len(ts.Blocks()), ts.Height()), nil
}
+1 -1
View File
@@ -27,7 +27,7 @@ func TestBaseFee(t *testing.T) {
for _, test := range tests {
test := test
t.Run(fmt.Sprintf("%v", test), func(t *testing.T) {
output := computeNextBaseFee(types.NewInt(test.basefee), test.limitUsed, test.noOfBlocks)
output := ComputeNextBaseFee(types.NewInt(test.basefee), test.limitUsed, test.noOfBlocks, 0)
assert.Equal(t, fmt.Sprintf("%d", test.output), output.String())
})
}
+1 -1
View File
@@ -5,8 +5,8 @@ import (
"os"
"strconv"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/specs-actors/actors/abi"
lru "github.com/hashicorp/golang-lru"
"golang.org/x/xerrors"
)
+1 -1
View File
@@ -5,11 +5,11 @@ import (
"context"
"testing"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/chain/gen"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types/mock"
"github.com/filecoin-project/lotus/lib/blockstore"
"github.com/filecoin-project/specs-actors/actors/abi"
datastore "github.com/ipfs/go-datastore"
syncds "github.com/ipfs/go-datastore/sync"
"github.com/stretchr/testify/assert"
+28 -15
View File
@@ -10,11 +10,11 @@ import (
"strconv"
"sync"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/minio/blake2b-simd"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/specs-actors/actors/util/adt"
"github.com/filecoin-project/lotus/api"
@@ -49,6 +49,7 @@ var chainHeadKey = dstore.NewKey("head")
var blockValidationCacheKeyPrefix = dstore.NewKey("blockValidation")
var DefaultTipSetCacheSize = 8192
var DefaultMsgMetaCacheSize = 2048
func init() {
if s := os.Getenv("LOTUS_CHAIN_TIPSET_CACHE"); s != "" {
@@ -58,6 +59,14 @@ func init() {
}
DefaultTipSetCacheSize = tscs
}
if s := os.Getenv("LOTUS_CHAIN_MSGMETA_CACHE"); s != "" {
mmcs, err := strconv.Atoi(s)
if err != nil {
log.Errorf("failed to parse 'LOTUS_CHAIN_MSGMETA_CACHE' env var: %s", err)
}
DefaultMsgMetaCacheSize = mmcs
}
}
// ReorgNotifee represents a callback that gets called upon reorgs.
@@ -97,7 +106,7 @@ type ChainStore struct {
}
func NewChainStore(bs bstore.Blockstore, ds dstore.Batching, vmcalls vm.SyscallBuilder) *ChainStore {
c, _ := lru.NewARC(2048)
c, _ := lru.NewARC(DefaultMsgMetaCacheSize)
tsc, _ := lru.NewARC(DefaultTipSetCacheSize)
cs := &ChainStore{
bs: bs,
@@ -462,7 +471,7 @@ func (cs *ChainStore) IsAncestorOf(a, b *types.TipSet) (bool, error) {
cur := b
for !a.Equals(cur) && cur.Height() > a.Height() {
next, err := cs.LoadTipSet(b.Parents())
next, err := cs.LoadTipSet(cur.Parents())
if err != nil {
return false, err
}
@@ -834,7 +843,7 @@ type mmCids struct {
secpk []cid.Cid
}
func (cs *ChainStore) readMsgMetaCids(mmc cid.Cid) ([]cid.Cid, []cid.Cid, error) {
func (cs *ChainStore) ReadMsgMetaCids(mmc cid.Cid) ([]cid.Cid, []cid.Cid, error) {
o, ok := cs.mmCache.Get(mmc)
if ok {
mmcids := o.(*mmCids)
@@ -890,7 +899,7 @@ func (cs *ChainStore) GetPath(ctx context.Context, from types.TipSetKey, to type
}
func (cs *ChainStore) MessagesForBlock(b *types.BlockHeader) ([]*types.Message, []*types.SignedMessage, error) {
blscids, secpkcids, err := cs.readMsgMetaCids(b.Messages)
blscids, secpkcids, err := cs.ReadMsgMetaCids(b.Messages)
if err != nil {
return nil, nil, err
}
@@ -1114,7 +1123,7 @@ func (cs *ChainStore) GetTipsetByHeight(ctx context.Context, h abi.ChainEpoch, t
return cs.LoadTipSet(lbts.Parents())
}
func recurseLinks(bs bstore.Blockstore, root cid.Cid, in []cid.Cid) ([]cid.Cid, error) {
func recurseLinks(bs bstore.Blockstore, walked *cid.Set, root cid.Cid, in []cid.Cid) ([]cid.Cid, error) {
if root.Prefix().Codec != cid.DagCBOR {
return in, nil
}
@@ -1131,9 +1140,14 @@ func recurseLinks(bs bstore.Blockstore, root cid.Cid, in []cid.Cid) ([]cid.Cid,
return
}
// traversed this already...
if !walked.Visit(c) {
return
}
in = append(in, c)
var err error
in, err = recurseLinks(bs, c, in)
in, err = recurseLinks(bs, walked, c, in)
if err != nil {
rerr = err
}
@@ -1145,12 +1159,13 @@ func recurseLinks(bs bstore.Blockstore, root cid.Cid, in []cid.Cid) ([]cid.Cid,
return in, rerr
}
func (cs *ChainStore) Export(ctx context.Context, ts *types.TipSet, w io.Writer) error {
func (cs *ChainStore) Export(ctx context.Context, ts *types.TipSet, inclRecentRoots abi.ChainEpoch, w io.Writer) error {
if ts == nil {
ts = cs.GetHeaviestTipSet()
}
seen := cid.NewSet()
walked := cid.NewSet()
h := &car.CarHeader{
Roots: ts.Cids(),
@@ -1182,7 +1197,7 @@ func (cs *ChainStore) Export(ctx context.Context, ts *types.TipSet, w io.Writer)
return xerrors.Errorf("unmarshaling block header (cid=%s): %w", blk, err)
}
cids, err := recurseLinks(cs.bs, b.Messages, []cid.Cid{b.Messages})
cids, err := recurseLinks(cs.bs, walked, b.Messages, []cid.Cid{b.Messages})
if err != nil {
return xerrors.Errorf("recursing messages failed: %w", err)
}
@@ -1198,8 +1213,8 @@ func (cs *ChainStore) Export(ctx context.Context, ts *types.TipSet, w io.Writer)
out := cids
if b.Height == 0 {
cids, err := recurseLinks(cs.bs, b.ParentStateRoot, []cid.Cid{b.ParentStateRoot})
if b.Height == 0 || b.Height > ts.Height()-inclRecentRoots {
cids, err := recurseLinks(cs.bs, walked, b.ParentStateRoot, []cid.Cid{b.ParentStateRoot})
if err != nil {
return xerrors.Errorf("recursing genesis state failed: %w", err)
}
@@ -1282,14 +1297,12 @@ func (cs *ChainStore) GetLatestBeaconEntry(ts *types.TipSet) (*types.BeaconEntry
type chainRand struct {
cs *ChainStore
blks []cid.Cid
bh abi.ChainEpoch
}
func NewChainRand(cs *ChainStore, blks []cid.Cid, bheight abi.ChainEpoch) vm.Rand {
func NewChainRand(cs *ChainStore, blks []cid.Cid) vm.Rand {
return &chainRand{
cs: cs,
blks: blks,
bh: bheight,
}
}
+4 -4
View File
@@ -7,12 +7,12 @@ import (
datastore "github.com/ipfs/go-datastore"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/lotus/chain/gen"
"github.com/filecoin-project/lotus/chain/store"
@@ -96,7 +96,7 @@ func TestChainExportImport(t *testing.T) {
}
buf := new(bytes.Buffer)
if err := cg.ChainStore().Export(context.TODO(), last, buf); err != nil {
if err := cg.ChainStore().Export(context.TODO(), last, 0, buf); err != nil {
t.Fatal(err)
}
+2 -2
View File
@@ -4,10 +4,10 @@ import (
"context"
"math/big"
big2 "github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/lotus/chain/types"
big2 "github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
cbor "github.com/ipfs/go-ipld-cbor"
@@ -41,7 +41,7 @@ func (cs *ChainStore) Weight(ctx context.Context, ts *types.TipSet) (types.BigIn
var st power.State
if err := cst.Get(ctx, act.Head, &st); err != nil {
return types.NewInt(0), xerrors.Errorf("get power actor head: %w", err)
return types.NewInt(0), xerrors.Errorf("get power actor head (%s, height=%d): %w", act.Head, ts.Height(), err)
}
tpow = st.TotalQualityAdjPower // TODO: REVIEW: Is this correct?
}
+44 -25
View File
@@ -3,6 +3,7 @@ package sub
import (
"bytes"
"context"
"errors"
"fmt"
"sync"
"time"
@@ -40,6 +41,9 @@ import (
var log = logging.Logger("sub")
var ErrSoftFailure = errors.New("soft validation failure")
var ErrInsufficientPower = errors.New("incoming block's miner does not have minimum power")
func HandleIncomingBlocks(ctx context.Context, bsub *pubsub.Subscription, s *chain.Syncer, bserv bserv.BlockService, cmgr connmgr.ConnManager) {
for {
msg, err := bsub.Next(ctx)
@@ -258,16 +262,15 @@ func (bv *BlockValidator) Validate(ctx context.Context, pid peer.ID, msg *pubsub
stats.Record(ctx, metrics.BlockReceived.M(1))
recordFailure := func(what string) {
ctx, _ = tag.New(ctx, tag.Insert(metrics.FailureType, what))
stats.Record(ctx, metrics.BlockValidationFailure.M(1))
recordFailureFlagPeer := func(what string) {
recordFailure(ctx, metrics.BlockValidationFailure, what)
bv.flagPeer(pid)
}
blk, what, err := bv.decodeAndCheckBlock(msg)
if err != nil {
log.Error("got invalid block over pubsub: ", err)
recordFailure(what)
recordFailureFlagPeer(what)
return pubsub.ValidationReject
}
@@ -275,7 +278,7 @@ func (bv *BlockValidator) Validate(ctx context.Context, pid peer.ID, msg *pubsub
err = bv.validateMsgMeta(ctx, blk)
if err != nil {
log.Warnf("error validating message metadata: %s", err)
recordFailure("invalid_block_meta")
recordFailureFlagPeer("invalid_block_meta")
return pubsub.ValidationReject
}
@@ -288,11 +291,12 @@ func (bv *BlockValidator) Validate(ctx context.Context, pid peer.ID, msg *pubsub
// if we are synced and the miner is unknown, then the block is rejcected.
key, err := bv.checkPowerAndGetWorkerKey(ctx, blk.Header)
if err != nil {
if bv.isChainNearSynced() {
if err != ErrSoftFailure && bv.isChainNearSynced() {
log.Warnf("received block from unknown miner or miner that doesn't meet min power over pubsub; rejecting message")
recordFailure("unknown_miner")
recordFailureFlagPeer("unknown_miner")
return pubsub.ValidationReject
}
log.Warnf("cannot validate block message; unknown miner or miner that doesn't meet min power in unsynced chain")
return pubsub.ValidationIgnore
}
@@ -300,13 +304,13 @@ func (bv *BlockValidator) Validate(ctx context.Context, pid peer.ID, msg *pubsub
err = sigs.CheckBlockSignature(ctx, blk.Header, key)
if err != nil {
log.Errorf("block signature verification failed: %s", err)
recordFailure("signature_verification_failed")
recordFailureFlagPeer("signature_verification_failed")
return pubsub.ValidationReject
}
if blk.Header.ElectionProof.WinCount < 1 {
log.Errorf("block is not claiming to be winning")
recordFailure("not_winning")
recordFailureFlagPeer("not_winning")
return pubsub.ValidationReject
}
@@ -365,9 +369,8 @@ func (bv *BlockValidator) decodeAndCheckBlock(msg *pubsub.Message) (*types.Block
func (bv *BlockValidator) isChainNearSynced() bool {
ts := bv.chain.GetHeaviestTipSet()
timestamp := ts.MinTimestamp()
now := build.Clock.Now().UnixNano()
cutoff := uint64(now) - uint64(6*time.Hour)
return timestamp > cutoff
timestampTime := time.Unix(int64(timestamp), 0)
return build.Clock.Since(timestampTime) < 6*time.Hour
}
func (bv *BlockValidator) validateMsgMeta(ctx context.Context, msg *types.BlockMsg) error {
@@ -473,19 +476,19 @@ func (bv *BlockValidator) checkPowerAndGetWorkerKey(ctx context.Context, bh *typ
baseTs := bv.chain.GetHeaviestTipSet()
lbts, err := stmgr.GetLookbackTipSetForRound(ctx, bv.stmgr, baseTs, bh.Height)
if err != nil {
log.Warnf("failed to load lookback tipset for incoming block")
return address.Undef, err
log.Warnf("failed to load lookback tipset for incoming block: %s", err)
return address.Undef, ErrSoftFailure
}
hmp, err := stmgr.MinerHasMinPower(ctx, bv.stmgr, bh.Miner, lbts)
if err != nil {
log.Warnf("failed to determine if incoming block's miner has minimum power")
return address.Undef, err
log.Warnf("failed to determine if incoming block's miner has minimum power: %s", err)
return address.Undef, ErrSoftFailure
}
if !hmp {
log.Warnf("incoming block's miner does not have minimum power")
return address.Undef, xerrors.New("incoming block's miner does not have minimum power")
return address.Undef, ErrInsufficientPower
}
return key, nil
@@ -541,14 +544,18 @@ func (mv *MessageValidator) Validate(ctx context.Context, pid peer.ID, msg *pubs
log.Debugf("failed to add message from network to message pool (From: %s, To: %s, Nonce: %d, Value: %s): %s", m.Message.From, m.Message.To, m.Message.Nonce, types.FIL(m.Message.Value), err)
ctx, _ = tag.New(
ctx,
tag.Insert(metrics.FailureType, "add"),
tag.Upsert(metrics.Local, "false"),
)
stats.Record(ctx, metrics.MessageValidationFailure.M(1))
recordFailure(ctx, metrics.MessageValidationFailure, "add")
switch {
case xerrors.Is(err, messagepool.ErrBroadcastAnyway):
case xerrors.Is(err, messagepool.ErrSoftValidationFailure):
fallthrough
case xerrors.Is(err, messagepool.ErrRBFTooLowPremium):
fallthrough
case xerrors.Is(err, messagepool.ErrTooManyPendingMessages):
fallthrough
case xerrors.Is(err, messagepool.ErrNonceGap):
fallthrough
case xerrors.Is(err, messagepool.ErrNonceTooLow):
return pubsub.ValidationIgnore
default:
@@ -560,37 +567,41 @@ func (mv *MessageValidator) Validate(ctx context.Context, pid peer.ID, msg *pubs
}
func (mv *MessageValidator) validateLocalMessage(ctx context.Context, msg *pubsub.Message) pubsub.ValidationResult {
ctx, _ = tag.New(
ctx,
tag.Upsert(metrics.Local, "true"),
)
// do some lightweight validation
stats.Record(ctx, metrics.MessagePublished.M(1))
m, err := types.DecodeSignedMessage(msg.Message.GetData())
if err != nil {
log.Warnf("failed to decode local message: %s", err)
stats.Record(ctx, metrics.MessageValidationFailure.M(1))
recordFailure(ctx, metrics.MessageValidationFailure, "decode")
return pubsub.ValidationIgnore
}
if m.Size() > 32*1024 {
log.Warnf("local message is too large! (%dB)", m.Size())
stats.Record(ctx, metrics.MessageValidationFailure.M(1))
recordFailure(ctx, metrics.MessageValidationFailure, "oversize")
return pubsub.ValidationIgnore
}
if m.Message.To == address.Undef {
log.Warn("local message has invalid destination address")
stats.Record(ctx, metrics.MessageValidationFailure.M(1))
recordFailure(ctx, metrics.MessageValidationFailure, "undef-addr")
return pubsub.ValidationIgnore
}
if !m.Message.Value.LessThan(types.TotalFilecoinInt) {
log.Warnf("local messages has too high value: %s", m.Message.Value)
stats.Record(ctx, metrics.MessageValidationFailure.M(1))
recordFailure(ctx, metrics.MessageValidationFailure, "value-too-high")
return pubsub.ValidationIgnore
}
if err := mv.mpool.VerifyMsgSig(m); err != nil {
log.Warnf("signature verification failed for local message: %s", err)
stats.Record(ctx, metrics.MessageValidationFailure.M(1))
recordFailure(ctx, metrics.MessageValidationFailure, "verify-sig")
return pubsub.ValidationIgnore
}
@@ -613,3 +624,11 @@ func HandleIncomingMessages(ctx context.Context, mpool *messagepool.MessagePool,
// Do nothing... everything happens in validate
}
}
func recordFailure(ctx context.Context, metric *stats.Int64Measure, failureType string) {
ctx, _ = tag.New(
ctx,
tag.Upsert(metrics.FailureType, failureType),
)
stats.Record(ctx, metric.M(1))
}
+123 -29
View File
@@ -9,8 +9,13 @@ import (
"sort"
"strconv"
"strings"
"sync"
"time"
"github.com/filecoin-project/lotus/node/modules/dtypes"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/Gurpartap/async"
"github.com/hashicorp/go-multierror"
"github.com/ipfs/go-cid"
@@ -25,18 +30,18 @@ import (
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/filecoin-project/lotus/extern/sector-storage/ffiwrapper"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/builtin"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/specs-actors/actors/util/adt"
blst "github.com/supranational/blst/bindings/go"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/beacon"
"github.com/filecoin-project/lotus/chain/blocksync"
"github.com/filecoin-project/lotus/chain/exchange"
"github.com/filecoin-project/lotus/chain/gen"
"github.com/filecoin-project/lotus/chain/state"
"github.com/filecoin-project/lotus/chain/stmgr"
@@ -50,7 +55,7 @@ import (
)
// Blocks that are more than MaxHeightDrift epochs above
//the theoretical max height based on systime are quickly rejected
// the theoretical max height based on systime are quickly rejected
const MaxHeightDrift = 5
var defaultMessageFetchWindowSize = 200
@@ -87,7 +92,7 @@ var LocalIncoming = "incoming"
// The Syncer does not run workers itself. It's mainly concerned with
// ensuring a consistent state of chain consensus. The reactive and network-
// interfacing processes are part of other components, such as the SyncManager
// (which owns the sync scheduler and sync workers), BlockSync, the HELLO
// (which owns the sync scheduler and sync workers), ChainExchange, the HELLO
// protocol, and the gossipsub block propagation layer.
//
// {hint/concept} The fork-choice rule as it currently stands is: "pick the
@@ -98,7 +103,7 @@ type Syncer struct {
store *store.ChainStore
// handle to the random beacon for verification
beacon beacon.RandomBeacon
beacon beacon.Schedule
// the state manager handles making state queries
sm *stmgr.StateManager
@@ -110,7 +115,7 @@ type Syncer struct {
bad *BadBlockCache
// handle to the block sync service
Bsync *blocksync.BlockSync
Exchange exchange.Client
self peer.ID
@@ -123,10 +128,20 @@ type Syncer struct {
receiptTracker *blockReceiptTracker
verifier ffiwrapper.Verifier
windowSize int
tickerCtxCancel context.CancelFunc
checkptLk sync.Mutex
checkpt types.TipSetKey
ds dtypes.MetadataDS
}
// NewSyncer creates a new Syncer object.
func NewSyncer(sm *stmgr.StateManager, bsync *blocksync.BlockSync, connmgr connmgr.ConnManager, self peer.ID, beacon beacon.RandomBeacon, verifier ffiwrapper.Verifier) (*Syncer, error) {
func NewSyncer(ds dtypes.MetadataDS, sm *stmgr.StateManager, exchange exchange.Client, connmgr connmgr.ConnManager, self peer.ID, beacon beacon.Schedule, verifier ffiwrapper.Verifier) (*Syncer, error) {
gen, err := sm.ChainStore().GetGenesis()
if err != nil {
return nil, xerrors.Errorf("getting genesis block: %w", err)
@@ -137,17 +152,25 @@ func NewSyncer(sm *stmgr.StateManager, bsync *blocksync.BlockSync, connmgr connm
return nil, err
}
cp, err := loadCheckpoint(ds)
if err != nil {
return nil, xerrors.Errorf("error loading mpool config: %w", err)
}
s := &Syncer{
ds: ds,
checkpt: cp,
beacon: beacon,
bad: NewBadBlockCache(),
Genesis: gent,
Bsync: bsync,
Exchange: exchange,
store: sm.ChainStore(),
sm: sm,
self: self,
receiptTracker: newBlockReceiptTracker(),
connmgr: connmgr,
verifier: verifier,
windowSize: defaultMessageFetchWindowSize,
incoming: pubsub.New(50),
}
@@ -163,11 +186,35 @@ func NewSyncer(sm *stmgr.StateManager, bsync *blocksync.BlockSync, connmgr connm
}
func (syncer *Syncer) Start() {
tickerCtx, tickerCtxCancel := context.WithCancel(context.Background())
syncer.syncmgr.Start()
syncer.tickerCtxCancel = tickerCtxCancel
go syncer.runMetricsTricker(tickerCtx)
}
func (syncer *Syncer) runMetricsTricker(tickerCtx context.Context) {
genesisTime := time.Unix(int64(syncer.Genesis.MinTimestamp()), 0)
ticker := build.Clock.Ticker(time.Duration(build.BlockDelaySecs) * time.Second)
defer ticker.Stop()
for {
select {
case <-ticker.C:
sinceGenesis := build.Clock.Now().Sub(genesisTime)
expectedHeight := int64(sinceGenesis.Seconds()) / int64(build.BlockDelaySecs)
stats.Record(tickerCtx, metrics.ChainNodeHeightExpected.M(expectedHeight))
case <-tickerCtx.Done():
return
}
}
}
func (syncer *Syncer) Stop() {
syncer.syncmgr.Stop()
syncer.tickerCtxCancel()
}
// InformNewHead informs the syncer about a new potential tipset
@@ -217,7 +264,7 @@ func (syncer *Syncer) InformNewHead(from peer.ID, fts *store.FullTipSet) bool {
return false
}
syncer.Bsync.AddPeer(from)
syncer.Exchange.AddPeer(from)
bestPweight := syncer.store.GetHeaviestTipSet().ParentWeight()
targetWeight := fts.TipSet().ParentWeight()
@@ -448,7 +495,7 @@ func computeMsgMeta(bs cbor.IpldStore, bmsgCids, smsgCids []cid.Cid) (cid.Cid, e
}
// FetchTipSet tries to load the provided tipset from the store, and falls back
// to the network (BlockSync) by querying the supplied peer if not found
// to the network (client) by querying the supplied peer if not found
// locally.
//
// {hint/usage} This is used from the HELLO protocol, to fetch the greeting
@@ -459,7 +506,7 @@ func (syncer *Syncer) FetchTipSet(ctx context.Context, p peer.ID, tsk types.TipS
}
// fall back to the network.
return syncer.Bsync.GetFullTipSet(ctx, p, tsk)
return syncer.Exchange.GetFullTipSet(ctx, p, tsk)
}
// tryLoadFullTipSet queries the tipset in the ChainStore, and returns a full
@@ -655,7 +702,7 @@ func (syncer *Syncer) ValidateBlock(ctx context.Context, b *types.FullBlock) (er
validationStart := build.Clock.Now()
defer func() {
stats.Record(ctx, metrics.BlockValidationDurationMilliseconds.M(metrics.SinceInMilliseconds(validationStart)))
log.Infow("block validation", "took", time.Since(validationStart), "height", b.Header.Height)
log.Infow("block validation", "took", time.Since(validationStart), "height", b.Header.Height, "age", time.Since(time.Unix(int64(b.Header.Timestamp), 0)))
}()
ctx, span := trace.StartSpan(ctx, "validateBlock")
@@ -832,7 +879,7 @@ func (syncer *Syncer) ValidateBlock(ctx context.Context, b *types.FullBlock) (er
return nil
}
if err := beacon.ValidateBlockValues(syncer.beacon, h, *prevBeacon); err != nil {
if err := beacon.ValidateBlockValues(syncer.beacon, h, baseTs.Height(), *prevBeacon); err != nil {
return xerrors.Errorf("failed to validate blocks random beacon values: %w", err)
}
return nil
@@ -844,10 +891,12 @@ func (syncer *Syncer) ValidateBlock(ctx context.Context, b *types.FullBlock) (er
return xerrors.Errorf("failed to marshal miner address to cbor: %w", err)
}
beaconBase := *prevBeacon
if len(h.BeaconEntries) == 0 {
if h.Height > build.UpgradeSmokeHeight {
buf.Write(baseTs.MinTicket().VRFProof)
} else {
}
beaconBase := *prevBeacon
if len(h.BeaconEntries) != 0 {
beaconBase = h.BeaconEntries[len(h.BeaconEntries)-1]
}
@@ -950,7 +999,7 @@ func (syncer *Syncer) VerifyWinningPoStProof(ctx context.Context, h *types.Block
return xerrors.Errorf("getting winning post sector set: %w", err)
}
ok, err := ffiwrapper.ProofVerifier.VerifyWinningPoSt(ctx, abi.WinningPoStVerifyInfo{
ok, err := ffiwrapper.ProofVerifier.VerifyWinningPoSt(ctx, proof.WinningPoStVerifyInfo{
Randomness: rand,
Proofs: h.WinPoStProof,
ChallengedSectors: sectors,
@@ -1161,7 +1210,7 @@ func extractSyncState(ctx context.Context) *SyncerState {
// total equality of the BeaconEntries in each block.
// 3. Traverse the chain backwards, for each tipset:
// 3a. Load it from the chainstore; if found, it move on to its parent.
// 3b. Query our peers via BlockSync in batches, requesting up to a
// 3b. Query our peers via client in batches, requesting up to a
// maximum of 500 tipsets every time.
//
// Once we've concluded, if we find a mismatching tipset at the height where the
@@ -1262,7 +1311,7 @@ loop:
if gap := int(blockSet[len(blockSet)-1].Height() - untilHeight); gap < window {
window = gap
}
blks, err := syncer.Bsync.GetBlocks(ctx, at, window)
blks, err := syncer.Exchange.GetBlocks(ctx, at, window)
if err != nil {
// Most likely our peers aren't fully synced yet, but forwarded
// new block message (ideally we'd find better peers)
@@ -1280,7 +1329,7 @@ loop:
// have. Since we fetch from the head backwards our reassembled chain
// is sorted in reverse here: we have a child -> parent order, our last
// tipset then should be child of the first tipset retrieved.
// FIXME: The reassembly logic should be part of the `BlockSync`
// FIXME: The reassembly logic should be part of the `client`
// service, the consumer should not be concerned with the
// `MaxRequestLength` limitation, it should just be able to request
// an segment of arbitrary length. The same burden is put on
@@ -1330,7 +1379,7 @@ loop:
log.Warnf("(fork detected) synced header chain (%s - %d) does not link to our best block (%s - %d)", incoming.Cids(), incoming.Height(), known.Cids(), known.Height())
fork, err := syncer.syncFork(ctx, base, known)
if err != nil {
if xerrors.Is(err, ErrForkTooLong) {
if xerrors.Is(err, ErrForkTooLong) || xerrors.Is(err, ErrForkCheckpoint) {
// TODO: we're marking this block bad in the same way that we mark invalid blocks bad. Maybe distinguish?
log.Warn("adding forked chain to our bad tipset cache")
for _, b := range incoming.Blocks() {
@@ -1346,15 +1395,24 @@ loop:
}
var ErrForkTooLong = fmt.Errorf("fork longer than threshold")
var ErrForkCheckpoint = fmt.Errorf("fork would require us to diverge from checkpointed block")
// syncFork tries to obtain the chain fragment that links a fork into a common
// ancestor in our view of the chain.
//
// If the fork is too long (build.ForkLengthThreshold), we add the entire subchain to the
// denylist. Else, we find the common ancestor, and add the missing chain
// If the fork is too long (build.ForkLengthThreshold), or would cause us to diverge from the checkpoint (ErrForkCheckpoint),
// we add the entire subchain to the denylist. Else, we find the common ancestor, and add the missing chain
// fragment until the fork point to the returned []TipSet.
func (syncer *Syncer) syncFork(ctx context.Context, incoming *types.TipSet, known *types.TipSet) ([]*types.TipSet, error) {
tips, err := syncer.Bsync.GetBlocks(ctx, incoming.Parents(), int(build.ForkLengthThreshold))
chkpt := syncer.GetCheckpoint()
if known.Key() == chkpt {
return nil, ErrForkCheckpoint
}
// TODO: Does this mean we always ask for ForkLengthThreshold blocks from the network, even if we just need, like, 2?
// Would it not be better to ask in smaller chunks, given that an ~ForkLengthThreshold is very rare?
tips, err := syncer.Exchange.GetBlocks(ctx, incoming.Parents(), int(build.ForkLengthThreshold))
if err != nil {
return nil, err
}
@@ -1379,12 +1437,18 @@ func (syncer *Syncer) syncFork(ctx context.Context, incoming *types.TipSet, know
if nts.Height() < tips[cur].Height() {
cur++
} else {
// We will be forking away from nts, check that it isn't checkpointed
if nts.Key() == chkpt {
return nil, ErrForkCheckpoint
}
nts, err = syncer.store.LoadTipSet(nts.Parents())
if err != nil {
return nil, xerrors.Errorf("loading next local tipset: %w", err)
}
}
}
return nil, ErrForkTooLong
}
@@ -1408,12 +1472,14 @@ func (syncer *Syncer) syncMessagesAndCheckState(ctx context.Context, headers []*
// fills out each of the given tipsets with messages and calls the callback with it
func (syncer *Syncer) iterFullTipsets(ctx context.Context, headers []*types.TipSet, cb func(context.Context, *store.FullTipSet) error) error {
ss := extractSyncState(ctx)
ctx, span := trace.StartSpan(ctx, "iterFullTipsets")
defer span.End()
span.AddAttributes(trace.Int64Attribute("num_headers", int64(len(headers))))
windowSize := defaultMessageFetchWindowSize
windowSize := syncer.windowSize
mainLoop:
for i := len(headers) - 1; i >= 0; {
fts, err := syncer.store.TryFillTipSet(headers[i])
if err != nil {
@@ -1434,19 +1500,27 @@ func (syncer *Syncer) iterFullTipsets(ctx context.Context, headers []*types.TipS
nextI := (i + 1) - batchSize // want to fetch batchSize values, 'i' points to last one we want to fetch, so its 'inclusive' of our request, thus we need to add one to our request start index
var bstout []*blocksync.CompactedMessages
ss.SetStage(api.StageFetchingMessages)
var bstout []*exchange.CompactedMessages
for len(bstout) < batchSize {
next := headers[nextI]
nreq := batchSize - len(bstout)
bstips, err := syncer.Bsync.GetChainMessages(ctx, next, uint64(nreq))
bstips, err := syncer.Exchange.GetChainMessages(ctx, next, uint64(nreq))
if err != nil {
// TODO check errors for temporary nature
if windowSize > 1 {
windowSize /= 2
log.Infof("error fetching messages: %s; reducing window size to %d and trying again", err, windowSize)
continue mainLoop
}
return xerrors.Errorf("message processing failed: %w", err)
}
bstout = append(bstout, bstips...)
nextI += len(bstips)
}
ss.SetStage(api.StageMessages)
for bsi := 0; bsi < len(bstout); bsi++ {
// temp storage so we don't persist data we dont want to
@@ -1475,13 +1549,28 @@ func (syncer *Syncer) iterFullTipsets(ctx context.Context, headers []*types.TipS
return xerrors.Errorf("message processing failed: %w", err)
}
}
if i >= windowSize {
newWindowSize := windowSize + 10
if newWindowSize > int(exchange.MaxRequestLength) {
newWindowSize = int(exchange.MaxRequestLength)
}
if newWindowSize > windowSize {
windowSize = newWindowSize
log.Infof("successfully fetched %d messages; increasing window size to %d", len(bstout), windowSize)
}
}
i -= batchSize
}
// remember our window size
syncer.windowSize = windowSize
return nil
}
func persistMessages(bs bstore.Blockstore, bst *blocksync.CompactedMessages) error {
func persistMessages(bs bstore.Blockstore, bst *exchange.CompactedMessages) error {
for _, m := range bst.Bls {
//log.Infof("putting BLS message: %s", m.Cid())
if _, err := store.PutMessage(bs, m); err != nil {
@@ -1588,6 +1677,11 @@ func (syncer *Syncer) MarkBad(blk cid.Cid) {
syncer.bad.Add(blk, NewBadBlockReason([]cid.Cid{blk}, "manually marked bad"))
}
// UnmarkBad manually adds a block to the "bad blocks" cache.
func (syncer *Syncer) UnmarkBad(blk cid.Cid) {
syncer.bad.Remove(blk)
}
func (syncer *Syncer) CheckBadBlockCache(blk cid.Cid) (string, bool) {
bbr, ok := syncer.bad.Has(blk)
return bbr.String(), ok
+120 -4
View File
@@ -7,6 +7,8 @@ import (
"testing"
"time"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/ipfs/go-cid"
ds "github.com/ipfs/go-datastore"
@@ -16,8 +18,8 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin/miner"
"github.com/filecoin-project/specs-actors/actors/builtin/power"
"github.com/filecoin-project/specs-actors/actors/builtin/verifreg"
@@ -331,6 +333,36 @@ func (tu *syncTestUtil) compareSourceState(with int) {
}
}
func (tu *syncTestUtil) assertBad(node int, ts *types.TipSet) {
for _, blk := range ts.Cids() {
rsn, err := tu.nds[node].SyncCheckBad(context.TODO(), blk)
require.NoError(tu.t, err)
require.True(tu.t, len(rsn) != 0)
}
}
func (tu *syncTestUtil) getHead(node int) *types.TipSet {
ts, err := tu.nds[node].ChainHead(context.TODO())
require.NoError(tu.t, err)
return ts
}
func (tu *syncTestUtil) checkpointTs(node int, tsk types.TipSetKey) {
require.NoError(tu.t, tu.nds[node].SyncCheckpoint(context.TODO(), tsk))
}
func (tu *syncTestUtil) waitUntilNodeHasTs(node int, tsk types.TipSetKey) {
for {
_, err := tu.nds[node].ChainGetTipSet(context.TODO(), tsk)
if err != nil {
break
}
}
// Time to allow for syncing and validation
time.Sleep(2 * time.Second)
}
func (tu *syncTestUtil) waitUntilSync(from, to int) {
target, err := tu.nds[from].ChainHead(tu.ctx)
if err != nil {
@@ -442,8 +474,8 @@ func (wpp badWpp) GenerateCandidates(context.Context, abi.PoStRandomness, uint64
return []uint64{1}, nil
}
func (wpp badWpp) ComputeProof(context.Context, []abi.SectorInfo, abi.PoStRandomness) ([]abi.PoStProof, error) {
return []abi.PoStProof{
func (wpp badWpp) ComputeProof(context.Context, []proof.SectorInfo, abi.PoStRandomness) ([]proof.PoStProof, error) {
return []proof.PoStProof{
{
PoStProof: abi.RegisteredPoStProof_StackedDrgWinning2KiBV1,
ProofBytes: []byte("evil"),
@@ -676,3 +708,87 @@ func TestSyncInputs(t *testing.T) {
t.Fatal("should error on block with nil election proof")
}
}
func TestSyncCheckpointHead(t *testing.T) {
H := 10
tu := prepSyncTest(t, H)
p1 := tu.addClientNode()
p2 := tu.addClientNode()
fmt.Println("GENESIS: ", tu.g.Genesis().Cid())
tu.loadChainToNode(p1)
tu.loadChainToNode(p2)
base := tu.g.CurTipset
fmt.Println("Mining base: ", base.TipSet().Cids(), base.TipSet().Height())
// The two nodes fork at this point into 'a' and 'b'
a1 := tu.mineOnBlock(base, p1, []int{0}, true, false, nil)
a := tu.mineOnBlock(a1, p1, []int{0}, true, false, nil)
a = tu.mineOnBlock(a, p1, []int{0}, true, false, nil)
tu.waitUntilSyncTarget(p1, a.TipSet())
tu.checkpointTs(p1, a.TipSet().Key())
require.NoError(t, tu.g.ResyncBankerNonce(a1.TipSet()))
// chain B will now be heaviest
b := tu.mineOnBlock(base, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
fmt.Println("A: ", a.Cids(), a.TipSet().Height())
fmt.Println("B: ", b.Cids(), b.TipSet().Height())
// Now for the fun part!! p1 should mark p2's head as BAD.
require.NoError(t, tu.mn.LinkAll())
tu.connect(p1, p2)
tu.waitUntilNodeHasTs(p1, b.TipSet().Key())
p1Head := tu.getHead(p1)
require.Equal(tu.t, p1Head, a.TipSet())
tu.assertBad(p1, b.TipSet())
}
func TestSyncCheckpointEarlierThanHead(t *testing.T) {
H := 10
tu := prepSyncTest(t, H)
p1 := tu.addClientNode()
p2 := tu.addClientNode()
fmt.Println("GENESIS: ", tu.g.Genesis().Cid())
tu.loadChainToNode(p1)
tu.loadChainToNode(p2)
base := tu.g.CurTipset
fmt.Println("Mining base: ", base.TipSet().Cids(), base.TipSet().Height())
// The two nodes fork at this point into 'a' and 'b'
a1 := tu.mineOnBlock(base, p1, []int{0}, true, false, nil)
a := tu.mineOnBlock(a1, p1, []int{0}, true, false, nil)
a = tu.mineOnBlock(a, p1, []int{0}, true, false, nil)
tu.waitUntilSyncTarget(p1, a.TipSet())
tu.checkpointTs(p1, a1.TipSet().Key())
require.NoError(t, tu.g.ResyncBankerNonce(a1.TipSet()))
// chain B will now be heaviest
b := tu.mineOnBlock(base, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
b = tu.mineOnBlock(b, p2, []int{1}, true, false, nil)
fmt.Println("A: ", a.Cids(), a.TipSet().Height())
fmt.Println("B: ", b.Cids(), b.TipSet().Height())
// Now for the fun part!! p1 should mark p2's head as BAD.
require.NoError(t, tu.mn.LinkAll())
tu.connect(p1, p2)
tu.waitUntilNodeHasTs(p1, b.TipSet().Key())
p1Head := tu.getHead(p1)
require.Equal(tu.t, p1Head, a.TipSet())
tu.assertBad(p1, b.TipSet())
}
+1 -19
View File
@@ -1,34 +1,16 @@
package chain
import (
"fmt"
"sync"
"time"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/types"
)
func SyncStageString(v api.SyncStateStage) string {
switch v {
case api.StageHeaders:
return "header sync"
case api.StagePersistHeaders:
return "persisting headers"
case api.StageMessages:
return "message sync"
case api.StageSyncComplete:
return "complete"
case api.StageSyncErrored:
return "error"
default:
return fmt.Sprintf("<unknown: %d>", v)
}
}
type SyncerState struct {
lk sync.Mutex
Target *types.TipSet
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"fmt"
"math/big"
big2 "github.com/filecoin-project/specs-actors/actors/abi/big"
big2 "github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
)
+5 -3
View File
@@ -4,10 +4,12 @@ import (
"bytes"
"math/big"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
"github.com/minio/blake2b-simd"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
block "github.com/ipfs/go-block-format"
"github.com/ipfs/go-cid"
@@ -53,7 +55,7 @@ type BlockHeader struct {
BeaconEntries []BeaconEntry // 3
WinPoStProof []abi.PoStProof // 4
WinPoStProof []proof.PoStProof // 4
Parents []cid.Cid // 5
+5 -3
View File
@@ -7,12 +7,14 @@ import (
"reflect"
"testing"
"github.com/filecoin-project/specs-actors/actors/runtime/proof"
cid "github.com/ipfs/go-cid"
"github.com/stretchr/testify/require"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
)
func testBlockHeader(t testing.TB) *BlockHeader {
@@ -80,7 +82,7 @@ func TestInteropBH(t *testing.T) {
t.Fatal(err)
}
posts := []abi.PoStProof{
posts := []proof.PoStProof{
{PoStProof: abi.RegisteredPoStProof_StackedDrgWinning2KiBV1, ProofBytes: []byte{0x07}},
}
+8 -7
View File
@@ -6,9 +6,10 @@ import (
"fmt"
"io"
abi "github.com/filecoin-project/specs-actors/actors/abi"
crypto "github.com/filecoin-project/specs-actors/actors/crypto"
exitcode "github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
abi "github.com/filecoin-project/go-state-types/abi"
crypto "github.com/filecoin-project/go-state-types/crypto"
exitcode "github.com/filecoin-project/go-state-types/exitcode"
proof "github.com/filecoin-project/specs-actors/actors/runtime/proof"
cid "github.com/ipfs/go-cid"
cbg "github.com/whyrusleeping/cbor-gen"
xerrors "golang.org/x/xerrors"
@@ -58,7 +59,7 @@ func (t *BlockHeader) MarshalCBOR(w io.Writer) error {
}
}
// t.WinPoStProof ([]abi.PoStProof) (slice)
// t.WinPoStProof ([]proof.PoStProof) (slice)
if len(t.WinPoStProof) > cbg.MaxLength {
return xerrors.Errorf("Slice value in field t.WinPoStProof was too long")
}
@@ -243,7 +244,7 @@ func (t *BlockHeader) UnmarshalCBOR(r io.Reader) error {
t.BeaconEntries[i] = v
}
// t.WinPoStProof ([]abi.PoStProof) (slice)
// t.WinPoStProof ([]proof.PoStProof) (slice)
maj, extra, err = cbg.CborReadHeaderBuf(br, scratch)
if err != nil {
@@ -259,12 +260,12 @@ func (t *BlockHeader) UnmarshalCBOR(r io.Reader) error {
}
if extra > 0 {
t.WinPoStProof = make([]abi.PoStProof, extra)
t.WinPoStProof = make([]proof.PoStProof, extra)
}
for i := 0; i < int(extra); i++ {
var v abi.PoStProof
var v proof.PoStProof
if err := v.UnmarshalCBOR(br); err != nil {
return err
}
+2 -2
View File
@@ -4,9 +4,9 @@ import (
"bytes"
"fmt"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/abi/big"
block "github.com/ipfs/go-block-format"
"github.com/ipfs/go-cid"
xerrors "golang.org/x/xerrors"
+1 -1
View File
@@ -3,7 +3,7 @@ package types
import (
"bytes"
"github.com/filecoin-project/specs-actors/actors/runtime/exitcode"
"github.com/filecoin-project/go-state-types/exitcode"
)
type MessageReceipt struct {
+1 -1
View File
@@ -5,7 +5,7 @@ import (
"github.com/stretchr/testify/require"
"github.com/filecoin-project/specs-actors/actors/abi/big"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/specs-actors/actors/builtin"
)
+4 -2
View File
@@ -5,10 +5,11 @@ import (
"fmt"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/specs-actors/actors/abi"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/crypto"
"github.com/ipfs/go-cid"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/wallet"
)
@@ -80,6 +81,7 @@ func MkBlock(parents *types.TipSet, weightInc uint64, ticketNonce uint64) *types
Height: height,
ParentStateRoot: pstateRoot,
BlockSig: &crypto.Signature{Type: crypto.SigTypeBLS, Data: []byte("boo! im a signature")},
ParentBaseFee: types.NewInt(uint64(build.MinimumBaseFee)),
}
}
+1 -1
View File
@@ -4,7 +4,7 @@ import (
"bytes"
"testing"
"github.com/filecoin-project/specs-actors/actors/crypto"
"github.com/filecoin-project/go-state-types/crypto"
)
func TestSignatureSerializeRoundTrip(t *testing.T) {

Some files were not shown because too many files have changed in this diff Show More