Compare commits

...
Author SHA1 Message Date
Phi-rjan c6968f8d61 Merge pull request #10643 from filecoin-project/phi/v1210rc3
build: release: v1.21.0-rc3
2023-04-10 18:02:30 +02:00
Phi 8096f7135b build: release: v1.21.0-rc3
build: release: v1.21.0-rc3
2023-04-10 17:44:21 +02:00
ZenGround0 5e890165e2 Merge pull request #10641 from filecoin-project/phi/backport-splitstore-oos
Backport #10392 into v1.21.0
2023-04-10 11:28:42 -04:00
Steven Allen 34f46d736f fix: revert chain sync mutex to a regular lock
Conditions always call "unlock" so we can't safely use the condition
with both the read and write side of lock. So we might as well revert
back to a regular lock.

fixes #10616
2023-04-10 17:16:20 +02:00
zenground0 d77ac5ddcb Stop swallowing errors 2023-04-10 17:16:13 +02:00
zenground0 0ce9ae4809 Implement yield friendly online GC 2023-04-10 17:16:01 +02:00
zenground0 fd66db680f check yield before GC 2023-04-10 17:15:52 +02:00
zenground0 b5da33c013 signal chain in and out of sync to compaction workers 2023-04-10 17:15:42 +02:00
Phi-rjan 95e1c4909c Merge pull request #10627 from filecoin-project/phi/v1210
build: release: v1.21.0 prep
2023-04-05 20:00:15 +02:00
Phi e426d05bc8 fix typos
fix typos
2023-04-05 19:09:31 +02:00
Phi b6b3802bb7 build: release: v1.21.0 prep
build: release: v1.21.0 prep
2023-04-05 18:55:18 +02:00
Phi-rjanandzenground0 b240031605 Limit moving gc threads (#10623)
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2023-04-05 11:21:32 -04:00
Phi-rjan 33039d2cec Merge pull request #10620 from filecoin-project/phi/backport-splitstore-config
Backport #10605 to v1.21.0
2023-04-04 22:36:55 +02:00
zenground0 0314db7894 Update config default value 2023-04-04 20:35:28 +02:00
Aayush Rajasekaran ef5c6c4c48 Merge pull request #10607 from filecoin-project/phi/backport-update-gst
Backport #10606 to v1.21.0
2023-03-31 10:01:54 -04:00
Aayush 8a7ae61dae chore: deps: update to go-state-types v0.11.0-alpha-3 2023-03-31 09:17:11 +02:00
Jiaying Wang caa6dd9c41 Merge pull request #10594 from filecoin-project/phi/v1210-rc2
build: release: v1.21.0-rc2
2023-03-29 15:59:56 -04:00
Aayush a724a713da refactor: stop using deprecated io/ioutil 2023-03-29 15:38:46 -04:00
Phi d5a3ddbf7e Add longer validation time cause to changelog
Add longer validation time cause to changelog
2023-03-29 20:39:29 +02:00
Aayush Rajasekaran b40ab4bd44 Merge pull request #10595 from filecoin-project/asr/backport
Backport  #10546, #10553
2023-03-29 14:31:43 -04:00
Phi 7a95ed063d Bump version in changelog
Bump version in changelog
2023-03-29 20:31:05 +02:00
Phi 199a1d9e75 Update Changelog with backports
Update Changelog with backports
2023-03-29 20:30:01 +02:00
Aayush 3120852eb9 feat: supply: only grab genesis msig locks for writes 2023-03-29 14:26:15 -04:00
Aayush 24381411f2 feat: supply: drop genesis market locked funds 2023-03-29 14:26:08 -04:00
06294351fa perf: eth: gas estimate set applyTsMessages false (#10546)
* have gas estimate call callInternal with applyTsMessages = false and other calls with applyTsMessages=true for gas caclulation optimization

* set applyTsMessages = true in CallWithGas call in shed

* update test with new callwithgas api optimization for eth call

* Update chain/stmgr/call.go

Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* make sure that even if we arent apply ts messages we grab ts messages from the particular user who is requesting gas estimation

---------

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-10-0-4-29.us-east-2.compute.internal>
2023-03-29 14:25:40 -04:00
Phi d3d5f92985 Update changelog for v1.21.0-rc2
Update changelog for v1.21.0-rc2
2023-03-29 20:11:50 +02:00
Phi ca1247e0d9 Update go.mod
Update go.mod
2023-03-29 19:50:14 +02:00
Phi a647390bef build: release: v1.21.0-rc2
build: release: v1.21.0-rc2
2023-03-29 19:38:47 +02:00
Phi-rjan db869853c4 Merge pull request #10591 from filecoin-project/phi/backport-go-update
build: docker: Update GO-version
2023-03-29 17:31:46 +02:00
Phi 9d5e789d88 build: docker: Update GO-version
build: docker: Update GO-version
2023-03-29 17:12:43 +02:00
Aayush Rajasekaran 0b018e56cd Merge pull request #10555 from filecoin-project/asr/fix-extend-sectors-backport
backport: fix: miner: correctly count sector extensions
2023-03-28 09:40:42 -04:00
Phi-rjan 7eae362c1e Merge pull request #10563 from Jorropo/release/v1.21.0
chore: all: bump go-libipfs
2023-03-28 08:42:46 +02:00
Phi-rjan af428b6298 Merge pull request #10574 from filecoin-project/phi/backport-slice-fix
fix: proving: Initialize slice with with same length as partition
2023-03-27 16:28:02 +02:00
Phi 8bb7272bb1 Initialize with same length as partition
Initialize the postParam.Partitions slice with the same length as i.Partitions before iterating over it in the loop.
2023-03-27 15:54:59 +02:00
Jorropo 0e73c69035 chore: all: bump go-libipfs
(cherry picked from commit f48c6268f8)
2023-03-25 10:15:21 +01:00
Aayush 7b55de21bd fix: miner: call ExtendSectorExpiration2 2023-03-23 17:32:55 -04:00
Aayush a467e1ab31 fix: miner: correctly count sector extensions 2023-03-23 17:32:51 -04:00
Phi-rjan 3f8a41cfe8 Merge pull request #10524 from filecoin-project/jen/v121
chore: build: v1.21.0-rc1
2023-03-22 17:13:12 +01:00
Phi a8580e234d Add callout to exchanges
Add callout to exchanges about the execution traces. Also moving the `execution trace format changes` section to above the contributors section.
2023-03-22 15:07:21 +01:00
Phi 938d2d9f5b Add RPC API highlight
Add RPC API highlight
2023-03-21 19:55:06 +01:00
Phi c8af33f163 chore: build: Update Changelog for v1.21.0
chore: build: Update Changelog for v1.21.0
2023-03-21 14:33:36 +01:00
Jennifer Wang 57133a8f66 release v1.21.0-rc1 prep 2023-03-21 01:10:32 -04:00
ZenGround0andzenground0 43da108466 feat:config:force existing users to opt into new defaults (#10488)
* Config default does not comment out EnableSplitstore

* Loadability check

* Remove test used for debugging

* regexp for properly safe check that config is set

* regexp for safely matching the EnableSpitstore field in the config

* Add instructions for undeleting config and remind users to set splitstore false for full archive

* UpdateConfig small docs and functional opts

* make gen

* Lint

* Fix

* nil pointer check on validate

* Unit testing of EnableSplitstore cases

* Address Review

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2023-03-20 12:19:14 -04:00
ZenGround0andzenground0 2b3a86eefb Demote now common logs (#10516)
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2023-03-20 10:54:30 -04:00
Aayush Rajasekaran 2a3dcf6cbd Merge pull request #10502 from filecoin-project/fix/splitstore-handles-dangling-event-ptr
fix:splitstore:Don't enforce walking receipt tree during compaction
2023-03-17 13:29:48 -04:00
zenground0 252387768d Don't enforce walking receipt tree during compaction 2023-03-17 09:29:18 -06:00
Łukasz Magiera eed1a098c6 Merge pull request #10495 from filecoin-project/steb/fix-div-by-zero
fix: eth: handle a potential divide by zero in receipt handling
2023-03-17 11:32:27 +01:00
Steven Allen 2207ab3c93 fix: eth: handle a potential divide by zero in receipt handling
This isn't really possible to hit on-chain at this piont (message won't
be accepted) but we might as well be extra careful.

fixes #10471
2023-03-16 08:53:52 -07:00
Aayush Rajasekaran 97a3b6e06d Merge pull request #10490 from filecoin-project/jen/m
chore: releases to master
2023-03-16 09:59:32 -04:00
Łukasz Magiera 093ff95580 Merge pull request #10467 from filecoin-project/steb/gw-eth-lookback-limit
feat: apply gateway lookback limit to eth API lookback
2023-03-16 08:50:59 +01:00
Łukasz Magiera c0aed3c4a2 Merge pull request #10474 from filecoin-project/raulk/fix-pending-support
fix: revert "Eth API: drop support for 'pending' block parameter."
2023-03-16 08:49:49 +01:00
Jennifer Wang d56980823b Merge branch 'releases' into jen/m 2023-03-15 21:23:08 -04:00
Steven Allen 17274e72da feat: expose limited apis on the gateway 2023-03-15 16:12:18 -07:00
Steven Allen 3776ee3b62 fix: actually apply limit 2023-03-15 16:11:13 -07:00
Steven Allen fee480193e feat: apply gateway lookback limit to eth API lookback
This change:

1. Introduces new "limited" API endpoints for EthGetTransactionByHash
   and EthGetTransactionReceipt that accept lookback-limits.
2. Implements the gateway version of these API endpoints by calling the
   limited variants with the default message search lookback limit.

fixes #10412
2023-03-15 16:11:11 -07:00
Łukasz Magiera 4c80e179ae Merge pull request #10485 from filecoin-project/fix/ethsub-hang
fix: ethrpc: Don't lock up when eth subscriber goes away
2023-03-15 20:37:33 +01:00
Łukasz Magiera 720e95991c make lint happy 2023-03-15 18:52:11 +01:00
Łukasz Magiera 470dd8da3d ethrpc: Buffer sub messages if subscriber is slow 2023-03-15 18:34:10 +01:00
Łukasz Magiera a44f598cc2 update go-jsonrpc to v0.2.3 2023-03-15 17:19:32 +01:00
Aayush Rajasekaran 7036e76e17 Merge pull request #10476 from filecoin-project/asr/drop-incentinet-servers
fix: build: drop drand incentinet servers
2023-03-15 11:09:45 -04:00
Łukasz Magiera 8121f379a3 Merge pull request #10483 from filecoin-project/deps/update-pubsub
deps: update go-libp2p-pubsub to v0.9.3
2023-03-15 14:01:42 +01:00
vyzo 7e21486103 update go-libp2p-pubsub to v0.9.3 2023-03-15 13:57:20 +02:00
Łukasz Magiera b8589e8c51 Merge pull request #10382 from filecoin-project/feat/go-data-transfer-v2
Update to go-data-transfer v2 and libp2p, still wip
2023-03-15 12:45:01 +01:00
Łukasz Magiera 311645b559 make mod-tidy ci happy 2023-03-15 12:13:57 +01:00
Łukasz Magiera 7e414ac390 Merge remote-tracking branch 'origin/master' into feat/go-data-transfer-v2 2023-03-15 11:37:53 +01:00
Łukasz Magiera 486904bc92 Merge pull request #10464 from filecoin-project/steb/fee-history-gw-lookback-max
fix: gateway: correctly apply the fee history lookback max
2023-03-15 10:52:24 +01:00
Łukasz Magiera a7c9a83091 Merge pull request #10465 from ribasushi/flush_bs
chore: blockstore: Plumb through a proper Flush() method on all blockstores
2023-03-15 10:16:01 +01:00
Łukasz Magiera 81238b19ab Merge pull request #10477 from filecoin-project/asr/export-state-verifier
feat: gateway: export StateVerifierStatus
2023-03-15 10:13:42 +01:00
Marten Seemann 58306a87f1 chore: update go-libp2p to v0.26.2 2023-03-15 18:35:18 +13:00
Marten Seemann 21b74deab4 fix connection gater integration test 2023-03-15 18:35:17 +13:00
Steven Allen 6ef415744c chore: update ffi to increase execution parallelism (#10480)
This changes the default validation parallelism from 1 to 4, and makes
it configurable with a LOTUS_FVM_CONCURRENCY environment variable.
2023-03-14 19:53:39 -07:00
Marten Seemann 92dd629f42 chore: update go-libp2p-pubsub to v0.9.2 2023-03-15 15:12:22 +13:00
Aayush adfa766e1b feat: gateway: export StateVerifierStatus 2023-03-14 13:39:05 -04:00
Aayush 991895c0f2 fix: build: drop drand incentinet servers 2023-03-14 11:09:43 -04:00
Łukasz Magiera 162e07fc95 Merge pull request #10470 from filecoin-project/feat/applyblock-timing-log
feat: consensus: log ApplyBlock timing/gas stats
2023-03-14 15:25:31 +01:00
raulk e8325f9a98 Revert "Eth API: drop support for 'pending' block parameter."
This reverts commit 9412753ba3.
2023-03-14 13:46:01 +00:00
Łukasz Magiera 27e3c0b89f fix: stmgr: don't attempt to lookup genesis state (#10472)
* fix: stmgr: don't attempt to lookup genesis state

* stmgr: More correct fix for genesis state compute
2023-03-14 06:45:23 -07:00
Łukasz Magiera 51c6b64c91 feat: consensus: log ApplyBlock timing/gas stats 2023-03-14 13:58:36 +01:00
Peter Rabbitson 6afa99d0a1 Plumb through a proper Flush() method on all blockstores 2023-03-14 02:30:15 +01:00
Steven Allen 3556a4b07b fix: gateway: correctly apply the fee history lookback max
This fix ensures that the full fee-history range falls into the
allowable lookback rang.
2023-03-13 16:45:26 -07:00
Steven Allen dcb49dc8ee refactor: update cache to the new generic version (#10463)
- Adds type safety.
- Reduces allocations.
- Fixes the drand cache (was storing by value, but retrieving by pointer)
2023-03-13 15:29:09 -07:00
Aayush Rajasekaran 97a9921cdd Merge pull request #10461 from filecoin-project/steb/deflake-fail-after-finish
fix: itest: avoid failing the test when we race the miner
2023-03-13 15:13:08 -04:00
Aayush Rajasekaran 748e535cc6 Merge pull request #10444 from filecoin-project/chore/update-ffi
chore: update the FFI for release (#10435)
2023-03-13 15:04:18 -04:00
Steven Allenandraulk 73a2b4152b feat: eth: Avoid StateCompute in EthTxnReceipt lookup (#10460)
Instead:

1. Use the receipt from the message search.
2. Re-compute the gas fees that would have been charged.

fixes #10418

Co-authored-by: raulk <raul.kripalani@gmail.com>
2023-03-13 11:44:59 -07:00
Aayush Rajasekaran afa36d3f62 Merge pull request #10451 from filecoin-project/asr/fix-msig-vested
gateway: fix: drop overzealous guard on MsigGetVested
2023-03-13 14:33:50 -04:00
Steven Allen b852f5e2fd fix: itest: avoid failing the test when we race the miner
I've been seeing quite a few tests failing here, so I'm hoping this will
deflake them a bit.
2023-03-13 11:16:26 -07:00
raulk 75c279ab97 Merge pull request #10456 from filecoin-project/raulk/fix-eth-net_version 2023-03-13 11:25:19 +00:00
raulk 34d7135717 Merge pull request #10409 from filecoin-project/asr/fix-eth-api-gateway
fix: eth API: correct gateway restrictions, drop unimplemented methods
2023-03-12 16:31:42 +00:00
Raúl Kripalani 2e56237898 Merge branch 'master' into asr/fix-eth-api-gateway 2023-03-12 15:56:55 +00:00
Raúl Kripalani e65a5988e1 Eth API: make eth_getTransactionByBlock* ops return ErrUnsupported. 2023-03-12 15:53:56 +00:00
raulk aea195aeb4 Merge pull request #10446 from filecoin-project/steb/optimize-eth-block
feat: eth: optimize eth block loading + eth_feeHistory
2023-03-12 15:46:03 +00:00
Raúl Kripalani 571a84b390 drop irrelevant test. 2023-03-12 00:45:02 +00:00
Raúl Kripalani 5a4b5ff97e remove superfluous filter. 2023-03-12 00:44:49 +00:00
Raúl Kripalani 4763104b99 fix imports. 2023-03-12 00:42:56 +00:00
Raúl Kripalani 7ce92f1422 Eth API: make net_version return the chain ID. 2023-03-11 23:26:26 +00:00
Raúl Kripalani 9412753ba3 Eth API: drop support for 'pending' block parameter.
After transitioning from using StateCompute to loading receipts,
we can no longer handle the 'pending' block without forcing
computation. Eth Core Devs are evaluating a proposal to remove
support on their end too.
2023-03-11 23:03:21 +00:00
Raúl Kripalani f7a979d825 eth_feeHistory: migrate to using TipSetState. 2023-03-11 17:25:00 +00:00
Raúl Kripalani d1c6ab7dc6 Merge branch 'master' into steb/optimize-eth-block 2023-03-11 16:21:19 +00:00
raulk d7a2fdb5ba Merge pull request #10445 from filecoin-project/steb/skip-tipset-execution 2023-03-11 16:19:08 +00:00
Aayush 82e2c3538c gateway: fix: drop overzealous guard on MsigGetVested 2023-03-11 09:35:51 -05:00
Steven Allen 59bebf8a35 test: eth: reduce chances of chain-reorgs affecting the test
Now that this API is _much_ faster, we're more likely to "catch up" to
the head faster than it can stabilize. I'm pretty sure the test was
intended to be written this way anyways.
2023-03-10 16:29:43 -08:00
Steven Allen eba270d1e2 feat: api: optimize ChainGetParentReceipts
Read the receipts all at once instead of fetching them one-by-one.
2023-03-10 16:01:34 -08:00
Steven Allen 1cf57ffe2d feat: eth: optimize receipt reading
This optimizes the eth APIs (except the fee history one) to lookup the
tipset state/receipts instead of computing the state.
2023-03-10 15:33:45 -08:00
Steven Allen 92bca485b6 feat: stmgr: skip tipset execution when possible 2023-03-10 13:21:17 -08:00
Steven Allen d2c1962e34 chore: update the FFI for release (#10435)
Specifically, this updates the proofs libraries and enables CUDA by default.
2023-03-10 12:13:59 -08:00
Aayush Rajasekaran 9d994dac9e Merge pull request #10440 from filecoin-project/steb/reject-masked-id
feat: eth API: reject masked ID addresses embedded in f410f payloads
2023-03-10 14:34:15 -05:00
Aayush Rajasekaran 816ca59d6d Merge pull request #10443 from filecoin-project/steb/evm-bytecode-cmd
feat: cli: Add an EVM command to fetch a contract's bytecode
2023-03-10 14:34:01 -05:00
Steven Allen f7603f6c13 feat: eth API: reject masked ID addresses embedded in f410f payloads
We'll never get an actor/account deployed to one of these
addresses (although we might get a placeholder). However, converting
such an address to an f4 address is definitely wrong.
2023-03-10 10:57:39 -08:00
Steven Allen 7a2eb86dd5 feat: cli: Add an EVM command to fetch a contract's bytecode 2023-03-10 10:52:13 -08:00
Jiaying Wang b332db0a38 Merge pull request #10439 from filecoin-project/jen/v1203
fix: cherrypick/conflict wrongfully resolved
2023-03-09 22:58:36 -05:00
Jennifer Wang e3b3b0ca6c make gen 2023-03-09 22:11:11 -05:00
Jennifer Wang c8b4f5f0a3 go mod tidy 2023-03-09 22:03:27 -05:00
Jennifer Wang 8c6ffcf4d4 retract v1.20.2 2023-03-09 21:10:30 -05:00
Jennifer Wang 80cba5f176 fiix bakcport error #10427 2023-03-09 21:09:04 -05:00
Jiaying Wang 99749e9b9b Merge pull request #10437 from filecoin-project/release/v1.20.2
release: v1.20.2
2023-03-09 20:58:50 -05:00
Jiaying Wang a1f912c71d Merge pull request #10436 from filecoin-project/jen/v1202
chore: build: v1.20.2 prep
2023-03-09 19:54:08 -05:00
Jennifer Wang 4def0a53e0 change log 2023-03-09 19:39:07 -05:00
Jennifer Wang 51d0d5e549 make gen 2023-03-09 19:24:18 -05:00
Steven Allen b7db4cb280 feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (#10416)
However, we're leaving the default at 1.25x for backwards compatibility, for now.

Also:

1. Actually use the configured replace fee ratio.
2. Store said ratios as percentages instead of floats. 1.25, or 1+1/(2^2),
can be represented as a float. 1.1, or 1 + 1/(2 * 5), cannot.

fixes #10415
2023-03-09 19:20:01 -05:00
Aayush Rajasekaran 36ffa09b05 fix: eth API: return correct txIdx around null blocks (#10419) 2023-03-09 19:17:41 -05:00
ychiao 0e58b3fbbc fix: EthAPI: use StateCompute for feeHistory; apply minimum gas premium (#10413) 2023-03-09 19:16:52 -05:00
Raúl Kripalani 1da2d59066 simplify test. 2023-03-09 19:14:01 -05:00
Raúl Kripalani a72879b2b2 switch to ChainAPI#ChainGetTipSetByHeight. 2023-03-09 19:13:58 -05:00
Raúl Kripalani 2909a31e8f fix TestEthBlockHashesCorrect_MultiBlockTipset: skip null rounds. 2023-03-09 19:13:54 -05:00
Raúl Kripalani 558eaf8b61 fix lint. 2023-03-09 19:13:49 -05:00
Raúl Kripalani 6a7895bf49 fmt. 2023-03-09 19:13:38 -05:00
Raúl Kripalani e73ec088d5 add a test to verify block parameter soundness. 2023-03-09 19:13:34 -05:00
Raúl Kripalani a0821506bc Eth API: fail when requesting future epochs. 2023-03-09 19:13:29 -05:00
Raúl Kripalani 5df2af5189 Eth API: make block parameter parsing sounder. 2023-03-09 19:13:24 -05:00
Arsenii Petrovich 649b0769ba run make gen 2023-03-09 19:13:16 -05:00
Arsenii Petrovich fb569d4c2c add two more methods: StateCall and StateDecodeParams 2023-03-09 19:11:32 -05:00
Arsenii Petrovich 3aac4b45ab add MPoolGetNonce 2023-03-09 19:11:28 -05:00
Arsenii Petrovich 857b0205a8 add method implementation 2023-03-09 19:11:25 -05:00
Arsenii Petrovich b5c88d5714 try to add statenetworkname method 2023-03-09 19:11:20 -05:00
Steven Allen 58900a7033 feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (#10416)
However, we're leaving the default at 1.25x for backwards compatibility, for now.

Also:

1. Actually use the configured replace fee ratio.
2. Store said ratios as percentages instead of floats. 1.25, or 1+1/(2^2),
can be represented as a float. 1.1, or 1 + 1/(2 * 5), cannot.

fixes #10415
2023-03-09 13:17:17 -08:00
Aayush Rajasekaran b75ebcac81 Merge pull request #10423 from filecoin-project/asr/fast-index
feat: chain: make chain tipset fetching 1000x faster
2023-03-09 15:38:55 -05:00
Aayush 90c89288b6 feat: chain: make fetching tipset by height 1000x faster 2023-03-09 15:12:13 -05:00
Travis Person f641139bf2 Merge pull request #10282 from filecoin-project/feat/cache-migrated-stateroot
feat: stmgr: cache migrated stateroots
2023-03-09 11:47:43 -08:00
Jiaying Wang 1096ead73e Merge pull request #10427 from filecoin-project/raulk/fix-eth-parseBlkParam
fix: Eth API: make block parameter parsing sounder.
2023-03-09 14:21:24 -05:00
Raúl Kripalani 3d21d71231 simplify test. 2023-03-09 19:05:31 +00:00
Travis Person b4e589a0f7 review comments 2023-03-09 18:54:29 +00:00
Raúl Kripalani fce0813821 switch to ChainAPI#ChainGetTipSetByHeight. 2023-03-09 18:51:38 +00:00
Aayush Rajasekaran 9fa81673da Merge pull request #10411 from filecoin-project/asr/simplity-new-eth-tx-rct
refactor: EthAPI: Drop unnecessary param from newEthTxReceipt
2023-03-09 12:44:44 -05:00
Raúl Kripalani 13d6211ed0 fix TestEthBlockHashesCorrect_MultiBlockTipset: skip null rounds. 2023-03-09 17:14:23 +00:00
Łukasz Magiera faedc12531 Merge pull request #10391 from filecoin-project/feat/record-hotstore-space
feat:splitstore:Configure max space used by hotstore and GC makes best effort to respect
2023-03-09 17:41:36 +01:00
Łukasz Magiera 366ebe3155 make gen 2023-03-09 17:18:04 +01:00
zenground0 87d5a3723f lint 2023-03-09 08:57:14 -07:00
zenground0 bd0c010be6 Configur buffer and threshold 2023-03-09 08:40:14 -07:00
Raúl Kripalani d38bdcebfd fix lint. 2023-03-09 14:59:56 +00:00
zenground0 aac30cd840 Review Response 2023-03-09 07:57:35 -07:00
ZenGround0andzenground0 c80783dbea feat:splitstore:Splitstore enabled by default (#10429)
Discard mode
---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2023-03-09 08:37:34 -05:00
zenground0 fb7eb08624 lint 2023-03-09 06:36:35 -07:00
Raúl Kripalani 941ef84b71 Merge branch 'master' into raulk/fix-eth-parseBlkParam 2023-03-09 12:16:23 +00:00
Raúl Kripalani fd3ddc860e fmt. 2023-03-09 12:15:35 +00:00
Aayush Rajasekaran ab91ab100f fix: eth API: return correct txIdx around null blocks (#10419) 2023-03-09 12:14:57 +00:00
Łukasz Magiera 3e85591c6c Merge pull request #10390 from beck-8/chore/compute-wdpost-human-readable
chore: make debugging windowPoSt-failures human readable
2023-03-09 11:08:40 +01:00
zenground0 140f2099bd lint 2023-03-08 22:49:30 -07:00
Marten Seemann 4f060b87bb only register metrics once 2023-03-09 16:44:37 +13:00
zenground0 14af4b27cd docsgen-cli 2023-03-08 18:41:31 -07:00
zenground0 0b0913f2f3 lint 2023-03-08 18:40:23 -07:00
zenground0 7931414735 Lint 2023-03-08 18:20:27 -07:00
zenground0 efbb63582e Review Response 2023-03-08 18:11:39 -07:00
zenground0 ee9ff563d2 computing szPurge leads to deadlock and unneeded, remove 2023-03-08 17:51:59 -07:00
zenground0 e5553554d1 cid key size 2023-03-08 17:30:42 -07:00
Łukasz Magiera 80ccd14447 Merge pull request #10356 from filecoin-project/feat/assigner-experiments
feat: sched: Assigner experiments
2023-03-09 01:28:31 +01:00
ychiao 0a0df61d77 fix: EthAPI: use StateCompute for feeHistory; apply minimum gas premium (#10413) 2023-03-09 00:28:17 +00:00
Łukasz Magiera f9c8cb044e Merge pull request #10422 from ArseniiPetrovich/patch-1
docs: API Gateway: patch documentation note about make gen command
2023-03-09 01:25:18 +01:00
Arsenii Petrovich 4427986ac1 replace make all with make deps according to review 2023-03-09 02:10:22 +02:00
Marten Seemann 0f5f91c094 register rcmgr metrics 2023-03-09 11:52:53 +13:00
Łukasz Magiera 1aac945bf6 Merge pull request #10420 from protofire/add_missing_methods_master
fix: Lotus Gateway: Add missing methods - master
2023-03-08 22:59:28 +01:00
Raúl Kripalani b6dc0d2002 add a test to verify block parameter soundness. 2023-03-08 21:15:07 +00:00
Raúl Kripalani 28d8b4cd65 Eth API: fail when requesting future epochs. 2023-03-08 20:49:36 +00:00
Raúl Kripalani cef416e2b0 Eth API: make block parameter parsing sounder. 2023-03-08 20:38:09 +00:00
Łukasz Magiera 52ae215104 Merge pull request #10363 from filecoin-project/fix/fsm-shutdown-removed-sectors
fix: fsm: shutdown removed sectors FSMs
2023-03-08 19:32:47 +01:00
Łukasz Magiera dcfbf7d0a6 fix TestPrintGroupInfo 2023-03-08 18:13:58 +01:00
Arsenii Petrovich a450e74954 small doc patch on how to make gen after api changes 2023-03-08 18:55:51 +02:00
Arsenii Petrovich 0ccef4e5c0 run make gen 2023-03-08 18:53:19 +02:00
zenground0 efbc0ff12e Fix up approximation and logging 2023-03-08 09:43:37 -07:00
Łukasz Magiera d114d8fe7e deprecate MarketListRetrievalDeals, make gen 2023-03-08 17:15:16 +01:00
zenground0 e17ec37d10 fix logging 2023-03-08 09:13:00 -07:00
Arsenii Petrovich fb5c24b2b5 add two more methods: StateCall and StateDecodeParams 2023-03-08 17:49:12 +02:00
Arsenii Petrovich 17ca3a1ac9 add MPoolGetNonce 2023-03-08 17:44:28 +02:00
Arsenii Petrovich b8aaec6c37 add method implementation 2023-03-08 17:44:04 +02:00
Arsenii Petrovich b67e6db527 try to add statenetworkname method 2023-03-08 17:43:48 +02:00
Łukasz Magiera 2c6d025ab3 un-un-update filecoin-ffi 2023-03-08 15:47:09 +01:00
Łukasz Magiera 9038e83639 Merge pull request #10406 from filecoin-project/fix/rpcenc-falky-test
fix: rpcenc: deflake TestReaderRedirectDrop
2023-03-08 10:29:38 +01:00
Jiaying Wang a0a1d371a2 Update service_developer_bug_report.yml 2023-03-08 00:12:18 -05:00
Jiaying Wang 925a385200 Update service_developer_bug_report.yml 2023-03-08 00:10:41 -05:00
Jiaying Wang 2a857275dd Merge pull request #10321 from filecoin-project/chore/service-dev-issue-template
chore: github: Service-provider/dev bug template
2023-03-08 00:08:06 -05:00
Jiaying Wang e5055c25b0 Update service_developer_bug_report.yml 2023-03-08 00:07:48 -05:00
Jiaying Wang a896d88b51 Update .github/ISSUE_TEMPLATE/service_developer_bug_report.yml 2023-03-08 00:06:41 -05:00
Jiaying Wang 04a0fddb95 Update .github/ISSUE_TEMPLATE/service_developer_bug_report.yml 2023-03-08 00:06:36 -05:00
Jiaying Wang 113d49ae5b Merge pull request #10291 from filecoin-project/chore/update-feature-enhancements
chore: github: update enhancement and feature templates
2023-03-08 00:02:39 -05:00
Jiaying Wang decfb32b2b Merge pull request #10289 from filecoin-project/chore/update-bug-template
chore: github: Update bug_report template
2023-03-08 00:02:19 -05:00
Jiaying Wang 52197cc912 Merge pull request #10268 from filecoin-project/add-discussion-fip-links
fix: github: Discussion and FIP links in `New Issue`
2023-03-08 00:00:49 -05:00
Marten Seemann 5b5ea083ce fix drand test 2023-03-08 17:50:58 +13:00
Marten Seemann db6efe6512 use go-libipfs/files instead of ipfs/go-ipfs-files 2023-03-08 17:50:56 +13:00
Marten Seemann b77f2f09ab update minimum Go version to 1.19.7 2023-03-08 14:29:08 +13:00
hannahhoward 7bc6185077 chore(deps): update to get to compile 2023-03-07 11:45:30 -08:00
hannahhoward c3198d7446 chore(deps): update go-libipfs 2023-03-07 11:44:32 -08:00
hannahhoward 16dad6342c feat(deps): update to go-data-transfer v2, still wip 2023-03-07 11:44:29 -08:00
Aayush f1b1eb8b36 refactor: EthAPI: Drop unnecessary param from newEthTxReceipt 2023-03-07 13:40:39 -05:00
Aayush 8b5bc45fcf fix: eth API: correct gateway restrictions, drop unimplemented methods 2023-03-07 12:53:04 -05:00
Łukasz Magiera ec5fde466a rpcenc: Propagate closeOnce in beginPost 2023-03-07 16:50:03 +01:00
Łukasz Magiera f2a652f29f rpcenc: More reliably failing TestReaderRedirectDrop 2023-03-07 16:48:09 +01:00
zenground0 bf29d41993 lint 2023-03-07 08:21:15 -07:00
zenground0 a994153e27 GC respects target for max hotstore space 2023-03-07 07:39:17 -07:00
zenground0 0fe91846cd Plan out moving GC limiting 2023-03-07 07:39:17 -07:00
zenground0 5d9739a863 Track size of dags relevant to compaction 2023-03-07 07:39:17 -07:00
zenground0 dd998d6b24 Begin account for size during walks 2023-03-07 07:39:17 -07:00
Łukasz Magiera 64b9b532c0 Merge pull request #10387 from filecoin-project/feat/lotus-badger-gc
feat:splitstore:Badger GC of hotstore command
2023-03-07 08:52:15 +01:00
Jiaying Wang 8a510e68e0 Merge pull request #10398 from filecoin-project/release/v1.20.1
build:release: v1.20.1
2023-03-06 22:15:12 -05:00
Jiaying Wang 144906ff51 Merge pull request #10396 from filecoin-project/jen/1201
build: v1.20.1 prep
2023-03-06 16:40:55 -05:00
Aayush 202cd41eee update changelog 2023-03-06 16:02:23 -05:00
Aayush 54e7fa8f70 fix: state: short-circuit genesis state computation 2023-03-06 16:01:21 -05:00
Aayush Rajasekaran b88ecb7896 Merge pull request #10397 from filecoin-project/asr/fix-genesis-state-compute
fix: state: short-circuit genesis state computation
2023-03-06 15:57:28 -05:00
Aayush Rajasekaran 6ecf9aaf49 Merge pull request #10395 from filecoin-project/deps/jsonrpc-v0.2.2
deps: Update go-jsonrpc to v0.2.2
2023-03-06 15:44:00 -05:00
Aayush 74d94af034 fix: state: short-circuit genesis state computation 2023-03-06 15:39:28 -05:00
Jennifer Wang c3e1d26ddf v1.20.1 prep 2023-03-06 15:29:03 -05:00
Aayush a59e1d5220 fix: EthAPI: Correctly get parent hash 2023-03-06 15:22:42 -05:00
Aayush 1631c6ac8d fix: EthAPI: Make newEthBlockFromFilecoinTipSet faster and correct 2023-03-06 15:22:28 -05:00
zenground0 a2d331536b Timing info and updated docs 2023-03-06 12:05:04 -07:00
zenground0 5534755f2d Add command to trigger moving GC of hotstore manually 2023-03-06 11:59:08 -07:00
Łukasz Magiera 53fb6adfd6 deps: Update go-jsonrpc to v0.2.2 2023-03-06 18:19:02 +01:00
zenground0 394a7dbb34 Unbump api version as its not necessary 2023-03-06 08:37:28 -07:00
Aayush Rajasekaran fea7dfd212 Merge pull request #10394 from filecoin-project/fix/disabled-post-worker-handling
fix: wdpost: disabled post worker handling
2023-03-06 10:22:46 -05:00
Aayush Rajasekaran 12496e5631 Merge pull request #10389 from filecoin-project/asr/fix-eth-api-genesis
fix: EthAPI: Correctly get parent hash
2023-03-06 09:15:48 -05:00
Łukasz Magiera 9c2f8ee995 itests: Wait for both workers in TestWindowPostWorkerDisconnected 2023-03-06 15:11:21 +01:00
Łukasz Magiera b0ebdb6882 make gen 2023-03-06 15:07:49 +01:00
Łukasz Magiera 9a295e58b0 post worker sched: Retry on alternative worker on RPC errors 2023-03-06 14:30:58 +01:00
Łukasz Magiera 4b99472b35 itests: Test PoSt worker RPC error handling 2023-03-06 14:30:22 +01:00
Łukasz Magiera e170487faf post worker sched: Filter out disabled workers correctly 2023-03-06 13:56:23 +01:00
beck f0ba0ebb64 indent output 2023-03-06 18:04:55 +08:00
zenground0 c81a51af5b make gen 2023-03-05 18:48:02 -07:00
Phi-rjan 6c0a2feade Merge pull request #10384 from filecoin-project/fix/hide-set-worker-cmd
fix: cli: Hide `lotus-worker set` command
2023-03-05 12:30:51 +01:00
beck 668b24fb23 make debugging windowPoSt-failures human readable 2023-03-04 15:00:44 +08:00
Jiaying Wang d3dfb5ca62 Merge pull request #10380 from filecoin-project/asr/fix-eth-api
fix: EthAPI: Make newEthBlockFromFilecoinTipSet faster and correct
2023-03-03 21:10:20 -05:00
Aayush 920fdfdf85 fix: EthAPI: Correctly get parent hash 2023-03-03 16:42:57 -05:00
zenground0 649bcad1bb remove accidental add 2023-03-03 11:23:28 -05:00
zenground0 71b21db0d9 chain prune hot -- hotstore online gc 2023-03-03 11:14:52 -05:00
Phi 51b645f8d0 Hide lotus-worker set command
Hide the `lotus-worker set` command and print that this command will be deprecated.
2023-03-03 14:13:35 +01:00
Aayush Rajasekaran 4588523de7 Merge pull request #10328 from filecoin-project/asr/fip-0045-tooling
feat: update renew-sectors with FIP-0045 logic
2023-03-02 16:53:35 -05:00
Aayush 4e57626e42 feat: update renew-sectors with FIP-0045 logic 2023-03-02 16:38:05 -05:00
Aayush cfb5eac015 fix: EthAPI: Make newEthBlockFromFilecoinTipSet faster and correct 2023-03-02 12:34:09 -05:00
Steven Allen dbbcf4b2ee feat: vm: switch to the new exec trace format (#10372)
This is now "FVM" native. Changes include:

1. Don't treat "trace" messages like off-chain messages. E.g., don't
include CIDs, versions, etc.
2. Include IPLD codecs where applicable.
3. Remove fields that aren't filled by the FVM (timing, some errors,
code locations, etc.).
2023-03-01 16:02:18 -08:00
Travis Person 814c146626 feat: stmgr: cache migrated stateroots 2023-03-01 16:17:34 +00:00
Łukasz Magiera c484c38735 worker sched: Separate resource def for preparing window 2023-02-28 10:52:50 +01:00
Łukasz Magiera 04fe9062a6 fix: fsm: shutdown removed sectors FSMs 2023-02-28 09:48:17 +01:00
Łukasz Magiera 2316363f7a sched: Share active/preparing task counters 2023-02-28 09:08:49 +01:00
Łukasz Magiera 745476c9ab feat: sched: Assigner experiments 2023-02-27 17:54:25 +01:00
Phi 03f2efd0f9 Add tooling textbox
Add tooling textbox to template
2023-02-21 08:52:47 +01:00
Phi 7f6fbe22c6 Add initial service/dev bug template
Initial service/dev bug template
2023-02-17 10:33:52 +01:00
Phi 27d4d02fe5 Update enhancement and feature templates
Update enhancement and feature_request templates:

- Removing unnecessary checklist options.
   - These have been move to the Github New Issue UI to reduce text overhead: #10268
- Remove deprecated components options
- Add FVM/FEVM component
2023-02-16 13:11:10 +01:00
Phi d3e715594b chore: github: Update bug_report template
Update bug_report template:

- Removing unnecessary checklist options.
   - Some of these has been move to the Github New Issue UI to reduce overhead
- Remove deprecated components options
- Add FVM/FEVM component
2023-02-16 10:59:45 +01:00
Phi-rjan 3879c80e6f Add link to discussion and FIPs in New Issue
In our current issue flow, we ask a lot checklist questions :

`This is not a question or a support request. If you have any lotus related questions, please ask in the lotus forum.`
`This is not a new feature or an enhancement to the Filecoin protocol. If it is, please open an FIP issue. *`

This information is better displayed in the `New Issue` chooser, to guide users to the relevant place for posting. This file will add two new entries in the issue chooser:
- A entry that links to the Lotus Github Discussion pager for Lotus questions or troubleshooting
- A entry that links to the FIPs repo for Filecoin protocol feature or enhancements
2023-02-14 11:13:55 +01:00
260 changed files with 7008 additions and 4612 deletions
+2 -2
View File
@@ -7,12 +7,12 @@ executors:
golang:
docker:
# Must match GO_VERSION_MIN in project root
- image: cimg/go:1.18.8
- image: cimg/go:1.19.7
resource_class: medium+
golang-2xl:
docker:
# Must match GO_VERSION_MIN in project root
- image: cimg/go:1.18.8
- image: cimg/go:1.19.7
resource_class: 2xlarge
ubuntu:
docker:
+2 -2
View File
@@ -7,12 +7,12 @@ executors:
golang:
docker:
# Must match GO_VERSION_MIN in project root
- image: cimg/go:1.18.8
- image: cimg/go:1.19.7
resource_class: medium+
golang-2xl:
docker:
# Must match GO_VERSION_MIN in project root
- image: cimg/go:1.18.8
- image: cimg/go:1.19.7
resource_class: 2xlarge
ubuntu:
docker:
+20 -35
View File
@@ -9,15 +9,9 @@ body:
options:
- label: This is **not** a security-related bug/issue. If it is, please follow please follow the [security policy](https://github.com/filecoin-project/lotus/security/policy).
required: true
- label: This is **not** a question or a support request. If you have any lotus related questions, please ask in the [lotus forum](https://github.com/filecoin-project/lotus/discussions).
required: true
- label: This is **not** a new feature request. If it is, please file a [feature request](https://github.com/filecoin-project/lotus/issues/new?assignees=&labels=need%2Ftriage%2Ckind%2Ffeature&template=feature_request.yml) instead.
required: true
- label: This is **not** an enhancement request. If it is, please file a [improvement suggestion](https://github.com/filecoin-project/lotus/issues/new?assignees=&labels=need%2Ftriage%2Ckind%2Fenhancement&template=enhancement.yml) instead.
required: true
- label: I **have** searched on the [issue tracker](https://github.com/filecoin-project/lotus/issues) and the [lotus forum](https://github.com/filecoin-project/lotus/discussions), and there is no existing related issue or discussion.
required: true
- label: I am running the [`Latest release`](https://github.com/filecoin-project/lotus/releases), or the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
- label: I am running the [`Latest release`](https://github.com/filecoin-project/lotus/releases), the most recent RC(release canadiate) for the upcoming release or the dev branch(master), or have an issue updating to any of these.
required: true
- label: I did not make any code changes to lotus.
required: false
@@ -28,19 +22,11 @@ body:
options:
- label: lotus daemon - chain sync
required: false
- label: lotus miner - mining and block production
- label: lotus fvm/fevm - Lotus FVM and FEVM interactions
required: false
- label: lotus miner/worker - sealing
required: false
- label: lotus miner - proving(WindowPoSt)
required: false
- label: lotus miner/market - storage deal
required: false
- label: lotus miner/market - retrieval deal
required: false
- label: lotus miner/market - data transfer
required: false
- label: lotus client
- label: lotus miner - proving(WindowPoSt/WinningPoSt)
required: false
- label: lotus JSON-RPC API
required: false
@@ -56,22 +42,33 @@ body:
description: Enter the output of `lotus version` and `lotus-miner version` if applicable.
placeholder: |
e.g.
Daemon:1.11.0-rc2+debug+git.0519cd371.dirty+api1.3.0
Local: lotus version 1.11.0-rc2+debug+git.0519cd371.dirty
Daemon: 1.19.0+mainnet+git.64059ca87+api1.5.0
Local: lotus-miner version 1.19.0+mainnet+git.64059ca87
validations:
required: true
- type: textarea
id: ReproSteps
attributes:
label: Repro Steps
description: "Steps to reproduce the behavior"
value: |
1. Run '...'
2. Do '...'
3. See error '...'
...
validations:
required: false
- type: textarea
id: Description
attributes:
label: Describe the Bug
description: |
This is where you get to tell us what went wrong, when doing so, please try to provide a clear and concise description of the bug with all related information:
* What you were doding when you experienced the bug?
* What you were doing when you experienced the bug?
* Any *error* messages you saw, *where* you saw them, and what you believe may have caused them (if you have any ideas).
* What is the expected behaviour?
* For sealing issues, include the output of `lotus-miner sectors status --log <sectorId>` for the failed sector(s).
* For proving issues, include the output of `lotus-miner proving` info.
* For deal making issues, include the output of `lotus client list-deals -v` and/or `lotus-miner storage-deals|retrieval-deals|data-transfers list [-v]` commands for the deal(s) in question.
validations:
required: true
- type: textarea
@@ -83,18 +80,6 @@ body:
Please provide debug logs of the problem, remember you can get set log level control for:
* lotus: use `lotus log list` to get all log systems available and set level by `lotus log set-level`. An example can be found [here](https://lotus.filecoin.io/lotus/configure/defaults/#log-level-control).
* lotus-miner:`lotus-miner log list` to get all log systems available and set level by `lotus-miner log set-level
If you don't provide detailed logs when you raise the issue it will almost certainly be the first request I make before furthur diagnosing the problem.
If you don't provide detailed logs when you raise the issue it will almost certainly be the first request we make before furthur diagnosing the problem.
validations:
required: true
- type: textarea
id: RepoSteps
attributes:
label: Repo Steps
description: "Steps to reproduce the behavior"
value: |
1. Run '...'
2. Do '...'
3. See error '...'
...
validations:
required: false
required: true
+8
View File
@@ -0,0 +1,8 @@
blank_issues_enabled: true
contact_links:
- name: Ask a question about Lotus or get support
url: https://github.com/filecoin-project/lotus/discussions/new/choose
about: Ask a question or request support for using Lotus
- name: Filecoin protocol feature or enhancement
url: https://github.com/filecoin-project/FIPs/discussions/new/choose
about: Write a discussion in the Filecoin Improvement Proposal repo
+14 -20
View File
@@ -7,13 +7,7 @@ body:
label: Checklist
description: Please check off the following boxes before continuing to create an improvement suggestion!
options:
- label: This is **not** a new feature or an enhancement to the Filecoin protocol. If it is, please open an [FIP issue](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0001.md).
required: true
- label: This is **not** a new feature request. If it is, please file a [feature request](https://github.com/filecoin-project/lotus/issues/new?assignees=&labels=need%2Ftriage%2Ckind%2Ffeature&template=feature_request.yml) instead.
required: true
- label: This is **not** brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on [the lotus forum](https://github.com/filecoin-project/lotus/discussions/categories/ideas) and select the category as `Ideas`.
required: true
- label: I **have** a specific, actionable, and well motivated improvement to propose.
- label: I **have** a specific, actionable, and well motivated improvement to an existing lotus feature.
required: true
- type: checkboxes
attributes:
@@ -22,19 +16,11 @@ body:
options:
- label: lotus daemon - chain sync
required: false
- label: lotus miner - mining and block production
- label: lotus fvm/fevm - Lotus FVM and FEVM interactions
required: false
- label: lotus miner/worker - sealing
required: false
- label: lotus miner - proving(WindowPoSt)
required: false
- label: lotus miner/market - storage deal
required: false
- label: lotus miner/market - retrieval deal
required: false
- label: lotus miner/market - data transfer
required: false
- label: lotus client
- label: lotus miner - proving(WindowPoSt/WinningPoSt)
required: false
- label: lotus JSON-RPC API
required: false
@@ -45,9 +31,17 @@ body:
- type: textarea
id: request
attributes:
label: Improvement Suggestion
description: A clear and concise description of what the motivation or the current problem is and what is the suggested improvement?
placeholder: Ex. Currently lotus... However, as a storage provider, I'd like...
label: Enhancement Suggestion
description: A clear and concise description of the suggested enhancement?
placeholder: Ex. Currently lotus... However it would be great if [enhancement] was implemented... With the ability to...
validations:
required: true
- type: textarea
id: request
attributes:
label: Use-Case
description: How would this enhancement help you?
placeholder: Ex. With the [enhancement] node operators would be able to... For Storage Providers it would enable...
validations:
required: true
+3 -14
View File
@@ -7,8 +7,6 @@ body:
label: Checklist
description: Please check off the following boxes before continuing to create a new feature request!
options:
- label: This is **not** a new feature or an enhancement to the Filecoin protocol. If it is, please open an [FIP issue](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0001.md).
required: true
- label: This is **not** brainstorming ideas. If you have an idea you'd like to discuss, please open a new discussion on [the lotus forum](https://github.com/filecoin-project/lotus/discussions/categories/ideas) and select the category as `Ideas`.
required: true
- label: I **have** a specific, actionable, and well motivated feature request to propose.
@@ -20,19 +18,11 @@ body:
options:
- label: lotus daemon - chain sync
required: false
- label: lotus miner - mining and block production
- label: lotus fvm/fevm - Lotus FVM and FEVM interactions
required: false
- label: lotus miner/worker - sealing
required: false
- label: lotus miner - proving(WindowPoSt)
required: false
- label: lotus miner/market - storage deal
required: false
- label: lotus miner/market - retrieval deal
required: false
- label: lotus miner/market - data transfer
required: false
- label: lotus client
- label: lotus miner - proving(WindowPoSt/WinningPoSt)
required: false
- label: lotus JSON-RPC API
required: false
@@ -56,7 +46,7 @@ body:
validations:
required: true
- type: textarea
id: alternates
id: alternatives
attributes:
label: Describe alternatives you've considered
description: A clear and concise description of any alternative solutions or features you've considered.
@@ -69,4 +59,3 @@ body:
description: Add any other context, design docs or screenshots about the feature request here.
validations:
required: false
@@ -0,0 +1,83 @@
name: "Bug Report - developer/service provider"
description: "Bug report template about FEVM/FVM for developers/service providers"
labels: [need/triage, kind/bug, area/fevm]
body:
- type: checkboxes
attributes:
label: Checklist
description: Please check off the following boxes before continuing to file a bug report!
options:
- label: This is **not** a security-related bug/issue. If it is, please follow please follow the [security policy](https://github.com/filecoin-project/lotus/security/policy).
required: true
- label: I **have** searched on the [issue tracker](https://github.com/filecoin-project/lotus/issues) and the [lotus forum](https://github.com/filecoin-project/lotus/discussions), and there is no existing related issue or discussion.
required: true
- label: I did not make any code changes to lotus.
required: false
- type: checkboxes
attributes:
label: Lotus component
description: Please select the lotus component you are filing a bug for
options:
- label: lotus Ethereum RPC
required: false
- label: lotus FVM - Lotus FVM interactions
required: false
- label: FEVM tooling
required: false
- label: Other
required: false
- type: textarea
id: version
attributes:
label: Lotus Version
render: text
description: Enter the output of `lotus version` if applicable.
placeholder: |
e.g.
Daemon: 1.19.0+mainnet+git.64059ca87+api1.5.0
Local: lotus-miner version 1.19.0+mainnet+git.64059ca87
validations:
required: true
- type: textarea
id: repro
attributes:
label: Repro Steps
description: "Steps to reproduce the behavior"
value: |
1. Run '...'
2. Do '...'
3. See error '...'
...
validations:
required: false
- type: textarea
id: Description
attributes:
label: Describe the Bug
description: |
This is where you get to tell us what went wrong, when doing so, please try to provide a clear and concise description of the bug with all related information:
* What you were doing when you experienced the bug? What are you trying to build?
* Any *error* messages and logs you saw, *where* you saw them, and what you believe may have caused them (if you have any ideas).
* What is the expected behaviour? Links to the actual code?
validations:
required: true
- type: textarea
id: toolingInfo
attributes:
label: Tooling
render: text
description: |
What kind of tooling are you using:
* Are you using ether.js, Alchemy, Hardhat, etc.
validations:
required: true
- type: textarea
id: extraInfo
attributes:
label: Configuration Options
render: text
description: |
Please provide your updated FEVM related configuration options, or custome enviroment variables related to Lotus FEVM
* lotus: use `lotus config updated` to get your configuration options, and copy the [FEVM] section
validations:
required: true
+482
View File
@@ -1,5 +1,487 @@
# Lotus changelog
# v1.21.0-rc3 / 2023-04-10
This is an optional but highly recommended feature release of Lotus. It includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.
## ☢️ Upgrade Warnings ☢️
Before upgrading to this feature release read carefully through these bullet points:
- Starting from this release, the SplitStore feature is automatically activated on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the `EnableSplitstore` option in your `config.toml` file. To enable SplitStore, set `EnableSplitstore=true`, and to disable it, set `EnableSplitstore=false`. **It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!**
- This feature release requires a **minimum Go version of v1.19.7 or higher to successfully build Lotus**. Additionally, Go version v1.20 and higher is now also supported.
- **Storage Providers:** The proofs libraries now have CUDA enabled by default, which requires you to install (CUDA)[https://lotus.filecoin.io/tutorials/lotus-miner/cuda/] if you haven't already done so. If you prefer to use OpenCL on your GPUs instead, you can use the `FFI_USE_OPENCL=1` flag when building from source. On the other hand, if you want to disable GPUs altogether, you can use the `FFI_NO_GPU=1` environment variable when building from source.
- **Storage Providers:** The `lotus-miner sectors extend` command has been refactored to the functionality of `lotus-miner sectors renew`.
- **Exchanges/Node operators/RPC-providers::** Execution traces (returned from `lotus state exec-trace`, `lotus state replay`, etc.), has changed to account for changes introduced by the by the FVM. **Please make sure to read the `Execution trace format change` section carefully, as these are interface breaking changes**
- **Syncing issues:** If you have been struggling with syncing issues in normal operations you can try to adjust the amount of threads used for more concurrent FMV execution through via the `LOTUS_FVM_CONCURRENCY` enviroment variable. It is set to 4 threads by default. Recommended formula for concurrency == YOUR_RAM/4 , but max is 128. If you are a Storage Provider and are pushing many messages within a short period of time, exporting `LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS=1` will also help with keeping in sync.
- **Catching up from a Snapshot:** Users have noticed that catching up sync from a snapshot is taking a lot longer these day. This is largely related to the built-in market actor consuming a lot of computational demand for block validation. A FIP for a short-term mitigation for this is currently in Last Call and will be included network version 19 upgrade if accepted. You [can read the FIP here.](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0060.md)
## Highlights
**SplitStore**
This feature release introduces numerous improvements and fixes to tackle SplitStore related issues that has been reported. With this feature release SplitStore is automatically activated by default on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the `EnableSplitstore` option in your `config.toml` file. To enable SplitStore, set `EnableSplitstore=true`, and to disable it, set `EnableSplitstore=false`. **It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!**
SplitStore also has some new configuration settings that you can set in your config.toml file:
- `HotstoreMaxSpaceTarget` suggests the max allowed space (in bytes) the hotstore can take.
- `HotstoreMaxSpaceThreshold` a moving GC will be triggered when total moving size exceeds this threshold (in bytes).
- `HotstoreMaxSpaceSafetyBuffer` a safety buffer to prevent moving GC from an overflowing disk.
The SplitStore also has two new commands:
- `lotus chain prune hot` is a much less resource-intensive GC and is best suited for situations where you don't have the spare disk space for a full GC.
- `lotus chain prune hot-moving` will run a full moving garbage collection of the hotstore. This commands create a new hotstore before deleting the old one so you need working room in the hotstore directory. The current size of a fully GC'd hotstore is around 295 GiB so you need to make sure you have at least that available.
You can read more about the new SplitStore commands in [the documentation](https://lotus.filecoin.io/lotus/configure/splitstore/#manual-chain-store-garbage-collection).
**RPC API improvements**
This feature release includes all the RPC API improvements made in the Lotus v1.20.x patch releases. It includes an updated FFI that sets the FVM parallelism to 4 by default.
Node operators with higher memory specs can experiment with setting LOTUS_FVM_CONCURRENCY to higher values, up to 48, to allow for more concurrent FVM execution.
**Experimental scheduler assigners**
In this release there are four new expirmental scheduler assigners:
- The `experiment-spread-qcount` - similar to the spread assigner but also takes into account task counts which are in running/preparing/queued states.
- The `experiment-spread-tasks` - similar to the spread assigner, but counts running tasks on a per-task-type basis
- The `experiment-spread-tasks-qcount` - similar to the spread assigner, but also takes into account task counts which are in running/preparing/queued states, as well as counting running tasks on a per-task-type basis. Check the results for this assigner on ([storage-only lotus-workers here](https://github.com/filecoin-project/lotus/issues/8566#issuecomment-1446978856)).
- The `experiment-random` - In each schedule loop the assinger figures a set of all workers which can handle the task and then picks a random one. Check the results for this assigner on ([storage-only lotus-workers here](https://github.com/filecoin-project/lotus/issues/8566#issuecomment-1447064218)).
**Graceful shutdown of lotus-workers**
We have cleaned up some commands in the `lotus-worker` to make it less confusing how to gracefully shutting down a `lotus-worker` while there are incoming sealing tasks in the pipeline. To shut down a `lotus-worker` gracefully:
1. `lotus-worker tasks disable --all` and wait for the worker to finish processing its current tasks.
2. `lotus-worker stop` to detach it and do maintenance/upgrades.
**CLI speedups**
The `lotus-miner sector list` is now running in parallel - which should speed up the process from anywhere between 2x-10x+. You can tune it additionally with the `check-parallelism` option in the command. The `Lotus-Miner info` command also has a large speed improvement, as calls to the lotus legacy market has been removed.
## New features
- feat: splitstore: Pause compaction when out of sync ([filecoin-project/lotus/#10641](https://github.com/filecoin-project/lotus/pull/10641))
- Pause the SplitStore compaction if the node is out of sync. Resumes the compation when its back in sync.
- feat: splitstore: limit moving gc threads (#10621) ([filecoin-project/lotus/#10621](https://github.com/filecoin-project/lotus/pull/10621))
- Makes moving gc less likely to cause node falling out of sync.
- feat: splitstore: Update config default value (#10605) ([filecoin-project/lotus/#10605](https://github.com/filecoin-project/lotus/pull/10605))
- Sets Splitstore HotStoreMaxSpaceTarget config to 650GB as default
- feat: splitstore: Splitstore enabled by default (#10429) ([filecoin-project/lotus#10429](https://github.com/filecoin-project/lotus/pull/10429))
- Enables SplitStore by default on new Lotus nodes. Existing Lotus users need to explicitly configure
- feat: splitstore: Configure max space used by hotstore and GC makes best effort to respect ([filecoin-project/lotus#10391](https://github.com/filecoin-project/lotus/pull/10391))
- Adds three new configs for setting the maximum allowed space the hotstore can take.
- feat: splitstore: Badger GC of hotstore command ([filecoin-project/lotus#10387](https://github.com/filecoin-project/lotus/pull/10387))
- Adds a `lotus chain prune hot` command, to run the garbage collection of the hotstore in a user driven way.
- feat: sched: Assigner experiments ([filecoin-project/lotus#10356](https://github.com/filecoin-project/lotus/pull/10356))
- Introduces experimental scheduler assigners that works better for setups that uses storage-only lotus-workers.
- fix: wdpost: disabled post worker handling ([filecoin-project/lotus#10394](https://github.com/filecoin-project/lotus/pull/10394))
- Improved scheduling logic for Proof-of-SpaceTime workers.
- feat: cli: list claims and remove expired claims ([filecoin-project/lotus#9875](https://github.com/filecoin-project/lotus/pull/9875))
- Adds a command to list claims made by a provider `lotus filplus list-claims`. And `lotus filplus remove-expired-claims` to remove expired claims.
- feat: cli: make sectors list much faster ([filecoin-project/lotus#10202](https://github.com/filecoin-project/lotus/pull/10202))
- Makes `lotus-miner sector list` checks run in parallel.
- feat: cli: Add an EVM command to fetch a contract's bytecode ([filecoin-project/lotus#10443](https://github.com/filecoin-project/lotus/pull/10443))
- Adds an `lotus evm bytecode` command to fetch a contract's bytecode.
- feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (#10416) ([filecoin-project/lotus#10416](https://github.com/filecoin-project/lotus/pull/10416))
- Reduces replacement message fee logic to help include update message replacements from developers using Ethereum tools like MetaMask.
- feat: update renew-sectors with FIP-0045 logic ([filecoin-project/lotus#10328](https://github.com/filecoin-project/lotus/pull/10328))
- Updates the `lotus-miner sectors extend` with FIP-0045 logic to include the ability to drop claims and set the maximum number of messages contained in a message.
- feat: IPC: Abstract common consensus functions and consensus interface ([filecoin-project/lotus#9481](https://github.com/filecoin-project/lotus/pull/9481))
- Add eudico's consensus interface to Lotus and implement EC behind that interface. This abstraction is the stepping-stone for Mir's integration.
- fix: worker: add all tasks flag ([filecoin-project/lotus#10232](https://github.com/filecoin-project/lotus/pull/10232))
- Adds an `all` flag for the `lotus-worker tasks enable/disable` cmds.
- feat:shed:add cid to cbor serialization command ([filecoin-project/lotus#10032](https://github.com/filecoin-project/lotus/pull/10032))
- Adds two `lotus-shed` commands, `lotus-shed cid bytes` and `lotus-shed cid cbor` to serialize cid to cbor and cid to bytes.
- feat: add toolshed commands to inspect statetree size ([filecoin-project/lotus#9982](https://github.com/filecoin-project/lotus/pull/9982))
- Adds two commands, `lotus-shed stat-actor` and `lotus-shed stat-obj` that work with an offline lotus repo to report dag size stats.
- feat: shed: encode address to bytes ([filecoin-project/lotus#10105](https://github.com/filecoin-project/lotus/pull/10105))
- Adds a `lotus-shed address encode` for encoding a filecoin address to hex bytes.
- feat: chain: export-range ([filecoin-project/lotus#10145](https://github.com/filecoin-project/lotus/pull/10145))
- Adds a `lotus chain export-range` command that can create archival-grade ranged exports of the chain as quickly as possible.
- feat: stmgr: cache migrated stateroots ([filecoin-project/lotus#10282](https://github.com/filecoin-project/lotus/pull/10282))
- Cache network migration results to avoid running migrations twice.
- feat: shed: Add a tool to read data from sectors ([filecoin-project/lotus#10169](https://github.com/filecoin-project/lotus/pull/10169))
- Adds a lotus-shed sectors read command that extract data from sectors from a running lotus-miner deployment.
- feat: cli: Refactor renew and remove extend ([filecoin-project/lotus#9920](https://github.com/filecoin-project/lotus/pull/9920))
- Refactors the `lotus-miner sectors extend` command to have the functionality of `lotus-miner sectors renew`. The `lotus-miner sectors renew` command has been deprecated.
- feat: shed: Add beneficiary commands ([filecoin-project/lotus#10037](https://github.com/filecoin-project/lotus/pull/10037))
- Adds the beneficiary address command to `lotus-shed`. You can now use `lotus-shed actor propose-change-beneficiary` and `lotus-shed actor confirm-change-beneficiary` to change beneficiary addresses.
## Improvements
- backport: fix: miner: correctly count sector extensions (10555) ([filecoin-project/lotus#10555](https://github.com/filecoin-project/lotus/pull/10555))
- Fixes the issue with sector extensions.
- fix: proving: Initialize slice with with same length as partition (#10574) ([filecoin-project/lotus#10574])(https://github.com/filecoin-project/lotus/pull/10574)
- Fixes an issue where `lotus-miner proving compute window-post` paniced when trying to make skipped sectors human readable.
- feat: stmgr: speed up calculation of genesis circ supply (#10553) ([filecoin-project/lotus#10553])(https://github.com/filecoin-project/lotus/pull/10553)
- perf: eth: gas estimate set applyTsMessages false (#10546) ([filecoin-project/lotus#10456](https://github.com/filecoin-project/lotus/pull/10546))
- feat: config: Force existing users to opt into new defaults (#10488) ([filecoin-project/lotus#10488](https://github.com/filecoin-project/lotus/pull/10488))
- Force existing users to opt into the new SplitStore defaults.
- fix: splitstore: Demote now common logs (#10516) ([filecoin-project/lotus#10516](https://github.com/filecoin-project/lotus/pull/10516))
- fix: splitstore: Don't enforce walking receipt tree during compaction ([filecoin-project/lotus#10502](https://github.com/filecoin-project/lotus/pull/10502))
- fix: splitstore: Fix the overzealous fix (#10366) ([filecoin-project/lotus#10366](https://github.com/filecoin-project/lotus/pull/10366))
- fix: splitstore: Two fixes, better logging and comments (#10332) ([filecoin-project/lotus#10332](https://github.com/filecoin-project/lotus/pull/10332))
- fix: fsm: shutdown removed sectors FSMs ([filecoin-project/lotus#10363](https://github.com/filecoin-project/lotus/pull/10363))
- Fixes an issue where removed sectors still got state machine events.
- fix: rpcenc: Don't hang when source dies ([filecoin-project/lotus#10116](https://github.com/filecoin-project/lotus/pull/10116))
- Fixes an issue where AddPiece tasks could get stuck if the Boost process was abruptly lost.
- fix: make debugging windowPoSt-failures human readable ([filecoin-project/lotus#10390](https://github.com/filecoin-project/lotus/pull/10390))
- Makes the skipped sector list in `lotus-miner proving compute window-post` human readable.
- fix: cli: Hide `lotus-worker set` command ([filecoin-project/lotus#10384](https://github.com/filecoin-project/lotus/pull/10384))
- Hides the `lotus-worker set` command. This command will be deprecated later.
- fix: worker: Hide `wait-quiet` cmd ([filecoin-project/lotus#10331](https://github.com/filecoin-project/lotus/pull/10331))
- Hides the `lotus-worker wait-quiet` command. This command will be deprecated later.
- fix: post: Tune down default post-parallel-reads ([filecoin-project/lotus#10365](https://github.com/filecoin-project/lotus/pull/10365))
- Tuning down the default post-parallel-reads to a more conservative number to prevent sectors from being skipped due to network timeouts.
- fix: cli: error if backup file already exists ([filecoin-project/lotus#10209](https://github.com/filecoin-project/lotus/pull/10209))
- Error out if a backup file with the same name already exists when using the `lotus-miner backup` or `lotus backup` command
- fix: cli: option to set-seal-delay in seconds ([filecoin-project/lotus#10208](https://github.com/filecoin-project/lotus/pull/10208))
- Adds the option to specify `lotus-miner sectors set-seal-delay` in seconds
- fix: cli: extend cmd to get the right sector number ([filecoin-project/lotus#10182](https://github.com/filecoin-project/lotus/pull/10182))
- Making sure the `lotus-miner sectors extend` command gets the correct sector number.
- feat: wdpost: Emit more detailed errors ([filecoin-project/lotus#10121](https://github.com/filecoin-project/lotus/pull/10121))
- Emits more detailed windowPoSt error messages, making it easier to debug PoSt issues.
- fix: Lotus Gateway: Add missing methods - master ([filecoin-project/lotus#10420](https://github.com/filecoin-project/lotus/pull/10420))
- Adds `StateNetworkName`, `MpoolGetNonce`, `StateCall` and `StateDecodeParams` methods to Lotus Gateway.
- fix: stmgr: don't attempt to lookup genesis state (#10472) ([filecoin-project/lotus#10472](https://github.com/filecoin-project/lotus/pull/10472))
- feat: gateway: export StateVerifierStatus ([filecoin-project/lotus#10477](https://github.com/filecoin-project/lotus/pull/10477))
- fix: gateway: correctly apply the fee history lookback max ([filecoin-project/lotus#10464](https://github.com/filecoin-project/lotus/pull/10464))
- fix: gateway: drop overzealous guard on MsigGetVested ([filecoin-project/lotus#10451](https://github.com/filecoin-project/lotus/pull/10451))
- feat: apply gateway lookback limit to eth API lookback ([filecoin-project/lotus#10467](https://github.com/filecoin-project/lotus/pull/10467))
- fix: revert "Eth API: drop support for 'pending' block parameter." ([filecoin-project/lotus#10474](https://github.com/filecoin-project/lotus/pull/10474))
- fix: Eth API: make net_version return the chain ID ([filecoin-project/lotus#10456](https://github.com/filecoin-project/lotus/pull/10456))
- fix: eth: handle a potential divide by zero in receipt handling ([filecoin-project/lotus#10495](https://github.com/filecoin-project/lotus/pull/10495))
- fix: ethrpc: Don't lock up when eth subscriber goes away ([filecoin-project/lotus#10485](https://github.com/filecoin-project/lotus/pull/10485))
- feat: eth: Avoid StateCompute in EthTxnReceipt lookup (#10460) ([filecoin-project/lotus#10460](https://github.com/filecoin-project/lotus/pull/10460))
- feat: eth: optimize eth block loading + eth_feeHistory ([filecoin-project/lotus#10446](https://github.com/filecoin-project/lotus/pull/10446))
- feat: state: skip tipset execution when possible ([filecoin-project/lotus#10445](https://github.com/filecoin-project/lotus/pull/10445))
- feat: eth API: reject masked ID addresses embedded in f410f payloads ([filecoin-project/lotus#10440](https://github.com/filecoin-project/lotus/pull/10440))
- fix: Eth API: make block parameter parsing sounder. ([filecoin-project/lotus#10427](https://github.com/filecoin-project/lotus/pull/10427))
- fix: eth API: return correct txIdx around null blocks (#10419) ([filecoin-project/lotus#10419](https://github.com/filecoin-project/lotus/pull/10419))
- fix: EthAPI: use StateCompute for feeHistory; apply minimum gas premium (#10413) ([filecoin-project/lotus#10413](https://github.com/filecoin-project/lotus/pull/10413))
- refactor: EthAPI: Drop unnecessary param from newEthTxReceipt ([filecoin-project/lotus#10411](https://github.com/filecoin-project/lotus/pull/10411))
- fix: eth API: correct gateway restrictions, drop unimplemented methods ([filecoin-project/lotus#10409](https://github.com/filecoin-project/lotus/pull/10409))
- fix: EthAPI: Correctly get parent hash ([filecoin-project/lotus#10389](https://github.com/filecoin-project/lotus/pull/10389))
- fix: EthAPI: Make newEthBlockFromFilecoinTipSet faster and correct ([filecoin-project/lotus#10380](https://github.com/filecoin-project/lotus/pull/10380))
- fix: eth: incorrect struct tags (#10309) ([filecoin-project/lotus#10309](https://github.com/filecoin-project/lotus/pull/10309))
- refactor: update cache to the new generic version (#10463) ([filecoin-project/lotus#10463](https://github.com/filecoin-project/lotus/pull/10463))
- feat: consensus: log ApplyBlock timing/gas stats ([filecoin-project/lotus#10470](https://github.com/filecoin-project/lotus/pull/10470))
- feat: chain: make chain tipset fetching 1000x faster ([filecoin-project/lotus#10423](https://github.com/filecoin-project/lotus/pull/10423))
- chain: explicitly check that gasLimit is above zero ([filecoin-project/lotus#10198](https://github.com/filecoin-project/lotus/pull/10198))
- feat: blockstore: Envvar can adjust badger compaction worker poolsize ([filecoin-project/lotus#9973](https://github.com/filecoin-project/lotus/pull/9973))
- feat: stmgr: add env to disable premigrations ([filecoin-project/lotus#10283](https://github.com/filecoin-project/lotus/pull/10283))
- chore: Remove legacy market info from lotus-miner info ([filecoin-project/lotus#10364](https://github.com/filecoin-project/lotus/pull/10364))
- Removes the legacy market info in the `Lotus-Miner info`. Speeds up the command significantly.
- chore: blockstore: Plumb through a proper Flush() method on all blockstores ([filecoin-project/lotus#10465](https://github.com/filecoin-project/lotus/pull/10465))
- fix: extend LOTUS_CHAIN_BADGERSTORE_DISABLE_FSYNC to the markset ([filecoin-project/lotus#10172](https://github.com/filecoin-project/lotus/pull/10172))
- feat: vm: switch to the new exec trace format (#10372) ([filecoin-project/lotus#10372](https://github.com/filecoin-project/lotus/pull/10372))
- fix: Remove workaround that is no longer needed ([filecoin-project/lotus#9995](https://github.com/filecoin-project/lotus/pull/9995))
- feat: Check for allocation expiry when waiting to seal sectors ([filecoin-project/lotus#9878](https://github.com/filecoin-project/lotus/pull/9878))
- feat: Allow libp2p user agent to be overriden ([filecoin-project/lotus#10149](https://github.com/filecoin-project/lotus/pull/10149))
- feat: cli: Add global color flag ([filecoin-project/lotus#10022](https://github.com/filecoin-project/lotus/pull/10022))
- fix: should not serve non v0 api in v0 ([filecoin-project/lotus#10066](https://github.com/filecoin-project/lotus/pull/10066))
- fix: build: drop drand incentinet servers ([filecoin-project/lotus#10476](https://github.com/filecoin-project/lotus/pull/10476))
- fix: sealing: stub out the FileSize function on Windows ([filecoin-project/lotus#10035](https://github.com/filecoin-project/lotus/pull/10035))
## Dependencies
- github.com/filecoin-project/go-dagaggregator-unixfs (v0.2.0 -> v0.3.0):
- github.com/filecoin-project/go-fil-markets (v1.25.2 -> v1.27.0-rc1):
- github.com/filecoin-project/go-jsonrpc (v0.2.1 -> v0.2.3):
- github.com/filecoin-project/go-statemachine (v1.0.2 -> v1.0.3):
- github.com/filecoin-project/go-state-types (v0.10.0 -> v0.11.0-alpha-3)
- github.com/ipfs/go-cid (v0.3.2 -> v0.4.0):
- github.com/ipfs/go-libipfs (v0.5.0 -> v0.7.0):
- github.com/ipfs/go-path (v0.3.0 -> v0.3.1):
- chore: deps: update to go-state-types v0.11.0-alpha-3 (([filecoin-project/lotus#10606](https://github.com/filecoin-project/lotus/pull/10606))
- deps: update go-libp2p-pubsub to v0.9.3 ([filecoin-project/lotus#10483](https://github.com/filecoin-project/lotus/pull/10483))
- deps: Update go-jsonrpc to v0.2.2 ([filecoin-project/lotus#10395](https://github.com/filecoin-project/lotus/pull/10395))
- Update to go-data-transfer v2 and libp2p, still wip ([filecoin-project/lotus#10382](https://github.com/filecoin-project/lotus/pull/10382))
- dep: ipld: update ipld prime to v0.20.0 ([filecoin-project/lotus#10247](https://github.com/filecoin-project/lotus/pull/10247))
- chore: node: migrate go-bitswap to go-libipfs/bitswap ([filecoin-project/lotus#10138](https://github.com/filecoin-project/lotus/pull/10138))
- chore: all: bump go-libipfs to replace go-block-format ([filecoin-project/lotus#10126](https://github.com/filecoin-project/lotus/pull/10126))
- chore: market: Upgrade to index-provider 0.10.0 ([filecoin-project/lotus#9981](https://github.com/filecoin-project/lotus/pull/9981))
- chore: all: bump go-libipfs ([filecoin-project/lotus#10563](https://github.com/filecoin-project/lotus/pull/10563))
## Others
- Update service_developer_bug_report.yml ([filecoin-project/lotus#10321](https://github.com/filecoin-project/lotus/pull/10321))
- Update service_developer_bug_report.yml ([filecoin-project/lotus#10321](https://github.com/filecoin-project/lotus/pull/10321))
- chore: github: Service-provider/dev bug template ([filecoin-project/lotus#10321](https://github.com/filecoin-project/lotus/pull/10321))
- chore: github: update enhancement and feature templates ([filecoin-project/lotus#10291](https://github.com/filecoin-project/lotus/pull/10291))
- chore: github: Update bug_report template ([filecoin-project/lotus#10289](https://github.com/filecoin-project/lotus/pull/10289))
- fix: itest: avoid failing the test when we race the miner ([filecoin-project/lotus#10461](https://github.com/filecoin-project/lotus/pull/10461))
- fix: github: Discussion and FIP links in `New Issue` ([filecoin-project/lotus#10268](https://github.com/filecoin-project/lotus/pull/10268))
- fix: state: short-circuit genesis state computation ([filecoin-project/lotus#10397](https://github.com/filecoin-project/lotus/pull/10397))
- fix: rpcenc: deflake TestReaderRedirectDrop ([filecoin-project/lotus#10406](https://github.com/filecoin-project/lotus/pull/10406))
- fix: tests: Fix TestMinerAllInfo test ([filecoin-project/lotus#10319](https://github.com/filecoin-project/lotus/pull/10319))
- fix: tests: Make TestWorkerKeyChange not flaky ([filecoin-project/lotus#10320](https://github.com/filecoin-project/lotus/pull/10320))
- test: eth: make sure we can deploy a new placeholder on transfer (#10281) ([filecoin-project/lotus#10281](https://github.com/filecoin-project/lotus/pull/10281))
- fix: itests: Fix flaky paych test ([filecoin-project/lotus#10100](https://github.com/filecoin-project/lotus/pull/10100))
- fix: cli: add ArgsUsage ([filecoin-project/lotus#10147](https://github.com/filecoin-project/lotus/pull/10147))
- chore: cli: cleanup cli ([filecoin-project/lotus#10114](https://github.com/filecoin-project/lotus/pull/10114))
- chore: cli: Remove unneeded individual color flags ([filecoin-project/lotus#10028](https://github.com/filecoin-project/lotus/pull/10028))
- fix: cli: remove requirements in helptext ([filecoin-project/lotus#9969](https://github.com/filecoin-project/lotus/pull/9969))
- chore: build: release v1.21.0-rc1 prep ([filecoin-project/lotus#10524](https://github.com/filecoin-project/lotus/pull/10524))
- chore: merge release/v1.20.0 into master ([filecoin-project/lotus#10308](https://github.com/filecoin-project/lotus/pull/10308))
- chore: merge release branch into master ([filecoin-project/lotus#10272](https://github.com/filecoin-project/lotus/pull/10272))
- chore: merge release/v1.20.0 into master ([filecoin-project/lotus#10238](https://github.com/filecoin-project/lotus/pull/10238))
- chore: releases to master ([filecoin-project/lotus#10490](https://github.com/filecoin-project/lotus/pull/10490))
- chore: merge releases into master ([filecoin-project/lotus#10377](https://github.com/filecoin-project/lotus/pull/10377))
- chore: merge release/v1.20.0 into master ([filecoin-project/lotus#10030](https://github.com/filecoin-project/lotus/pull/10030))
- chore: update ffi to increase execution parallelism (#10480) ([filecoin-project/lotus#10480](https://github.com/filecoin-project/lotus/pull/10480))
- chore: update the FFI for release (#10435) ([filecoin-project/lotus#10444](https://github.com/filecoin-project/lotus/pull/10444))
- build: bump version to v1.21.0-dev ([filecoin-project/lotus#10249](https://github.com/filecoin-project/lotus/pull/10249))
- build: docker: Update GO-version (#10591) ([filecoin-project/lotus#10249](https://github.com/filecoin-project/lotus/pull/10591))
- chore: merge release/v1.20.0 into master ([filecoin-project/lotus#10184](https://github.com/filecoin-project/lotus/pull/10184))
- docs: API Gateway: patch documentation note about make gen command ([filecoin-project/lotus#10422](https://github.com/filecoin-project/lotus/pull/10422))
- chore: docs: fix docs typos ([filecoin-project/lotus#10155](https://github.com/filecoin-project/lotus/pull/10155))
- chore: docker: Add back <<network>> parameter for docker push ([filecoin-project/lotus#10096](https://github.com/filecoin-project/lotus/pull/10096))
- chore: docker: Properly balance <<?>> in circleci docker config ([filecoin-project/lotus#10088](https://github.com/filecoin-project/lotus/pull/10088))
- chore: ci: Fix dirty git state when building docker images ([filecoin-project/lotus#10125](https://github.com/filecoin-project/lotus/pull/10125))
- chore: build: Remove AppImage and Snapcraft build automation ([filecoin-project/lotus#10003](https://github.com/filecoin-project/lotus/pull/10003))
- chore: ci: Update codeql to v2 ([filecoin-project/lotus#10120](https://github.com/filecoin-project/lotus/pull/10120))
- feat: ci: make ci more efficient ([filecoin-project/lotus#9910](https://github.com/filecoin-project/lotus/pull/9910))
- feat: scripts: go.mod dep diff script ([filecoin-project/lotus#9711](https://github.com/filecoin-project/lotus/pull/9711))
## Execution Trace Format Changes
Execution traces (returned from `lotus state exec-trace`, `lotus state replay`, etc.), has changed to account for changes introduced by the FVM. Specifically:
- The `Msg` field no longer matches the Filecoin message format as many of the message fields didn't make sense in on-chain sub-calls. Instead, it now has the fields `To`, `From`, `Value`, `Method`, `Params`, and `ParamsCodec` where `ParamsCodec` is a new field indicating the IPLD codec of the parameters.
- Importantly, the `Msg.CID` field has been removed. This field is still present in top-level invocation results, just not inside the execution trace itself.
- The `MsgRct` field no longer includes a `GasUsed` field and now has a `ReturnCodec` field to indicating the IPLD codec of the return value.
- The `Error` and `Duration` fields have been removed as these are not set by the FVM. The top-level message "invocation result" retains the `Error` and `Duration` fields, they've only been removed from the trace itself.
- Gas Charges no longer include "virtual" gas fields (those starting with `v...`) or source location information (`loc`) as neither field is set by the FVM.
A note on "codecs": FVM parameters and return values are IPLD blocks where the "codec" specifies the data encoding. The codec will generally be one of:
- `0x51`, `0x71` - CBOR or DagCBOR. You should generally treat these as equivalent.
- `0x55` - Raw bytes.
- `0x00` - Nothing. If the codec is `0x00`, the parameter and/or return value should be empty and should be treated as "void" (not specified).
<details>
<summary>
Old <code>ExecutionTrace</code>:
</summary>
```json
{
"Msg": {
"Version": 0,
"To": "f01234",
"From": "f04321",
"Nonce": 1,
"Value": "0",
"GasLimit": 0,
"GasFeeCap": "1234",
"GasPremium": "1234",
"Method": 42,
"Params": "<base64-data-or-null>",
"CID": {
"/": "bafyxyz....."
},
},
"MsgRct": {
"ExitCode": 0,
"Return": "<base64-data-or-null>",
"GasUsed": 12345,
},
"Error": "",
"Duration": 568191845,
"GasCharges": [
{
"Name": "OnMethodInvocation",
"loc": null,
"tg": 23856,
"cg": 23856,
"sg": 0,
"vtg": 0,
"vcg": 0,
"vsg": 0,
"tt": 0
},
{
"Name": "wasm_exec",
"loc": null,
"tg": 1764,
"cg": 1764,
"sg": 0,
"vtg": 0,
"vcg": 0,
"vsg": 0,
"tt": 0
},
{
"Name": "OnSyscall",
"loc": null,
"tg": 14000,
"cg": 14000,
"sg": 0,
"vtg": 0,
"vcg": 0,
"vsg": 0,
"tt": 0
},
],
"Subcalls": [
{
"Msg": { },
"MsgRct": { },
"Error": "",
"Duration": 1235,
"GasCharges": [],
"Subcalls": [],
},
]
}
```
</details>
<details>
<summary>
New <code>ExecutionTrace</code>:
</summary>
```json
{
"Msg": {
"To": "f01234",
"From": "f04321",
"Value": "0",
"Method": 42,
"Params": "<base64-data-or-null>",
"ParamsCodec": 81
},
"MsgRct": {
"ExitCode": 0,
"Return": "<base64-data-or-null>",
"ReturnCodec": 81
},
"GasCharges": [
{
"Name": "OnMethodInvocation",
"loc": null,
"tg": 23856,
"cg": 23856,
"tt": 0
},
{
"Name": "wasm_exec",
"loc": null,
"tg": 1764,
"cg": 1764,
"sg": 0,
"tt": 0
},
{
"Name": "OnSyscall",
"loc": null,
"tg": 14000,
"cg": 14000,
"sg": 0,
"tt": 0
},
],
"Subcalls": [
{
"Msg": { },
"MsgRct": { },
"GasCharges": [],
"Subcalls": [],
},
]
}
```
</details>
## Contributors
| Contributor | Commits | Lines ± | Files Changed |
|-------------|---------|---------|---------------|
| Hannah Howard | 2 | +2909/-6026 | 84 |
| Łukasz Magiera | 42 | +2967/-1848 | 95 |
| Steven Allen | 20 | +1703/-1345 | 88 |
| Alfonso de la Rocha | 17 | +823/-1808 | 86 |
| Peter Rabbitson | 9 | +1957/-219 | 34 |
| Geoff Stuart | 12 | +818/-848 | 29 |
| hannahhoward | 5 | +507/-718 | 36 |
| Hector Sanjuan | 6 | +443/-726 | 35 |
| Kevin Li | 1 | +1124/-14 | 22 |
| zenground0 | 30 | +791/-269 | 88 |
| frrist | 1 | +992/-16 | 13 |
| Travis Person | 4 | +837/-53 | 24 |
| Phi | 20 | +622/-254 | 34 |
| Ian Davis | 7 | +35/-729 | 20 |
| Aayush | 10 | +378/-177 | 40 |
| Raúl Kripalani | 15 | +207/-138 | 19 |
| Arsenii Petrovich | 7 | +248/-94 | 30 |
| ZenGround0 | 5 | +238/-39 | 15 |
| Neel Virdy | 1 | +109/-107 | 58 |
| ychiao | 1 | +135/-39 | 3 |
| Jorropo | 2 | +87/-82 | 67 |
| Marten Seemann | 8 | +69/-64 | 17 |
| Rod Vagg | 1 | +55/-16 | 3 |
| Masih H. Derkani | 3 | +39/-27 | 12 |
| raulk | 2 | +30/-29 | 5 |
| dependabot[bot] | 4 | +37/-17 | 8 |
| beck | 2 | +38/-2 | 2 |
| Jennifer Wang | 4 | +20/-19 | 19 |
| Richard Guan | 3 | +28/-8 | 5 |
| omahs | 7 | +14/-14 | 7 |
| dirkmc | 2 | +19/-7 | 6 |
| David Choi | 2 | +16/-5 | 2 |
| Mike Greenberg | 1 | +18/-1 | 1 |
| Adin Schmahmann | 1 | +19/-0 | 2 |
| Phi-rjan | 5 | +12/-4 | 5 |
| Dirk McCormick | 2 | +6/-6 | 3 |
| Aayush Rajasekaran | 2 | +9/-3 | 2 |
| Jiaying Wang | 5 | +6/-4 | 5 |
| Anjor Kanekar | 1 | +5/-5 | 1 |
| vyzo | 1 | +3/-3 | 2 |
| 0x5459 | 1 | +1/-1 | 1 |
# v1.20.3 / 2023-03-09
A 🐈 stepped on the ⌨️ and made a mistake while resolving conflicts 😨. This releases only includes #10439 to fix that mistake. v1.20.2 is retracted - Please skip v1.20.2 and **only** update to v1.20.3!!!
## Changelog
> compare to v1.20.1
This is a HIGHLY RECOMMENDED patch release for node operators/API service providers that run ETH RPC service and an optional release for Storage Providers.
## Bug fixes
- fix: EthAPI: use StateCompute for feeHistory; apply minimum gas premium #10413
- fix: eth API: return correct txIdx around null blocks #10419
- fix: Eth API: make block parameter parsing sounder. #10427
## Improvement
- feat: Lotus Gateway: Add missing methods - master #10420
- feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x #10416
- We recommend storage providers to update your nodes to this patch, that will help improve developers who use Ethereum tooling's experience.
# v1.20.2 / 2023-03-09
This is a HIGHLY RECOMMENDED patch release for node operators/API service providers that run ETH RPC service and an optional release for Storage Providers.
## Bug fixes
- fix: EthAPI: use StateCompute for feeHistory; apply minimum gas premium #10413
- fix: eth API: return correct txIdx around null blocks #10419
- fix: Eth API: make block parameter parsing sounder. #10427
## Improvement
- feat: Lotus Gateway: Add missing methods - master #10420
- feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x #10416
- We recommend storage providers to update your nodes to this patch, that will help improve developers who use Ethereum tooling's experience.
# v1.20.1 / 2023-03-06
This an optional patch releases for node operators/API service providers that run ETH RPC service.
## Bug fixes
- fix: EthAPI: Correctly get parent hash #10389
- fix: EthAPI: Make newEthBlockFromFilecoinTipSet faster and correct #10380
- fix: state: short-circuit genesis state computation
# 1.20.0 / 2023-02-28
This is a MANDATORY release of Lotus that delivers the [Hygge network upgrade](https://github.com/filecoin-project/community/discussions/74?sort=top#discussioncomment-4313888), introducing Filecoin network version 18. The centerpiece of the upgrade is the introduction of the [Filecoin Virtual Machine (FVM)s Milestone 2.1](https://fvm.filecoin.io/), which will allow for EVM-compatible contracts to be deployed on the Filecoin network. This upgrade delivers user-programmablity to the Filecoin network for the first time!
+1 -1
View File
@@ -1,5 +1,5 @@
#####################################
FROM golang:1.18.8-buster AS lotus-builder
FROM golang:1.19.7-buster AS lotus-builder
MAINTAINER Lotus Development Team
RUN apt-get update && apt-get install -y ca-certificates build-essential clang ocl-icd-opencl-dev ocl-icd-libopencl1 jq libhwloc-dev
+1 -1
View File
@@ -1,6 +1,6 @@
##### DEPRECATED
FROM golang:1.18.8-buster AS builder-deps
FROM golang:1.19.7-buster AS builder-deps
MAINTAINER Lotus Development Team
RUN apt-get update && apt-get install -y ca-certificates build-essential clang ocl-icd-opencl-dev ocl-icd-libopencl1 jq libhwloc-dev
+1 -1
View File
@@ -1 +1 @@
1.18.8
1.19.7
+2 -2
View File
@@ -71,10 +71,10 @@ For other distributions you can find the required dependencies [here.](https://l
#### Go
To build Lotus, you need a working installation of [Go 1.18.8 or higher](https://golang.org/dl/):
To build Lotus, you need a working installation of [Go 1.19.7 or higher](https://golang.org/dl/):
```bash
wget -c https://golang.org/dl/go1.18.8.linux-amd64.tar.gz -O - | sudo tar -xz -C /usr/local
wget -c https://golang.org/dl/go1.19.7.linux-amd64.tar.gz -O - | sudo tar -xz -C /usr/local
```
**TIP:**
+14 -2
View File
@@ -13,7 +13,7 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
datatransfer "github.com/filecoin-project/go-data-transfer"
datatransfer "github.com/filecoin-project/go-data-transfer/v2"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-fil-markets/storagemarket"
"github.com/filecoin-project/go-jsonrpc"
@@ -183,10 +183,14 @@ type FullNode interface {
// nodes.
ChainExportRangeInternal(ctx context.Context, head, tail types.TipSetKey, cfg ChainExportConfig) error //perm:admin
// ChainPrune prunes the stored chain state and garbage collects; only supported if you
// ChainPrune forces compaction on cold store and garbage collects; only supported if you
// are using the splitstore
ChainPrune(ctx context.Context, opts PruneOpts) error //perm:admin
// ChainHotGC does online (badger) GC on the hot store; only supported if you are using
// the splitstore
ChainHotGC(ctx context.Context, opts HotGCOpts) error //perm:admin
// ChainCheckBlockstore performs an (asynchronous) health check on the chain/state blockstore
// if supported by the underlying implementation.
ChainCheckBlockstore(context.Context) error //perm:admin
@@ -793,10 +797,12 @@ type FullNode interface {
EthGetBlockByHash(ctx context.Context, blkHash ethtypes.EthHash, fullTxInfo bool) (ethtypes.EthBlock, error) //perm:read
EthGetBlockByNumber(ctx context.Context, blkNum string, fullTxInfo bool) (ethtypes.EthBlock, error) //perm:read
EthGetTransactionByHash(ctx context.Context, txHash *ethtypes.EthHash) (*ethtypes.EthTx, error) //perm:read
EthGetTransactionByHashLimited(ctx context.Context, txHash *ethtypes.EthHash, limit abi.ChainEpoch) (*ethtypes.EthTx, error) //perm:read
EthGetTransactionHashByCid(ctx context.Context, cid cid.Cid) (*ethtypes.EthHash, error) //perm:read
EthGetMessageCidByTransactionHash(ctx context.Context, txHash *ethtypes.EthHash) (*cid.Cid, error) //perm:read
EthGetTransactionCount(ctx context.Context, sender ethtypes.EthAddress, blkOpt string) (ethtypes.EthUint64, error) //perm:read
EthGetTransactionReceipt(ctx context.Context, txHash ethtypes.EthHash) (*EthTxReceipt, error) //perm:read
EthGetTransactionReceiptLimited(ctx context.Context, txHash ethtypes.EthHash, limit abi.ChainEpoch) (*EthTxReceipt, error) //perm:read
EthGetTransactionByBlockHashAndIndex(ctx context.Context, blkHash ethtypes.EthHash, txIndex ethtypes.EthUint64) (ethtypes.EthTx, error) //perm:read
EthGetTransactionByBlockNumberAndIndex(ctx context.Context, blkNum ethtypes.EthUint64, txIndex ethtypes.EthUint64) (ethtypes.EthTx, error) //perm:read
@@ -1354,6 +1360,12 @@ type PruneOpts struct {
RetainState int64
}
type HotGCOpts struct {
Threshold float64
Periodic bool
Moving bool
}
type EthTxReceipt struct {
TransactionHash ethtypes.EthHash `json:"transactionHash"`
TransactionIndex ethtypes.EthUint64 `json:"transactionIndex"`
+10 -4
View File
@@ -15,6 +15,7 @@ import (
apitypes "github.com/filecoin-project/lotus/api/types"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/types/ethtypes"
"github.com/filecoin-project/lotus/node/modules/dtypes"
)
// MODIFYING THE API INTERFACE
@@ -25,7 +26,7 @@ import (
// When adding / changing methods in this file:
// * Do the change here
// * Adjust implementation in `node/impl/`
// * Run `make gen` - this will:
// * Run `make clean && make deps && make gen` - this will:
// * Generate proxy structs
// * Generate mocks
// * Generate markdown docs
@@ -47,15 +48,18 @@ type Gateway interface {
ChainReadObj(context.Context, cid.Cid) ([]byte, error)
ChainGetGenesis(context.Context) (*types.TipSet, error)
GasEstimateMessageGas(ctx context.Context, msg *types.Message, spec *MessageSendSpec, tsk types.TipSetKey) (*types.Message, error)
MpoolGetNonce(ctx context.Context, addr address.Address) (uint64, error)
MpoolPush(ctx context.Context, sm *types.SignedMessage) (cid.Cid, error)
MsigGetAvailableBalance(ctx context.Context, addr address.Address, tsk types.TipSetKey) (types.BigInt, error)
MsigGetPending(context.Context, address.Address, types.TipSetKey) ([]*MsigTransaction, error)
MsigGetVested(ctx context.Context, addr address.Address, start types.TipSetKey, end types.TipSetKey) (types.BigInt, error)
MsigGetVestingSchedule(ctx context.Context, addr address.Address, tsk types.TipSetKey) (MsigVesting, error)
StateAccountKey(ctx context.Context, addr address.Address, tsk types.TipSetKey) (address.Address, error)
StateCall(ctx context.Context, msg *types.Message, tsk types.TipSetKey) (*InvocResult, error)
StateDealProviderCollateralBounds(ctx context.Context, size abi.PaddedPieceSize, verified bool, tsk types.TipSetKey) (DealCollateralBounds, error)
StateDecodeParams(ctx context.Context, toAddr address.Address, method abi.MethodNum, params []byte, tsk types.TipSetKey) (interface{}, error)
StateGetActor(ctx context.Context, actor address.Address, ts types.TipSetKey) (*types.Actor, error)
StateReadState(ctx context.Context, actor address.Address, tsk types.TipSetKey) (*ActorState, error) //perm:read
StateReadState(ctx context.Context, actor address.Address, tsk types.TipSetKey) (*ActorState, error)
StateListMiners(ctx context.Context, tsk types.TipSetKey) ([]address.Address, error)
StateLookupID(ctx context.Context, addr address.Address, tsk types.TipSetKey) (address.Address, error)
StateMarketBalance(ctx context.Context, addr address.Address, tsk types.TipSetKey) (MarketBalance, error)
@@ -63,8 +67,10 @@ type Gateway interface {
StateMinerInfo(ctx context.Context, actor address.Address, tsk types.TipSetKey) (MinerInfo, error)
StateMinerProvingDeadline(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*dline.Info, error)
StateMinerPower(context.Context, address.Address, types.TipSetKey) (*MinerPower, error)
StateNetworkName(context.Context) (dtypes.NetworkName, error)
StateNetworkVersion(context.Context, types.TipSetKey) (apitypes.NetworkVersion, error)
StateSectorGetInfo(ctx context.Context, maddr address.Address, n abi.SectorNumber, tsk types.TipSetKey) (*miner.SectorOnChainInfo, error)
StateVerifierStatus(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*abi.StoragePower, error)
StateVerifiedClientStatus(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*abi.StoragePower, error)
StateSearchMsg(ctx context.Context, from types.TipSetKey, msg cid.Cid, limit abi.ChainEpoch, allowReplaced bool) (*MsgLookup, error)
StateWaitMsg(ctx context.Context, cid cid.Cid, confidence uint64, limit abi.ChainEpoch, allowReplaced bool) (*MsgLookup, error)
@@ -79,12 +85,12 @@ type Gateway interface {
EthGetBlockByHash(ctx context.Context, blkHash ethtypes.EthHash, fullTxInfo bool) (ethtypes.EthBlock, error)
EthGetBlockByNumber(ctx context.Context, blkNum string, fullTxInfo bool) (ethtypes.EthBlock, error)
EthGetTransactionByHash(ctx context.Context, txHash *ethtypes.EthHash) (*ethtypes.EthTx, error)
EthGetTransactionByHashLimited(ctx context.Context, txHash *ethtypes.EthHash, limit abi.ChainEpoch) (*ethtypes.EthTx, error)
EthGetTransactionHashByCid(ctx context.Context, cid cid.Cid) (*ethtypes.EthHash, error)
EthGetMessageCidByTransactionHash(ctx context.Context, txHash *ethtypes.EthHash) (*cid.Cid, error)
EthGetTransactionCount(ctx context.Context, sender ethtypes.EthAddress, blkOpt string) (ethtypes.EthUint64, error)
EthGetTransactionReceipt(ctx context.Context, txHash ethtypes.EthHash) (*EthTxReceipt, error)
EthGetTransactionByBlockHashAndIndex(ctx context.Context, blkHash ethtypes.EthHash, txIndex ethtypes.EthUint64) (ethtypes.EthTx, error)
EthGetTransactionByBlockNumberAndIndex(ctx context.Context, blkNum ethtypes.EthUint64, txIndex ethtypes.EthUint64) (ethtypes.EthTx, error)
EthGetTransactionReceiptLimited(ctx context.Context, txHash ethtypes.EthHash, limit abi.ChainEpoch) (*EthTxReceipt, error)
EthGetCode(ctx context.Context, address ethtypes.EthAddress, blkOpt string) (ethtypes.EthBytes, error)
EthGetStorageAt(ctx context.Context, address ethtypes.EthAddress, position ethtypes.EthBytes, blkParam string) (ethtypes.EthBytes, error)
EthGetBalance(ctx context.Context, address ethtypes.EthAddress, blkParam string) (ethtypes.EthBigInt, error)
+6 -4
View File
@@ -11,7 +11,7 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
datatransfer "github.com/filecoin-project/go-data-transfer"
datatransfer "github.com/filecoin-project/go-data-transfer/v2"
"github.com/filecoin-project/go-fil-markets/piecestore"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-fil-markets/storagemarket"
@@ -212,9 +212,11 @@ type StorageMiner interface {
StorageDetachLocal(ctx context.Context, path string) error //perm:admin
StorageRedeclareLocal(ctx context.Context, id *storiface.ID, dropMissing bool) error //perm:admin
MarketImportDealData(ctx context.Context, propcid cid.Cid, path string) error //perm:write
MarketListDeals(ctx context.Context) ([]*MarketDeal, error) //perm:read
MarketListRetrievalDeals(ctx context.Context) ([]retrievalmarket.ProviderDealState, error) //perm:read
MarketImportDealData(ctx context.Context, propcid cid.Cid, path string) error //perm:write
MarketListDeals(ctx context.Context) ([]*MarketDeal, error) //perm:read
// MarketListRetrievalDeals is deprecated, returns empty list
MarketListRetrievalDeals(ctx context.Context) ([]struct{}, error) //perm:read
MarketGetDealUpdates(ctx context.Context) (<-chan storagemarket.MinerDeal, error) //perm:read
MarketListIncompleteDeals(ctx context.Context) ([]storagemarket.MinerDeal, error) //perm:read
MarketSetAsk(ctx context.Context, price types.BigInt, verifiedPrice types.BigInt, duration abi.ChainEpoch, minPieceSize abi.PaddedPieceSize, maxPieceSize abi.PaddedPieceSize) error //perm:admin
+229 -227
View File
@@ -50,22 +50,6 @@ func (t *PaymentInfo) MarshalCBOR(w io.Writer) error {
return err
}
// t.WaitSentinel (cid.Cid) (struct)
if len("WaitSentinel") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"WaitSentinel\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("WaitSentinel"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("WaitSentinel")); err != nil {
return err
}
if err := cbg.WriteCid(cw, t.WaitSentinel); err != nil {
return xerrors.Errorf("failed to write cid field t.WaitSentinel: %w", err)
}
// t.Vouchers ([]*paych.SignedVoucher) (slice)
if len("Vouchers") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"Vouchers\" was too long")
@@ -90,6 +74,23 @@ func (t *PaymentInfo) MarshalCBOR(w io.Writer) error {
return err
}
}
// t.WaitSentinel (cid.Cid) (struct)
if len("WaitSentinel") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"WaitSentinel\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("WaitSentinel"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("WaitSentinel")); err != nil {
return err
}
if err := cbg.WriteCid(cw, t.WaitSentinel); err != nil {
return xerrors.Errorf("failed to write cid field t.WaitSentinel: %w", err)
}
return nil
}
@@ -140,19 +141,6 @@ func (t *PaymentInfo) UnmarshalCBOR(r io.Reader) (err error) {
return xerrors.Errorf("unmarshaling t.Channel: %w", err)
}
}
// t.WaitSentinel (cid.Cid) (struct)
case "WaitSentinel":
{
c, err := cbg.ReadCid(cr)
if err != nil {
return xerrors.Errorf("failed to read cid field t.WaitSentinel: %w", err)
}
t.WaitSentinel = c
}
// t.Vouchers ([]*paych.SignedVoucher) (slice)
case "Vouchers":
@@ -184,6 +172,20 @@ func (t *PaymentInfo) UnmarshalCBOR(r io.Reader) (err error) {
t.Vouchers[i] = &v
}
// t.WaitSentinel (cid.Cid) (struct)
case "WaitSentinel":
{
c, err := cbg.ReadCid(cr)
if err != nil {
return xerrors.Errorf("failed to read cid field t.WaitSentinel: %w", err)
}
t.WaitSentinel = c
}
default:
// Field doesn't exist on this type, so ignore it
cbg.ScanForLinks(r, func(cid.Cid) {})
@@ -204,19 +206,19 @@ func (t *SealedRef) MarshalCBOR(w io.Writer) error {
return err
}
// t.SectorID (abi.SectorNumber) (uint64)
if len("SectorID") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"SectorID\" was too long")
// t.Size (abi.UnpaddedPieceSize) (uint64)
if len("Size") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"Size\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("SectorID"))); err != nil {
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("Size"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("SectorID")); err != nil {
if _, err := io.WriteString(w, string("Size")); err != nil {
return err
}
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.SectorID)); err != nil {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.Size)); err != nil {
return err
}
@@ -236,19 +238,19 @@ func (t *SealedRef) MarshalCBOR(w io.Writer) error {
return err
}
// t.Size (abi.UnpaddedPieceSize) (uint64)
if len("Size") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"Size\" was too long")
// t.SectorID (abi.SectorNumber) (uint64)
if len("SectorID") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"SectorID\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("Size"))); err != nil {
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("SectorID"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("Size")); err != nil {
if _, err := io.WriteString(w, string("SectorID")); err != nil {
return err
}
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.Size)); err != nil {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.SectorID)); err != nil {
return err
}
@@ -293,8 +295,8 @@ func (t *SealedRef) UnmarshalCBOR(r io.Reader) (err error) {
}
switch name {
// t.SectorID (abi.SectorNumber) (uint64)
case "SectorID":
// t.Size (abi.UnpaddedPieceSize) (uint64)
case "Size":
{
@@ -305,7 +307,7 @@ func (t *SealedRef) UnmarshalCBOR(r io.Reader) (err error) {
if maj != cbg.MajUnsignedInt {
return fmt.Errorf("wrong type for uint64 field")
}
t.SectorID = abi.SectorNumber(extra)
t.Size = abi.UnpaddedPieceSize(extra)
}
// t.Offset (abi.PaddedPieceSize) (uint64)
@@ -323,8 +325,8 @@ func (t *SealedRef) UnmarshalCBOR(r io.Reader) (err error) {
t.Offset = abi.PaddedPieceSize(extra)
}
// t.Size (abi.UnpaddedPieceSize) (uint64)
case "Size":
// t.SectorID (abi.SectorNumber) (uint64)
case "SectorID":
{
@@ -335,7 +337,7 @@ func (t *SealedRef) UnmarshalCBOR(r io.Reader) (err error) {
if maj != cbg.MajUnsignedInt {
return fmt.Errorf("wrong type for uint64 field")
}
t.Size = abi.UnpaddedPieceSize(extra)
t.SectorID = abi.SectorNumber(extra)
}
@@ -474,6 +476,28 @@ func (t *SealTicket) MarshalCBOR(w io.Writer) error {
return err
}
// t.Epoch (abi.ChainEpoch) (int64)
if len("Epoch") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"Epoch\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("Epoch"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("Epoch")); err != nil {
return err
}
if t.Epoch >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.Epoch)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.Epoch-1)); err != nil {
return err
}
}
// t.Value (abi.SealRandomness) (slice)
if len("Value") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"Value\" was too long")
@@ -497,28 +521,6 @@ func (t *SealTicket) MarshalCBOR(w io.Writer) error {
if _, err := cw.Write(t.Value[:]); err != nil {
return err
}
// t.Epoch (abi.ChainEpoch) (int64)
if len("Epoch") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"Epoch\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("Epoch"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("Epoch")); err != nil {
return err
}
if t.Epoch >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.Epoch)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.Epoch-1)); err != nil {
return err
}
}
return nil
}
@@ -560,7 +562,33 @@ func (t *SealTicket) UnmarshalCBOR(r io.Reader) (err error) {
}
switch name {
// t.Value (abi.SealRandomness) (slice)
// t.Epoch (abi.ChainEpoch) (int64)
case "Epoch":
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.Epoch = abi.ChainEpoch(extraI)
}
// t.Value (abi.SealRandomness) (slice)
case "Value":
maj, extra, err = cr.ReadHeader()
@@ -582,32 +610,6 @@ func (t *SealTicket) UnmarshalCBOR(r io.Reader) (err error) {
if _, err := io.ReadFull(cr, t.Value[:]); err != nil {
return err
}
// t.Epoch (abi.ChainEpoch) (int64)
case "Epoch":
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.Epoch = abi.ChainEpoch(extraI)
}
default:
// Field doesn't exist on this type, so ignore it
@@ -629,6 +631,28 @@ func (t *SealSeed) MarshalCBOR(w io.Writer) error {
return err
}
// t.Epoch (abi.ChainEpoch) (int64)
if len("Epoch") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"Epoch\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("Epoch"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("Epoch")); err != nil {
return err
}
if t.Epoch >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.Epoch)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.Epoch-1)); err != nil {
return err
}
}
// t.Value (abi.InteractiveSealRandomness) (slice)
if len("Value") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"Value\" was too long")
@@ -652,28 +676,6 @@ func (t *SealSeed) MarshalCBOR(w io.Writer) error {
if _, err := cw.Write(t.Value[:]); err != nil {
return err
}
// t.Epoch (abi.ChainEpoch) (int64)
if len("Epoch") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"Epoch\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("Epoch"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("Epoch")); err != nil {
return err
}
if t.Epoch >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.Epoch)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.Epoch-1)); err != nil {
return err
}
}
return nil
}
@@ -715,7 +717,33 @@ func (t *SealSeed) UnmarshalCBOR(r io.Reader) (err error) {
}
switch name {
// t.Value (abi.InteractiveSealRandomness) (slice)
// t.Epoch (abi.ChainEpoch) (int64)
case "Epoch":
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.Epoch = abi.ChainEpoch(extraI)
}
// t.Value (abi.InteractiveSealRandomness) (slice)
case "Value":
maj, extra, err = cr.ReadHeader()
@@ -737,32 +765,6 @@ func (t *SealSeed) UnmarshalCBOR(r io.Reader) (err error) {
if _, err := io.ReadFull(cr, t.Value[:]); err != nil {
return err
}
// t.Epoch (abi.ChainEpoch) (int64)
case "Epoch":
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.Epoch = abi.ChainEpoch(extraI)
}
default:
// Field doesn't exist on this type, so ignore it
@@ -784,6 +786,22 @@ func (t *PieceDealInfo) MarshalCBOR(w io.Writer) error {
return err
}
// t.DealID (abi.DealID) (uint64)
if len("DealID") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"DealID\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("DealID"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("DealID")); err != nil {
return err
}
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.DealID)); err != nil {
return err
}
// t.PublishCid (cid.Cid) (struct)
if len("PublishCid") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"PublishCid\" was too long")
@@ -806,22 +824,6 @@ func (t *PieceDealInfo) MarshalCBOR(w io.Writer) error {
}
}
// t.DealID (abi.DealID) (uint64)
if len("DealID") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"DealID\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("DealID"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("DealID")); err != nil {
return err
}
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.DealID)); err != nil {
return err
}
// t.DealProposal (market.DealProposal) (struct)
if len("DealProposal") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"DealProposal\" was too long")
@@ -910,7 +912,22 @@ func (t *PieceDealInfo) UnmarshalCBOR(r io.Reader) (err error) {
}
switch name {
// t.PublishCid (cid.Cid) (struct)
// t.DealID (abi.DealID) (uint64)
case "DealID":
{
maj, extra, err = cr.ReadHeader()
if err != nil {
return err
}
if maj != cbg.MajUnsignedInt {
return fmt.Errorf("wrong type for uint64 field")
}
t.DealID = abi.DealID(extra)
}
// t.PublishCid (cid.Cid) (struct)
case "PublishCid":
{
@@ -932,21 +949,6 @@ func (t *PieceDealInfo) UnmarshalCBOR(r io.Reader) (err error) {
t.PublishCid = &c
}
}
// t.DealID (abi.DealID) (uint64)
case "DealID":
{
maj, extra, err = cr.ReadHeader()
if err != nil {
return err
}
if maj != cbg.MajUnsignedInt {
return fmt.Errorf("wrong type for uint64 field")
}
t.DealID = abi.DealID(extra)
}
// t.DealProposal (market.DealProposal) (struct)
case "DealProposal":
@@ -1140,28 +1142,6 @@ func (t *DealSchedule) MarshalCBOR(w io.Writer) error {
return err
}
// t.StartEpoch (abi.ChainEpoch) (int64)
if len("StartEpoch") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"StartEpoch\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("StartEpoch"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("StartEpoch")); err != nil {
return err
}
if t.StartEpoch >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.StartEpoch)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.StartEpoch-1)); err != nil {
return err
}
}
// t.EndEpoch (abi.ChainEpoch) (int64)
if len("EndEpoch") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"EndEpoch\" was too long")
@@ -1183,6 +1163,28 @@ func (t *DealSchedule) MarshalCBOR(w io.Writer) error {
return err
}
}
// t.StartEpoch (abi.ChainEpoch) (int64)
if len("StartEpoch") > cbg.MaxLength {
return xerrors.Errorf("Value in field \"StartEpoch\" was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len("StartEpoch"))); err != nil {
return err
}
if _, err := io.WriteString(w, string("StartEpoch")); err != nil {
return err
}
if t.StartEpoch >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.StartEpoch)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.StartEpoch-1)); err != nil {
return err
}
}
return nil
}
@@ -1224,33 +1226,7 @@ func (t *DealSchedule) UnmarshalCBOR(r io.Reader) (err error) {
}
switch name {
// t.StartEpoch (abi.ChainEpoch) (int64)
case "StartEpoch":
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.StartEpoch = abi.ChainEpoch(extraI)
}
// t.EndEpoch (abi.ChainEpoch) (int64)
// t.EndEpoch (abi.ChainEpoch) (int64)
case "EndEpoch":
{
maj, extra, err := cr.ReadHeader()
@@ -1267,7 +1243,7 @@ func (t *DealSchedule) UnmarshalCBOR(r io.Reader) (err error) {
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
@@ -1276,6 +1252,32 @@ func (t *DealSchedule) UnmarshalCBOR(r io.Reader) (err error) {
t.EndEpoch = abi.ChainEpoch(extraI)
}
// t.StartEpoch (abi.ChainEpoch) (int64)
case "StartEpoch":
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.StartEpoch = abi.ChainEpoch(extraI)
}
default:
// Field doesn't exist on this type, so ignore it
+7 -3
View File
@@ -27,7 +27,7 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
datatransfer "github.com/filecoin-project/go-data-transfer"
datatransfer "github.com/filecoin-project/go-data-transfer/v2"
"github.com/filecoin-project/go-fil-markets/filestore"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-jsonrpc/auth"
@@ -152,8 +152,8 @@ func init() {
addExample(map[string]int{"name": 42})
addExample(map[string]time.Time{"name": time.Unix(1615243938, 0).UTC()})
addExample(&types.ExecutionTrace{
Msg: ExampleValue("init", reflect.TypeOf(&types.Message{}), nil).(*types.Message),
MsgRct: ExampleValue("init", reflect.TypeOf(&types.MessageReceipt{}), nil).(*types.MessageReceipt),
Msg: ExampleValue("init", reflect.TypeOf(types.MessageTrace{}), nil).(types.MessageTrace),
MsgRct: ExampleValue("init", reflect.TypeOf(types.ReturnTrace{}), nil).(types.ReturnTrace),
})
addExample(map[string]types.Actor{
"t01236": ExampleValue("init", reflect.TypeOf(types.Actor{}), nil).(types.Actor),
@@ -401,6 +401,10 @@ func init() {
FromBlock: pstring("2301220"),
Address: []ethtypes.EthAddress{ethaddr},
})
percent := types.Percent(123)
addExample(percent)
addExample(&percent)
}
func GetAPIType(name, pkg string) (i interface{}, t reflect.Type, permStruct []reflect.Type) {
+45 -1
View File
@@ -21,7 +21,7 @@ import (
address "github.com/filecoin-project/go-address"
bitfield "github.com/filecoin-project/go-bitfield"
datatransfer "github.com/filecoin-project/go-data-transfer"
datatransfer "github.com/filecoin-project/go-data-transfer/v2"
retrievalmarket "github.com/filecoin-project/go-fil-markets/retrievalmarket"
jsonrpc "github.com/filecoin-project/go-jsonrpc"
auth "github.com/filecoin-project/go-jsonrpc/auth"
@@ -394,6 +394,20 @@ func (mr *MockFullNodeMockRecorder) ChainHead(arg0 interface{}) *gomock.Call {
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ChainHead", reflect.TypeOf((*MockFullNode)(nil).ChainHead), arg0)
}
// ChainHotGC mocks base method.
func (m *MockFullNode) ChainHotGC(arg0 context.Context, arg1 api.HotGCOpts) error {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "ChainHotGC", arg0, arg1)
ret0, _ := ret[0].(error)
return ret0
}
// ChainHotGC indicates an expected call of ChainHotGC.
func (mr *MockFullNodeMockRecorder) ChainHotGC(arg0, arg1 interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "ChainHotGC", reflect.TypeOf((*MockFullNode)(nil).ChainHotGC), arg0, arg1)
}
// ChainNotify mocks base method.
func (m *MockFullNode) ChainNotify(arg0 context.Context) (<-chan []*api.HeadChange, error) {
m.ctrl.T.Helper()
@@ -1282,6 +1296,21 @@ func (mr *MockFullNodeMockRecorder) EthGetTransactionByHash(arg0, arg1 interface
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "EthGetTransactionByHash", reflect.TypeOf((*MockFullNode)(nil).EthGetTransactionByHash), arg0, arg1)
}
// EthGetTransactionByHashLimited mocks base method.
func (m *MockFullNode) EthGetTransactionByHashLimited(arg0 context.Context, arg1 *ethtypes.EthHash, arg2 abi.ChainEpoch) (*ethtypes.EthTx, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "EthGetTransactionByHashLimited", arg0, arg1, arg2)
ret0, _ := ret[0].(*ethtypes.EthTx)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// EthGetTransactionByHashLimited indicates an expected call of EthGetTransactionByHashLimited.
func (mr *MockFullNodeMockRecorder) EthGetTransactionByHashLimited(arg0, arg1, arg2 interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "EthGetTransactionByHashLimited", reflect.TypeOf((*MockFullNode)(nil).EthGetTransactionByHashLimited), arg0, arg1, arg2)
}
// EthGetTransactionCount mocks base method.
func (m *MockFullNode) EthGetTransactionCount(arg0 context.Context, arg1 ethtypes.EthAddress, arg2 string) (ethtypes.EthUint64, error) {
m.ctrl.T.Helper()
@@ -1327,6 +1356,21 @@ func (mr *MockFullNodeMockRecorder) EthGetTransactionReceipt(arg0, arg1 interfac
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "EthGetTransactionReceipt", reflect.TypeOf((*MockFullNode)(nil).EthGetTransactionReceipt), arg0, arg1)
}
// EthGetTransactionReceiptLimited mocks base method.
func (m *MockFullNode) EthGetTransactionReceiptLimited(arg0 context.Context, arg1 ethtypes.EthHash, arg2 abi.ChainEpoch) (*api.EthTxReceipt, error) {
m.ctrl.T.Helper()
ret := m.ctrl.Call(m, "EthGetTransactionReceiptLimited", arg0, arg1, arg2)
ret0, _ := ret[0].(*api.EthTxReceipt)
ret1, _ := ret[1].(error)
return ret0, ret1
}
// EthGetTransactionReceiptLimited indicates an expected call of EthGetTransactionReceiptLimited.
func (mr *MockFullNodeMockRecorder) EthGetTransactionReceiptLimited(arg0, arg1, arg2 interface{}) *gomock.Call {
mr.mock.ctrl.T.Helper()
return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "EthGetTransactionReceiptLimited", reflect.TypeOf((*MockFullNode)(nil).EthGetTransactionReceiptLimited), arg0, arg1, arg2)
}
// EthMaxPriorityFeePerGas mocks base method.
func (m *MockFullNode) EthMaxPriorityFeePerGas(arg0 context.Context) (ethtypes.EthBigInt, error) {
m.ctrl.T.Helper()
+137 -33
View File
@@ -18,7 +18,7 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
datatransfer "github.com/filecoin-project/go-data-transfer"
datatransfer "github.com/filecoin-project/go-data-transfer/v2"
"github.com/filecoin-project/go-fil-markets/piecestore"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-fil-markets/storagemarket"
@@ -172,6 +172,8 @@ type FullNodeMethods struct {
ChainHead func(p0 context.Context) (*types.TipSet, error) `perm:"read"`
ChainHotGC func(p0 context.Context, p1 HotGCOpts) error `perm:"admin"`
ChainNotify func(p0 context.Context) (<-chan []*HeadChange, error) `perm:"read"`
ChainPrune func(p0 context.Context, p1 PruneOpts) error `perm:"admin"`
@@ -288,12 +290,16 @@ type FullNodeMethods struct {
EthGetTransactionByHash func(p0 context.Context, p1 *ethtypes.EthHash) (*ethtypes.EthTx, error) `perm:"read"`
EthGetTransactionByHashLimited func(p0 context.Context, p1 *ethtypes.EthHash, p2 abi.ChainEpoch) (*ethtypes.EthTx, error) `perm:"read"`
EthGetTransactionCount func(p0 context.Context, p1 ethtypes.EthAddress, p2 string) (ethtypes.EthUint64, error) `perm:"read"`
EthGetTransactionHashByCid func(p0 context.Context, p1 cid.Cid) (*ethtypes.EthHash, error) `perm:"read"`
EthGetTransactionReceipt func(p0 context.Context, p1 ethtypes.EthHash) (*EthTxReceipt, error) `perm:"read"`
EthGetTransactionReceiptLimited func(p0 context.Context, p1 ethtypes.EthHash, p2 abi.ChainEpoch) (*EthTxReceipt, error) `perm:"read"`
EthMaxPriorityFeePerGas func(p0 context.Context) (ethtypes.EthBigInt, error) `perm:"read"`
EthNewBlockFilter func(p0 context.Context) (ethtypes.EthFilterID, error) `perm:"write"`
@@ -690,18 +696,18 @@ type GatewayMethods struct {
EthGetStorageAt func(p0 context.Context, p1 ethtypes.EthAddress, p2 ethtypes.EthBytes, p3 string) (ethtypes.EthBytes, error) ``
EthGetTransactionByBlockHashAndIndex func(p0 context.Context, p1 ethtypes.EthHash, p2 ethtypes.EthUint64) (ethtypes.EthTx, error) ``
EthGetTransactionByBlockNumberAndIndex func(p0 context.Context, p1 ethtypes.EthUint64, p2 ethtypes.EthUint64) (ethtypes.EthTx, error) ``
EthGetTransactionByHash func(p0 context.Context, p1 *ethtypes.EthHash) (*ethtypes.EthTx, error) ``
EthGetTransactionByHashLimited func(p0 context.Context, p1 *ethtypes.EthHash, p2 abi.ChainEpoch) (*ethtypes.EthTx, error) ``
EthGetTransactionCount func(p0 context.Context, p1 ethtypes.EthAddress, p2 string) (ethtypes.EthUint64, error) ``
EthGetTransactionHashByCid func(p0 context.Context, p1 cid.Cid) (*ethtypes.EthHash, error) ``
EthGetTransactionReceipt func(p0 context.Context, p1 ethtypes.EthHash) (*EthTxReceipt, error) ``
EthGetTransactionReceiptLimited func(p0 context.Context, p1 ethtypes.EthHash, p2 abi.ChainEpoch) (*EthTxReceipt, error) ``
EthMaxPriorityFeePerGas func(p0 context.Context) (ethtypes.EthBigInt, error) ``
EthNewBlockFilter func(p0 context.Context) (ethtypes.EthFilterID, error) ``
@@ -722,6 +728,8 @@ type GatewayMethods struct {
GasEstimateMessageGas func(p0 context.Context, p1 *types.Message, p2 *MessageSendSpec, p3 types.TipSetKey) (*types.Message, error) ``
MpoolGetNonce func(p0 context.Context, p1 address.Address) (uint64, error) ``
MpoolPush func(p0 context.Context, p1 *types.SignedMessage) (cid.Cid, error) ``
MsigGetAvailableBalance func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (types.BigInt, error) ``
@@ -738,8 +746,12 @@ type GatewayMethods struct {
StateAccountKey func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (address.Address, error) ``
StateCall func(p0 context.Context, p1 *types.Message, p2 types.TipSetKey) (*InvocResult, error) ``
StateDealProviderCollateralBounds func(p0 context.Context, p1 abi.PaddedPieceSize, p2 bool, p3 types.TipSetKey) (DealCollateralBounds, error) ``
StateDecodeParams func(p0 context.Context, p1 address.Address, p2 abi.MethodNum, p3 []byte, p4 types.TipSetKey) (interface{}, error) ``
StateGetActor func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*types.Actor, error) ``
StateListMiners func(p0 context.Context, p1 types.TipSetKey) ([]address.Address, error) ``
@@ -756,9 +768,11 @@ type GatewayMethods struct {
StateMinerProvingDeadline func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*dline.Info, error) ``
StateNetworkName func(p0 context.Context) (dtypes.NetworkName, error) ``
StateNetworkVersion func(p0 context.Context, p1 types.TipSetKey) (apitypes.NetworkVersion, error) ``
StateReadState func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*ActorState, error) `perm:"read"`
StateReadState func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*ActorState, error) ``
StateSearchMsg func(p0 context.Context, p1 types.TipSetKey, p2 cid.Cid, p3 abi.ChainEpoch, p4 bool) (*MsgLookup, error) ``
@@ -766,6 +780,8 @@ type GatewayMethods struct {
StateVerifiedClientStatus func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*abi.StoragePower, error) ``
StateVerifierStatus func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*abi.StoragePower, error) ``
StateWaitMsg func(p0 context.Context, p1 cid.Cid, p2 uint64, p3 abi.ChainEpoch, p4 bool) (*MsgLookup, error) ``
Version func(p0 context.Context) (APIVersion, error) ``
@@ -945,7 +961,7 @@ type StorageMinerMethods struct {
MarketListIncompleteDeals func(p0 context.Context) ([]storagemarket.MinerDeal, error) `perm:"read"`
MarketListRetrievalDeals func(p0 context.Context) ([]retrievalmarket.ProviderDealState, error) `perm:"read"`
MarketListRetrievalDeals func(p0 context.Context) ([]struct{}, error) `perm:"read"`
MarketPendingDeals func(p0 context.Context) (PendingDealInfo, error) `perm:"write"`
@@ -1617,6 +1633,17 @@ func (s *FullNodeStub) ChainHead(p0 context.Context) (*types.TipSet, error) {
return nil, ErrNotSupported
}
func (s *FullNodeStruct) ChainHotGC(p0 context.Context, p1 HotGCOpts) error {
if s.Internal.ChainHotGC == nil {
return ErrNotSupported
}
return s.Internal.ChainHotGC(p0, p1)
}
func (s *FullNodeStub) ChainHotGC(p0 context.Context, p1 HotGCOpts) error {
return ErrNotSupported
}
func (s *FullNodeStruct) ChainNotify(p0 context.Context) (<-chan []*HeadChange, error) {
if s.Internal.ChainNotify == nil {
return nil, ErrNotSupported
@@ -2255,6 +2282,17 @@ func (s *FullNodeStub) EthGetTransactionByHash(p0 context.Context, p1 *ethtypes.
return nil, ErrNotSupported
}
func (s *FullNodeStruct) EthGetTransactionByHashLimited(p0 context.Context, p1 *ethtypes.EthHash, p2 abi.ChainEpoch) (*ethtypes.EthTx, error) {
if s.Internal.EthGetTransactionByHashLimited == nil {
return nil, ErrNotSupported
}
return s.Internal.EthGetTransactionByHashLimited(p0, p1, p2)
}
func (s *FullNodeStub) EthGetTransactionByHashLimited(p0 context.Context, p1 *ethtypes.EthHash, p2 abi.ChainEpoch) (*ethtypes.EthTx, error) {
return nil, ErrNotSupported
}
func (s *FullNodeStruct) EthGetTransactionCount(p0 context.Context, p1 ethtypes.EthAddress, p2 string) (ethtypes.EthUint64, error) {
if s.Internal.EthGetTransactionCount == nil {
return *new(ethtypes.EthUint64), ErrNotSupported
@@ -2288,6 +2326,17 @@ func (s *FullNodeStub) EthGetTransactionReceipt(p0 context.Context, p1 ethtypes.
return nil, ErrNotSupported
}
func (s *FullNodeStruct) EthGetTransactionReceiptLimited(p0 context.Context, p1 ethtypes.EthHash, p2 abi.ChainEpoch) (*EthTxReceipt, error) {
if s.Internal.EthGetTransactionReceiptLimited == nil {
return nil, ErrNotSupported
}
return s.Internal.EthGetTransactionReceiptLimited(p0, p1, p2)
}
func (s *FullNodeStub) EthGetTransactionReceiptLimited(p0 context.Context, p1 ethtypes.EthHash, p2 abi.ChainEpoch) (*EthTxReceipt, error) {
return nil, ErrNotSupported
}
func (s *FullNodeStruct) EthMaxPriorityFeePerGas(p0 context.Context) (ethtypes.EthBigInt, error) {
if s.Internal.EthMaxPriorityFeePerGas == nil {
return *new(ethtypes.EthBigInt), ErrNotSupported
@@ -4400,28 +4449,6 @@ func (s *GatewayStub) EthGetStorageAt(p0 context.Context, p1 ethtypes.EthAddress
return *new(ethtypes.EthBytes), ErrNotSupported
}
func (s *GatewayStruct) EthGetTransactionByBlockHashAndIndex(p0 context.Context, p1 ethtypes.EthHash, p2 ethtypes.EthUint64) (ethtypes.EthTx, error) {
if s.Internal.EthGetTransactionByBlockHashAndIndex == nil {
return *new(ethtypes.EthTx), ErrNotSupported
}
return s.Internal.EthGetTransactionByBlockHashAndIndex(p0, p1, p2)
}
func (s *GatewayStub) EthGetTransactionByBlockHashAndIndex(p0 context.Context, p1 ethtypes.EthHash, p2 ethtypes.EthUint64) (ethtypes.EthTx, error) {
return *new(ethtypes.EthTx), ErrNotSupported
}
func (s *GatewayStruct) EthGetTransactionByBlockNumberAndIndex(p0 context.Context, p1 ethtypes.EthUint64, p2 ethtypes.EthUint64) (ethtypes.EthTx, error) {
if s.Internal.EthGetTransactionByBlockNumberAndIndex == nil {
return *new(ethtypes.EthTx), ErrNotSupported
}
return s.Internal.EthGetTransactionByBlockNumberAndIndex(p0, p1, p2)
}
func (s *GatewayStub) EthGetTransactionByBlockNumberAndIndex(p0 context.Context, p1 ethtypes.EthUint64, p2 ethtypes.EthUint64) (ethtypes.EthTx, error) {
return *new(ethtypes.EthTx), ErrNotSupported
}
func (s *GatewayStruct) EthGetTransactionByHash(p0 context.Context, p1 *ethtypes.EthHash) (*ethtypes.EthTx, error) {
if s.Internal.EthGetTransactionByHash == nil {
return nil, ErrNotSupported
@@ -4433,6 +4460,17 @@ func (s *GatewayStub) EthGetTransactionByHash(p0 context.Context, p1 *ethtypes.E
return nil, ErrNotSupported
}
func (s *GatewayStruct) EthGetTransactionByHashLimited(p0 context.Context, p1 *ethtypes.EthHash, p2 abi.ChainEpoch) (*ethtypes.EthTx, error) {
if s.Internal.EthGetTransactionByHashLimited == nil {
return nil, ErrNotSupported
}
return s.Internal.EthGetTransactionByHashLimited(p0, p1, p2)
}
func (s *GatewayStub) EthGetTransactionByHashLimited(p0 context.Context, p1 *ethtypes.EthHash, p2 abi.ChainEpoch) (*ethtypes.EthTx, error) {
return nil, ErrNotSupported
}
func (s *GatewayStruct) EthGetTransactionCount(p0 context.Context, p1 ethtypes.EthAddress, p2 string) (ethtypes.EthUint64, error) {
if s.Internal.EthGetTransactionCount == nil {
return *new(ethtypes.EthUint64), ErrNotSupported
@@ -4466,6 +4504,17 @@ func (s *GatewayStub) EthGetTransactionReceipt(p0 context.Context, p1 ethtypes.E
return nil, ErrNotSupported
}
func (s *GatewayStruct) EthGetTransactionReceiptLimited(p0 context.Context, p1 ethtypes.EthHash, p2 abi.ChainEpoch) (*EthTxReceipt, error) {
if s.Internal.EthGetTransactionReceiptLimited == nil {
return nil, ErrNotSupported
}
return s.Internal.EthGetTransactionReceiptLimited(p0, p1, p2)
}
func (s *GatewayStub) EthGetTransactionReceiptLimited(p0 context.Context, p1 ethtypes.EthHash, p2 abi.ChainEpoch) (*EthTxReceipt, error) {
return nil, ErrNotSupported
}
func (s *GatewayStruct) EthMaxPriorityFeePerGas(p0 context.Context) (ethtypes.EthBigInt, error) {
if s.Internal.EthMaxPriorityFeePerGas == nil {
return *new(ethtypes.EthBigInt), ErrNotSupported
@@ -4576,6 +4625,17 @@ func (s *GatewayStub) GasEstimateMessageGas(p0 context.Context, p1 *types.Messag
return nil, ErrNotSupported
}
func (s *GatewayStruct) MpoolGetNonce(p0 context.Context, p1 address.Address) (uint64, error) {
if s.Internal.MpoolGetNonce == nil {
return 0, ErrNotSupported
}
return s.Internal.MpoolGetNonce(p0, p1)
}
func (s *GatewayStub) MpoolGetNonce(p0 context.Context, p1 address.Address) (uint64, error) {
return 0, ErrNotSupported
}
func (s *GatewayStruct) MpoolPush(p0 context.Context, p1 *types.SignedMessage) (cid.Cid, error) {
if s.Internal.MpoolPush == nil {
return *new(cid.Cid), ErrNotSupported
@@ -4664,6 +4724,17 @@ func (s *GatewayStub) StateAccountKey(p0 context.Context, p1 address.Address, p2
return *new(address.Address), ErrNotSupported
}
func (s *GatewayStruct) StateCall(p0 context.Context, p1 *types.Message, p2 types.TipSetKey) (*InvocResult, error) {
if s.Internal.StateCall == nil {
return nil, ErrNotSupported
}
return s.Internal.StateCall(p0, p1, p2)
}
func (s *GatewayStub) StateCall(p0 context.Context, p1 *types.Message, p2 types.TipSetKey) (*InvocResult, error) {
return nil, ErrNotSupported
}
func (s *GatewayStruct) StateDealProviderCollateralBounds(p0 context.Context, p1 abi.PaddedPieceSize, p2 bool, p3 types.TipSetKey) (DealCollateralBounds, error) {
if s.Internal.StateDealProviderCollateralBounds == nil {
return *new(DealCollateralBounds), ErrNotSupported
@@ -4675,6 +4746,17 @@ func (s *GatewayStub) StateDealProviderCollateralBounds(p0 context.Context, p1 a
return *new(DealCollateralBounds), ErrNotSupported
}
func (s *GatewayStruct) StateDecodeParams(p0 context.Context, p1 address.Address, p2 abi.MethodNum, p3 []byte, p4 types.TipSetKey) (interface{}, error) {
if s.Internal.StateDecodeParams == nil {
return nil, ErrNotSupported
}
return s.Internal.StateDecodeParams(p0, p1, p2, p3, p4)
}
func (s *GatewayStub) StateDecodeParams(p0 context.Context, p1 address.Address, p2 abi.MethodNum, p3 []byte, p4 types.TipSetKey) (interface{}, error) {
return nil, ErrNotSupported
}
func (s *GatewayStruct) StateGetActor(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*types.Actor, error) {
if s.Internal.StateGetActor == nil {
return nil, ErrNotSupported
@@ -4763,6 +4845,17 @@ func (s *GatewayStub) StateMinerProvingDeadline(p0 context.Context, p1 address.A
return nil, ErrNotSupported
}
func (s *GatewayStruct) StateNetworkName(p0 context.Context) (dtypes.NetworkName, error) {
if s.Internal.StateNetworkName == nil {
return *new(dtypes.NetworkName), ErrNotSupported
}
return s.Internal.StateNetworkName(p0)
}
func (s *GatewayStub) StateNetworkName(p0 context.Context) (dtypes.NetworkName, error) {
return *new(dtypes.NetworkName), ErrNotSupported
}
func (s *GatewayStruct) StateNetworkVersion(p0 context.Context, p1 types.TipSetKey) (apitypes.NetworkVersion, error) {
if s.Internal.StateNetworkVersion == nil {
return *new(apitypes.NetworkVersion), ErrNotSupported
@@ -4818,6 +4911,17 @@ func (s *GatewayStub) StateVerifiedClientStatus(p0 context.Context, p1 address.A
return nil, ErrNotSupported
}
func (s *GatewayStruct) StateVerifierStatus(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*abi.StoragePower, error) {
if s.Internal.StateVerifierStatus == nil {
return nil, ErrNotSupported
}
return s.Internal.StateVerifierStatus(p0, p1, p2)
}
func (s *GatewayStub) StateVerifierStatus(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*abi.StoragePower, error) {
return nil, ErrNotSupported
}
func (s *GatewayStruct) StateWaitMsg(p0 context.Context, p1 cid.Cid, p2 uint64, p3 abi.ChainEpoch, p4 bool) (*MsgLookup, error) {
if s.Internal.StateWaitMsg == nil {
return nil, ErrNotSupported
@@ -5632,15 +5736,15 @@ func (s *StorageMinerStub) MarketListIncompleteDeals(p0 context.Context) ([]stor
return *new([]storagemarket.MinerDeal), ErrNotSupported
}
func (s *StorageMinerStruct) MarketListRetrievalDeals(p0 context.Context) ([]retrievalmarket.ProviderDealState, error) {
func (s *StorageMinerStruct) MarketListRetrievalDeals(p0 context.Context) ([]struct{}, error) {
if s.Internal.MarketListRetrievalDeals == nil {
return *new([]retrievalmarket.ProviderDealState), ErrNotSupported
return *new([]struct{}), ErrNotSupported
}
return s.Internal.MarketListRetrievalDeals(p0)
}
func (s *StorageMinerStub) MarketListRetrievalDeals(p0 context.Context) ([]retrievalmarket.ProviderDealState, error) {
return *new([]retrievalmarket.ProviderDealState), ErrNotSupported
func (s *StorageMinerStub) MarketListRetrievalDeals(p0 context.Context) ([]struct{}, error) {
return *new([]struct{}), ErrNotSupported
}
func (s *StorageMinerStruct) MarketPendingDeals(p0 context.Context) (PendingDealInfo, error) {
+8 -10
View File
@@ -8,13 +8,15 @@ import (
"github.com/google/uuid"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-graphsync"
"github.com/ipld/go-ipld-prime"
"github.com/ipld/go-ipld-prime/codec/dagjson"
pubsub "github.com/libp2p/go-libp2p-pubsub"
"github.com/libp2p/go-libp2p/core/network"
"github.com/libp2p/go-libp2p/core/peer"
ma "github.com/multiformats/go-multiaddr"
"github.com/filecoin-project/go-address"
datatransfer "github.com/filecoin-project/go-data-transfer"
datatransfer "github.com/filecoin-project/go-data-transfer/v2"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/builtin/v9/miner"
@@ -110,16 +112,12 @@ func NewDataTransferChannel(hostID peer.ID, channelState datatransfer.ChannelSta
IsSender: channelState.Sender() == hostID,
Message: channelState.Message(),
}
stringer, ok := channelState.Voucher().(fmt.Stringer)
if ok {
channel.Voucher = stringer.String()
voucher := channelState.Voucher()
voucherJSON, err := ipld.Encode(voucher.Voucher, dagjson.Encode)
if err != nil {
channel.Voucher = fmt.Errorf("Voucher Serialization: %w", err).Error()
} else {
voucherJSON, err := json.Marshal(channelState.Voucher())
if err != nil {
channel.Voucher = fmt.Errorf("Voucher Serialization: %w", err).Error()
} else {
channel.Voucher = string(voucherJSON)
}
channel.Voucher = string(voucherJSON)
}
if channel.IsSender {
channel.IsInitiator = !channelState.IsPull()
+1 -1
View File
@@ -10,7 +10,7 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
datatransfer "github.com/filecoin-project/go-data-transfer"
datatransfer "github.com/filecoin-project/go-data-transfer/v2"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-fil-markets/storagemarket"
"github.com/filecoin-project/go-state-types/abi"
+5
View File
@@ -14,6 +14,7 @@ import (
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/node/modules/dtypes"
)
// MODIFYING THE API INTERFACE
@@ -44,12 +45,15 @@ type Gateway interface {
ChainNotify(context.Context) (<-chan []*api.HeadChange, error)
ChainReadObj(context.Context, cid.Cid) ([]byte, error)
GasEstimateMessageGas(ctx context.Context, msg *types.Message, spec *api.MessageSendSpec, tsk types.TipSetKey) (*types.Message, error)
MpoolGetNonce(ctx context.Context, addr address.Address) (uint64, error)
MpoolPush(ctx context.Context, sm *types.SignedMessage) (cid.Cid, error)
MsigGetAvailableBalance(ctx context.Context, addr address.Address, tsk types.TipSetKey) (types.BigInt, error)
MsigGetVested(ctx context.Context, addr address.Address, start types.TipSetKey, end types.TipSetKey) (types.BigInt, error)
MsigGetPending(context.Context, address.Address, types.TipSetKey) ([]*api.MsigTransaction, error)
StateAccountKey(ctx context.Context, addr address.Address, tsk types.TipSetKey) (address.Address, error)
StateCall(ctx context.Context, msg *types.Message, tsk types.TipSetKey) (*api.InvocResult, error)
StateDealProviderCollateralBounds(ctx context.Context, size abi.PaddedPieceSize, verified bool, tsk types.TipSetKey) (api.DealCollateralBounds, error)
StateDecodeParams(ctx context.Context, toAddr address.Address, method abi.MethodNum, params []byte, tsk types.TipSetKey) (interface{}, error)
StateGetActor(ctx context.Context, actor address.Address, ts types.TipSetKey) (*types.Actor, error)
StateGetReceipt(context.Context, cid.Cid, types.TipSetKey) (*types.MessageReceipt, error)
StateListMiners(ctx context.Context, tsk types.TipSetKey) ([]address.Address, error)
@@ -59,6 +63,7 @@ type Gateway interface {
StateMinerInfo(ctx context.Context, actor address.Address, tsk types.TipSetKey) (api.MinerInfo, error)
StateMinerProvingDeadline(ctx context.Context, addr address.Address, tsk types.TipSetKey) (*dline.Info, error)
StateMinerPower(context.Context, address.Address, types.TipSetKey) (*api.MinerPower, error)
StateNetworkName(context.Context) (dtypes.NetworkName, error)
StateNetworkVersion(context.Context, types.TipSetKey) (abinetwork.Version, error)
StateSearchMsg(ctx context.Context, msg cid.Cid) (*api.MsgLookup, error)
StateSectorGetInfo(ctx context.Context, maddr address.Address, n abi.SectorNumber, tsk types.TipSetKey) (*miner.SectorOnChainInfo, error)
+53 -1
View File
@@ -12,7 +12,7 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-bitfield"
datatransfer "github.com/filecoin-project/go-data-transfer"
datatransfer "github.com/filecoin-project/go-data-transfer/v2"
"github.com/filecoin-project/go-fil-markets/retrievalmarket"
"github.com/filecoin-project/go-fil-markets/storagemarket"
"github.com/filecoin-project/go-state-types/abi"
@@ -451,6 +451,8 @@ type GatewayMethods struct {
GasEstimateMessageGas func(p0 context.Context, p1 *types.Message, p2 *api.MessageSendSpec, p3 types.TipSetKey) (*types.Message, error) ``
MpoolGetNonce func(p0 context.Context, p1 address.Address) (uint64, error) ``
MpoolPush func(p0 context.Context, p1 *types.SignedMessage) (cid.Cid, error) ``
MsigGetAvailableBalance func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (types.BigInt, error) ``
@@ -461,8 +463,12 @@ type GatewayMethods struct {
StateAccountKey func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (address.Address, error) ``
StateCall func(p0 context.Context, p1 *types.Message, p2 types.TipSetKey) (*api.InvocResult, error) ``
StateDealProviderCollateralBounds func(p0 context.Context, p1 abi.PaddedPieceSize, p2 bool, p3 types.TipSetKey) (api.DealCollateralBounds, error) ``
StateDecodeParams func(p0 context.Context, p1 address.Address, p2 abi.MethodNum, p3 []byte, p4 types.TipSetKey) (interface{}, error) ``
StateGetActor func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*types.Actor, error) ``
StateGetReceipt func(p0 context.Context, p1 cid.Cid, p2 types.TipSetKey) (*types.MessageReceipt, error) ``
@@ -481,6 +487,8 @@ type GatewayMethods struct {
StateMinerProvingDeadline func(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*dline.Info, error) ``
StateNetworkName func(p0 context.Context) (dtypes.NetworkName, error) ``
StateNetworkVersion func(p0 context.Context, p1 types.TipSetKey) (abinetwork.Version, error) ``
StateSearchMsg func(p0 context.Context, p1 cid.Cid) (*api.MsgLookup, error) ``
@@ -2677,6 +2685,17 @@ func (s *GatewayStub) GasEstimateMessageGas(p0 context.Context, p1 *types.Messag
return nil, ErrNotSupported
}
func (s *GatewayStruct) MpoolGetNonce(p0 context.Context, p1 address.Address) (uint64, error) {
if s.Internal.MpoolGetNonce == nil {
return 0, ErrNotSupported
}
return s.Internal.MpoolGetNonce(p0, p1)
}
func (s *GatewayStub) MpoolGetNonce(p0 context.Context, p1 address.Address) (uint64, error) {
return 0, ErrNotSupported
}
func (s *GatewayStruct) MpoolPush(p0 context.Context, p1 *types.SignedMessage) (cid.Cid, error) {
if s.Internal.MpoolPush == nil {
return *new(cid.Cid), ErrNotSupported
@@ -2732,6 +2751,17 @@ func (s *GatewayStub) StateAccountKey(p0 context.Context, p1 address.Address, p2
return *new(address.Address), ErrNotSupported
}
func (s *GatewayStruct) StateCall(p0 context.Context, p1 *types.Message, p2 types.TipSetKey) (*api.InvocResult, error) {
if s.Internal.StateCall == nil {
return nil, ErrNotSupported
}
return s.Internal.StateCall(p0, p1, p2)
}
func (s *GatewayStub) StateCall(p0 context.Context, p1 *types.Message, p2 types.TipSetKey) (*api.InvocResult, error) {
return nil, ErrNotSupported
}
func (s *GatewayStruct) StateDealProviderCollateralBounds(p0 context.Context, p1 abi.PaddedPieceSize, p2 bool, p3 types.TipSetKey) (api.DealCollateralBounds, error) {
if s.Internal.StateDealProviderCollateralBounds == nil {
return *new(api.DealCollateralBounds), ErrNotSupported
@@ -2743,6 +2773,17 @@ func (s *GatewayStub) StateDealProviderCollateralBounds(p0 context.Context, p1 a
return *new(api.DealCollateralBounds), ErrNotSupported
}
func (s *GatewayStruct) StateDecodeParams(p0 context.Context, p1 address.Address, p2 abi.MethodNum, p3 []byte, p4 types.TipSetKey) (interface{}, error) {
if s.Internal.StateDecodeParams == nil {
return nil, ErrNotSupported
}
return s.Internal.StateDecodeParams(p0, p1, p2, p3, p4)
}
func (s *GatewayStub) StateDecodeParams(p0 context.Context, p1 address.Address, p2 abi.MethodNum, p3 []byte, p4 types.TipSetKey) (interface{}, error) {
return nil, ErrNotSupported
}
func (s *GatewayStruct) StateGetActor(p0 context.Context, p1 address.Address, p2 types.TipSetKey) (*types.Actor, error) {
if s.Internal.StateGetActor == nil {
return nil, ErrNotSupported
@@ -2842,6 +2883,17 @@ func (s *GatewayStub) StateMinerProvingDeadline(p0 context.Context, p1 address.A
return nil, ErrNotSupported
}
func (s *GatewayStruct) StateNetworkName(p0 context.Context) (dtypes.NetworkName, error) {
if s.Internal.StateNetworkName == nil {
return *new(dtypes.NetworkName), ErrNotSupported
}
return s.Internal.StateNetworkName(p0)
}
func (s *GatewayStub) StateNetworkName(p0 context.Context) (dtypes.NetworkName, error) {
return *new(dtypes.NetworkName), ErrNotSupported
}
func (s *GatewayStruct) StateNetworkVersion(p0 context.Context, p1 types.TipSetKey) (abinetwork.Version, error) {
if s.Internal.StateNetworkVersion == nil {
return *new(abinetwork.Version), ErrNotSupported
+1 -1
View File
@@ -20,7 +20,7 @@ import (
address "github.com/filecoin-project/go-address"
bitfield "github.com/filecoin-project/go-bitfield"
datatransfer "github.com/filecoin-project/go-data-transfer"
datatransfer "github.com/filecoin-project/go-data-transfer/v2"
retrievalmarket "github.com/filecoin-project/go-fil-markets/retrievalmarket"
storagemarket "github.com/filecoin-project/go-fil-markets/storagemarket"
auth "github.com/filecoin-project/go-jsonrpc/auth"
+85 -6
View File
@@ -20,6 +20,7 @@ import (
pool "github.com/libp2p/go-buffer-pool"
"github.com/multiformats/go-base32"
"go.uber.org/zap"
"golang.org/x/xerrors"
"github.com/filecoin-project/lotus/blockstore"
)
@@ -44,7 +45,8 @@ const (
// MemoryMap is equivalent to badger/options.MemoryMap.
MemoryMap = options.MemoryMap
// LoadToRAM is equivalent to badger/options.LoadToRAM.
LoadToRAM = options.LoadToRAM
LoadToRAM = options.LoadToRAM
defaultGCThreshold = 0.125
)
// Options embeds the badger options themselves, and augments them with
@@ -394,6 +396,9 @@ func (b *Blockstore) doCopy(from, to *badger.DB) error {
if workers < 2 {
workers = 2
}
if workers > 8 {
workers = 8
}
stream := from.NewStream()
stream.NumGo = workers
@@ -439,7 +444,7 @@ func (b *Blockstore) deleteDB(path string) {
}
}
func (b *Blockstore) onlineGC() error {
func (b *Blockstore) onlineGC(ctx context.Context, threshold float64, checkFreq time.Duration, check func() error) error {
b.lockDB()
defer b.unlockDB()
@@ -448,14 +453,26 @@ func (b *Blockstore) onlineGC() error {
if nworkers < 2 {
nworkers = 2
}
if nworkers > 7 { // max out at 1 goroutine per badger level
nworkers = 7
}
err := b.db.Flatten(nworkers)
if err != nil {
return err
}
checkTick := time.NewTimer(checkFreq)
defer checkTick.Stop()
for err == nil {
err = b.db.RunValueLogGC(0.125)
select {
case <-ctx.Done():
err = ctx.Err()
case <-checkTick.C:
err = check()
checkTick.Reset(checkFreq)
default:
err = b.db.RunValueLogGC(threshold)
}
}
if err == badger.ErrNoRewrite {
@@ -468,7 +485,7 @@ func (b *Blockstore) onlineGC() error {
// CollectGarbage compacts and runs garbage collection on the value log;
// implements the BlockstoreGC trait
func (b *Blockstore) CollectGarbage(opts ...blockstore.BlockstoreGCOption) error {
func (b *Blockstore) CollectGarbage(ctx context.Context, opts ...blockstore.BlockstoreGCOption) error {
if err := b.access(); err != nil {
return err
}
@@ -485,8 +502,58 @@ func (b *Blockstore) CollectGarbage(opts ...blockstore.BlockstoreGCOption) error
if options.FullGC {
return b.movingGC()
}
threshold := options.Threshold
if threshold == 0 {
threshold = defaultGCThreshold
}
checkFreq := options.CheckFreq
if checkFreq < 30*time.Second { // disallow checking more frequently than block time
checkFreq = 30 * time.Second
}
check := options.Check
if check == nil {
check = func() error {
return nil
}
}
return b.onlineGC(ctx, threshold, checkFreq, check)
}
return b.onlineGC()
// GCOnce runs garbage collection on the value log;
// implements BlockstoreGCOnce trait
func (b *Blockstore) GCOnce(ctx context.Context, opts ...blockstore.BlockstoreGCOption) error {
if err := b.access(); err != nil {
return err
}
defer b.viewers.Done()
var options blockstore.BlockstoreGCOptions
for _, opt := range opts {
err := opt(&options)
if err != nil {
return err
}
}
if options.FullGC {
return xerrors.Errorf("FullGC option specified for GCOnce but full GC is non incremental")
}
threshold := options.Threshold
if threshold == 0 {
threshold = defaultGCThreshold
}
b.lockDB()
defer b.unlockDB()
// Note no compaction needed before single GC as we will hit at most one vlog anyway
err := b.db.RunValueLogGC(threshold)
if err == badger.ErrNoRewrite {
// not really an error in this case, it signals the end of GC
return nil
}
return err
}
// Size returns the aggregate size of the blockstore
@@ -551,6 +618,18 @@ func (b *Blockstore) View(ctx context.Context, cid cid.Cid, fn func([]byte) erro
})
}
func (b *Blockstore) Flush(context.Context) error {
if err := b.access(); err != nil {
return err
}
defer b.viewers.Done()
b.lockDB()
defer b.unlockDB()
return b.db.Sync()
}
// Has implements Blockstore.Has.
func (b *Blockstore) Has(ctx context.Context, cid cid.Cid) (bool, error) {
if err := b.access(); err != nil {
+2 -2
View File
@@ -145,7 +145,7 @@ func testMove(t *testing.T, optsF func(string) Options) {
return nil
})
g.Go(func() error {
return db.CollectGarbage(blockstore.WithFullGC(true))
return db.CollectGarbage(ctx, blockstore.WithFullGC(true))
})
err = g.Wait()
@@ -230,7 +230,7 @@ func testMove(t *testing.T, optsF func(string) Options) {
checkPath()
// now do another FullGC to test the double move and following of symlinks
if err := db.CollectGarbage(blockstore.WithFullGC(true)); err != nil {
if err := db.CollectGarbage(ctx, blockstore.WithFullGC(true)); err != nil {
t.Fatal(err)
}
+46 -1
View File
@@ -2,6 +2,7 @@ package blockstore
import (
"context"
"time"
"github.com/ipfs/go-cid"
ds "github.com/ipfs/go-datastore"
@@ -18,6 +19,7 @@ type Blockstore interface {
blockstore.Blockstore
blockstore.Viewer
BatchDeleter
Flusher
}
// BasicBlockstore is an alias to the original IPFS Blockstore.
@@ -25,6 +27,10 @@ type BasicBlockstore = blockstore.Blockstore
type Viewer = blockstore.Viewer
type Flusher interface {
Flush(context.Context) error
}
type BatchDeleter interface {
DeleteMany(ctx context.Context, cids []cid.Cid) error
}
@@ -36,7 +42,12 @@ type BlockstoreIterator interface {
// BlockstoreGC is a trait for blockstores that support online garbage collection
type BlockstoreGC interface {
CollectGarbage(options ...BlockstoreGCOption) error
CollectGarbage(ctx context.Context, options ...BlockstoreGCOption) error
}
// BlockstoreGCOnce is a trait for a blockstore that supports incremental online garbage collection
type BlockstoreGCOnce interface {
GCOnce(ctx context.Context, options ...BlockstoreGCOption) error
}
// BlockstoreGCOption is a functional interface for controlling blockstore GC options
@@ -45,6 +56,12 @@ type BlockstoreGCOption = func(*BlockstoreGCOptions) error
// BlockstoreGCOptions is a struct with GC options
type BlockstoreGCOptions struct {
FullGC bool
// fraction of garbage in badger vlog before its worth processing in online GC
Threshold float64
// how often to call the check function
CheckFreq time.Duration
// function to call periodically to pause or early terminate GC
Check func() error
}
func WithFullGC(fullgc bool) BlockstoreGCOption {
@@ -54,6 +71,27 @@ func WithFullGC(fullgc bool) BlockstoreGCOption {
}
}
func WithThreshold(threshold float64) BlockstoreGCOption {
return func(opts *BlockstoreGCOptions) error {
opts.Threshold = threshold
return nil
}
}
func WithCheckFreq(f time.Duration) BlockstoreGCOption {
return func(opts *BlockstoreGCOptions) error {
opts.CheckFreq = f
return nil
}
}
func WithCheck(check func() error) BlockstoreGCOption {
return func(opts *BlockstoreGCOptions) error {
opts.Check = check
return nil
}
}
// BlockstoreSize is a trait for on-disk blockstores that can report their size
type BlockstoreSize interface {
Size() (int64, error)
@@ -92,6 +130,13 @@ type adaptedBlockstore struct {
var _ Blockstore = (*adaptedBlockstore)(nil)
func (a *adaptedBlockstore) Flush(ctx context.Context) error {
if flusher, canFlush := a.Blockstore.(Flusher); canFlush {
return flusher.Flush(ctx)
}
return nil
}
func (a *adaptedBlockstore) View(ctx context.Context, cid cid.Cid, callback func([]byte) error) error {
blk, err := a.Get(ctx, cid)
if err != nil {
+2
View File
@@ -46,6 +46,8 @@ var (
_ Viewer = (*BufferedBlockstore)(nil)
)
func (bs *BufferedBlockstore) Flush(ctx context.Context) error { return bs.write.Flush(ctx) }
func (bs *BufferedBlockstore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error) {
a, err := bs.read.AllKeysChan(ctx)
if err != nil {
+4
View File
@@ -38,6 +38,10 @@ func (b *discardstore) View(ctx context.Context, cid cid.Cid, f func([]byte) err
return b.bs.View(ctx, cid, f)
}
func (b *discardstore) Flush(ctx context.Context) error {
return nil
}
func (b *discardstore) Put(ctx context.Context, blk blocks.Block) error {
return nil
}
+4
View File
@@ -179,3 +179,7 @@ func (b *idstore) Close() error {
}
return nil
}
func (b *idstore) Flush(ctx context.Context) error {
return b.bs.Flush(ctx)
}
+2 -2
View File
@@ -3,7 +3,7 @@ package blockstore
import (
"bytes"
"context"
"io/ioutil"
"io"
"github.com/ipfs/go-cid"
httpapi "github.com/ipfs/go-ipfs-http-client"
@@ -103,7 +103,7 @@ func (i *IPFSBlockstore) Get(ctx context.Context, cid cid.Cid) (blocks.Block, er
return nil, xerrors.Errorf("getting ipfs block: %w", err)
}
data, err := ioutil.ReadAll(rd)
data, err := io.ReadAll(rd)
if err != nil {
return nil, err
}
+2
View File
@@ -17,6 +17,8 @@ func NewMemory() MemBlockstore {
// To match behavior of badger blockstore we index by multihash only.
type MemBlockstore map[string]blocks.Block
func (MemBlockstore) Flush(context.Context) error { return nil }
func (m MemBlockstore) DeleteBlock(ctx context.Context, k cid.Cid) error {
delete(m, string(k.Hash()))
return nil
+2
View File
@@ -410,6 +410,8 @@ func (n *NetworkStore) HashOnRead(enabled bool) {
return
}
func (*NetworkStore) Flush(context.Context) error { return nil }
func (n *NetworkStore) Stop(ctx context.Context) error {
close(n.closing)
+57
View File
@@ -115,6 +115,23 @@ type Config struct {
// A positive value is the number of compactions before a full GC is performed;
// a value of 1 will perform full GC in every compaction.
HotStoreFullGCFrequency uint64
// HotstoreMaxSpaceTarget suggests the max allowed space the hotstore can take.
// This is not a hard limit, it is possible for the hotstore to exceed the target
// for example if state grows massively between compactions. The splitstore
// will make a best effort to avoid overflowing the target and in practice should
// never overflow. This field is used when doing GC at the end of a compaction to
// adaptively choose moving GC
HotstoreMaxSpaceTarget uint64
// Moving GC will be triggered when total moving size exceeds
// HotstoreMaxSpaceTarget - HotstoreMaxSpaceThreshold
HotstoreMaxSpaceThreshold uint64
// Safety buffer to prevent moving GC from overflowing disk.
// Moving GC will not occur when total moving size exceeds
// HotstoreMaxSpaceTarget - HotstoreMaxSpaceSafetyBuffer
HotstoreMaxSpaceSafetyBuffer uint64
}
// ChainAccessor allows the Splitstore to access the chain. It will most likely
@@ -165,10 +182,16 @@ type SplitStore struct {
compactionIndex int64
pruneIndex int64
onlineGCCnt int64
ctx context.Context
cancel func()
outOfSync int32 // for fast checking
chainSyncMx sync.Mutex
chainSyncCond sync.Cond
chainSyncFinished bool // protected by chainSyncMx
debug *debugLog
// transactional protection for concurrent read/writes during compaction
@@ -195,6 +218,17 @@ type SplitStore struct {
// registered protectors
protectors []func(func(cid.Cid) error) error
// dag sizes measured during latest compaction
// logged and used for GC strategy
// protected by compaction lock
szWalk int64
szProtectedTxns int64
szKeys int64 // approximate, not counting keys protected when entering critical section
// protected by txnLk
szMarkedLiveRefs int64
}
var _ bstore.Blockstore = (*SplitStore)(nil)
@@ -232,6 +266,7 @@ func Open(path string, ds dstore.Datastore, hot, cold bstore.Blockstore, cfg *Co
ss.txnViewsCond.L = &ss.txnViewsMx
ss.txnSyncCond.L = &ss.txnSyncMx
ss.chainSyncCond.L = &ss.chainSyncMx
ss.ctx, ss.cancel = context.WithCancel(context.Background())
ss.reifyCond.L = &ss.reifyMx
@@ -447,6 +482,23 @@ func (s *SplitStore) GetSize(ctx context.Context, cid cid.Cid) (int, error) {
}
}
func (s *SplitStore) Flush(ctx context.Context) error {
s.txnLk.RLock()
defer s.txnLk.RUnlock()
if err := s.cold.Flush(ctx); err != nil {
return err
}
if err := s.hot.Flush(ctx); err != nil {
return err
}
if err := s.ds.Sync(ctx, dstore.Key{}); err != nil {
return err
}
return nil
}
func (s *SplitStore) Put(ctx context.Context, blk blocks.Block) error {
if isIdentiyCid(blk.Cid()) {
return nil
@@ -776,6 +828,11 @@ func (s *SplitStore) Close() error {
s.txnSyncCond.Broadcast()
s.txnSyncMx.Unlock()
s.chainSyncMx.Lock()
s.chainSyncFinished = true
s.chainSyncCond.Broadcast()
s.chainSyncMx.Unlock()
log.Warn("close with ongoing compaction in progress; waiting for it to finish...")
for atomic.LoadInt32(&s.compacting) == 1 {
time.Sleep(time.Second)
+2 -2
View File
@@ -95,7 +95,7 @@ func (s *SplitStore) doCheck(curTs *types.TipSet) error {
}
defer visitor.Close() //nolint
err = s.walkChain(curTs, boundaryEpoch, boundaryEpoch, visitor,
size := s.walkChain(curTs, boundaryEpoch, boundaryEpoch, visitor,
func(c cid.Cid) error {
if isUnitaryObject(c) {
return errStopWalk
@@ -133,7 +133,7 @@ func (s *SplitStore) doCheck(curTs *types.TipSet) error {
return err
}
log.Infow("check done", "cold", *coldCnt, "missing", *missingCnt)
log.Infow("check done", "cold", *coldCnt, "missing", *missingCnt, "walk size", size)
write("--")
write("cold: %d missing: %d", *coldCnt, *missingCnt)
write("DONE")
+161 -70
View File
@@ -66,7 +66,8 @@ var (
)
const (
batchSize = 16384
batchSize = 16384
cidKeySize = 128
)
func (s *SplitStore) HeadChange(_, apply []*types.TipSet) error {
@@ -90,7 +91,35 @@ func (s *SplitStore) HeadChange(_, apply []*types.TipSet) error {
// Regardless, we put a mutex in HeadChange just to be safe
if !atomic.CompareAndSwapInt32(&s.compacting, 0, 1) {
// we are currently compacting -- protect the new tipset(s)
// we are currently compacting
// 1. Signal sync condition to yield compaction when out of sync and resume when in sync
timestamp := time.Unix(int64(curTs.MinTimestamp()), 0)
if CheckSyncGap && time.Since(timestamp) > SyncGapTime {
/* Chain out of sync */
if atomic.CompareAndSwapInt32(&s.outOfSync, 0, 1) {
// transition from in sync to out of sync
s.chainSyncMx.Lock()
s.chainSyncFinished = false
s.chainSyncMx.Unlock()
}
// already out of sync, no signaling necessary
}
// TODO: ok to use hysteresis with no transitions between 30s and 1m?
if time.Since(timestamp) < SyncWaitTime {
/* Chain in sync */
if atomic.CompareAndSwapInt32(&s.outOfSync, 0, 0) {
// already in sync, no signaling necessary
} else {
// transition from out of sync to in sync
s.chainSyncMx.Lock()
s.chainSyncFinished = true
s.chainSyncCond.Broadcast()
s.chainSyncMx.Unlock()
}
}
// 2. protect the new tipset(s)
s.protectTipSets(apply)
return nil
}
@@ -199,9 +228,11 @@ func (s *SplitStore) markLiveRefs(cids []cid.Cid) {
log.Debugf("marking %d live refs", len(cids))
startMark := time.Now()
szMarked := new(int64)
count := new(int32)
visitor := newConcurrentVisitor()
walkObject := func(c cid.Cid) error {
walkObject := func(c cid.Cid) (int64, error) {
return s.walkObjectIncomplete(c, visitor,
func(c cid.Cid) error {
if isUnitaryObject(c) {
@@ -228,10 +259,12 @@ func (s *SplitStore) markLiveRefs(cids []cid.Cid) {
// optimize the common case of single put
if len(cids) == 1 {
if err := walkObject(cids[0]); err != nil {
sz, err := walkObject(cids[0])
if err != nil {
log.Errorf("error marking tipset refs: %s", err)
}
log.Debugw("marking live refs done", "took", time.Since(startMark), "marked", *count)
atomic.AddInt64(szMarked, sz)
return
}
@@ -243,9 +276,11 @@ func (s *SplitStore) markLiveRefs(cids []cid.Cid) {
worker := func() error {
for c := range workch {
if err := walkObject(c); err != nil {
sz, err := walkObject(c)
if err != nil {
return err
}
atomic.AddInt64(szMarked, sz)
}
return nil
@@ -268,7 +303,8 @@ func (s *SplitStore) markLiveRefs(cids []cid.Cid) {
log.Errorf("error marking tipset refs: %s", err)
}
log.Debugw("marking live refs done", "took", time.Since(startMark), "marked", *count)
log.Debugw("marking live refs done", "took", time.Since(startMark), "marked", *count, "size marked", *szMarked)
s.szMarkedLiveRefs += atomic.LoadInt64(szMarked)
}
// transactionally protect a view
@@ -361,6 +397,7 @@ func (s *SplitStore) protectTxnRefs(markSet MarkSet) error {
log.Infow("protecting transactional references", "refs", len(txnRefs))
count := 0
sz := new(int64)
workch := make(chan cid.Cid, len(txnRefs))
startProtect := time.Now()
@@ -393,10 +430,11 @@ func (s *SplitStore) protectTxnRefs(markSet MarkSet) error {
worker := func() error {
for c := range workch {
err := s.doTxnProtect(c, markSet)
szTxn, err := s.doTxnProtect(c, markSet)
if err != nil {
return xerrors.Errorf("error protecting transactional references to %s: %w", c, err)
}
atomic.AddInt64(sz, szTxn)
}
return nil
}
@@ -409,16 +447,16 @@ func (s *SplitStore) protectTxnRefs(markSet MarkSet) error {
if err := g.Wait(); err != nil {
return err
}
log.Infow("protecting transactional refs done", "took", time.Since(startProtect), "protected", count)
s.szProtectedTxns += atomic.LoadInt64(sz)
log.Infow("protecting transactional refs done", "took", time.Since(startProtect), "protected", count, "protected size", sz)
}
}
// transactionally protect a reference by walking the object and marking.
// concurrent markings are short circuited by checking the markset.
func (s *SplitStore) doTxnProtect(root cid.Cid, markSet MarkSet) error {
if err := s.checkClosing(); err != nil {
return err
func (s *SplitStore) doTxnProtect(root cid.Cid, markSet MarkSet) (int64, error) {
if err := s.checkYield(); err != nil {
return 0, err
}
// Note: cold objects are deleted heaviest first, so the consituents of an object
@@ -442,7 +480,7 @@ func (s *SplitStore) doTxnProtect(root cid.Cid, markSet MarkSet) error {
},
func(c cid.Cid) error {
if s.txnMissing != nil {
log.Warnf("missing object reference %s in %s", c, root)
log.Debugf("missing object reference %s in %s", c, root)
s.txnRefsMx.Lock()
s.txnMissing[c] = struct{}{}
s.txnRefsMx.Unlock()
@@ -509,6 +547,7 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
// might be potentially inconsistent; abort compaction and notify the user to intervene.
return xerrors.Errorf("checkpoint exists; aborting compaction")
}
s.clearSizeMeasurements()
currentEpoch := curTs.Height()
boundaryEpoch := currentEpoch - CompactionBoundary
@@ -534,7 +573,7 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
}
defer coldSet.Close() //nolint:errcheck
if err := s.checkClosing(); err != nil {
if err := s.checkYield(); err != nil {
return err
}
@@ -598,7 +637,6 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
}
err = s.walkChain(curTs, boundaryEpoch, inclMsgsEpoch, &noopVisitor{}, fHot, fCold)
if err != nil {
return xerrors.Errorf("error marking: %w", err)
}
@@ -607,7 +645,7 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
log.Infow("marking done", "took", time.Since(startMark), "marked", *count)
if err := s.checkClosing(); err != nil {
if err := s.checkYield(); err != nil {
return err
}
@@ -617,7 +655,7 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
return xerrors.Errorf("error protecting transactional refs: %w", err)
}
if err := s.checkClosing(); err != nil {
if err := s.checkYield(); err != nil {
return err
}
@@ -638,7 +676,7 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
defer purgew.Close() //nolint:errcheck
// some stats for logging
var hotCnt, coldCnt, purgeCnt int
var hotCnt, coldCnt, purgeCnt int64
err = s.hot.ForEachKey(func(c cid.Cid) error {
// was it marked?
mark, err := markSet.Has(c)
@@ -690,10 +728,11 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
log.Infow("cold collection done", "took", time.Since(startCollect))
log.Infow("compaction stats", "hot", hotCnt, "cold", coldCnt, "purge", purgeCnt)
stats.Record(s.ctx, metrics.SplitstoreCompactionHot.M(int64(hotCnt)))
stats.Record(s.ctx, metrics.SplitstoreCompactionCold.M(int64(coldCnt)))
s.szKeys = hotCnt * cidKeySize
stats.Record(s.ctx, metrics.SplitstoreCompactionHot.M(hotCnt))
stats.Record(s.ctx, metrics.SplitstoreCompactionCold.M(coldCnt))
if err := s.checkClosing(); err != nil {
if err := s.checkYield(); err != nil {
return err
}
@@ -702,7 +741,7 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
// possibly delete objects we didn't have when we were collecting cold objects)
s.waitForMissingRefs(markSet)
if err := s.checkClosing(); err != nil {
if err := s.checkYield(); err != nil {
return err
}
@@ -722,7 +761,7 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
}
log.Infow("moving done", "took", time.Since(startMove))
if err := s.checkClosing(); err != nil {
if err := s.checkYield(); err != nil {
return err
}
@@ -753,7 +792,7 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
}
// wait for the head to catch up so that the current tipset is marked
s.waitForSync()
s.waitForTxnSync()
if err := s.checkClosing(); err != nil {
return err
@@ -773,8 +812,8 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
return xerrors.Errorf("error purging cold objects: %w", err)
}
log.Infow("purging cold objects from hotstore done", "took", time.Since(startPurge))
s.endCriticalSection()
log.Infow("critical section done", "total protected size", s.szProtectedTxns, "total marked live size", s.szMarkedLiveRefs)
if err := checkpoint.Close(); err != nil {
log.Warnf("error closing checkpoint: %s", err)
@@ -794,7 +833,7 @@ func (s *SplitStore) doCompact(curTs *types.TipSet) error {
// we are done; do some housekeeping
s.endTxnProtect()
s.gcHotstore()
s.gcHotAfterCompaction()
err = s.setBaseEpoch(boundaryEpoch)
if err != nil {
@@ -854,7 +893,7 @@ func (s *SplitStore) beginCriticalSection(markSet MarkSet) error {
return nil
}
func (s *SplitStore) waitForSync() {
func (s *SplitStore) waitForTxnSync() {
log.Info("waiting for sync")
if !CheckSyncGap {
log.Warnf("If you see this outside of test it is a serious splitstore issue")
@@ -873,6 +912,25 @@ func (s *SplitStore) waitForSync() {
}
}
// Block compaction operations if chain sync has fallen behind
func (s *SplitStore) waitForSync() {
if atomic.LoadInt32(&s.outOfSync) == 0 {
return
}
s.chainSyncMx.Lock()
defer s.chainSyncMx.Unlock()
for !s.chainSyncFinished {
s.chainSyncCond.Wait()
}
}
// Combined sync and closing check
func (s *SplitStore) checkYield() error {
s.waitForSync()
return s.checkClosing()
}
func (s *SplitStore) endTxnProtect() {
s.txnLk.Lock()
defer s.txnLk.Unlock()
@@ -907,6 +965,7 @@ func (s *SplitStore) walkChain(ts *types.TipSet, inclState, inclMsgs abi.ChainEp
copy(toWalk, ts.Cids())
walkCnt := new(int64)
scanCnt := new(int64)
szWalk := new(int64)
tsRef := func(blkCids []cid.Cid) (cid.Cid, error) {
return types.NewTipSetKey(blkCids...).Cid()
@@ -942,48 +1001,64 @@ func (s *SplitStore) walkChain(ts *types.TipSet, inclState, inclMsgs abi.ChainEp
if err != nil {
return xerrors.Errorf("error computing cid reference to parent tipset")
}
if err := s.walkObjectIncomplete(pRef, visitor, fHot, stopWalk); err != nil {
sz, err := s.walkObjectIncomplete(pRef, visitor, fHot, stopWalk)
if err != nil {
return xerrors.Errorf("error walking parent tipset cid reference")
}
atomic.AddInt64(szWalk, sz)
// message are retained if within the inclMsgs boundary
if hdr.Height >= inclMsgs && hdr.Height > 0 {
if inclMsgs < inclState {
// we need to use walkObjectIncomplete here, as messages/receipts may be missing early on if we
// synced from snapshot and have a long HotStoreMessageRetentionPolicy.
if err := s.walkObjectIncomplete(hdr.Messages, visitor, fHot, stopWalk); err != nil {
sz, err := s.walkObjectIncomplete(hdr.Messages, visitor, fHot, stopWalk)
if err != nil {
return xerrors.Errorf("error walking messages (cid: %s): %w", hdr.Messages, err)
}
atomic.AddInt64(szWalk, sz)
if err := s.walkObjectIncomplete(hdr.ParentMessageReceipts, visitor, fHot, stopWalk); err != nil {
sz, err = s.walkObjectIncomplete(hdr.ParentMessageReceipts, visitor, fHot, stopWalk)
if err != nil {
return xerrors.Errorf("error walking messages receipts (cid: %s): %w", hdr.ParentMessageReceipts, err)
}
atomic.AddInt64(szWalk, sz)
} else {
if err := s.walkObject(hdr.Messages, visitor, fHot); err != nil {
sz, err = s.walkObject(hdr.Messages, visitor, fHot)
if err != nil {
return xerrors.Errorf("error walking messages (cid: %s): %w", hdr.Messages, err)
}
atomic.AddInt64(szWalk, sz)
if err := s.walkObject(hdr.ParentMessageReceipts, visitor, fHot); err != nil {
sz, err := s.walkObjectIncomplete(hdr.ParentMessageReceipts, visitor, fHot, stopWalk)
if err != nil {
return xerrors.Errorf("error walking message receipts (cid: %s): %w", hdr.ParentMessageReceipts, err)
}
atomic.AddInt64(szWalk, sz)
}
}
// messages and receipts outside of inclMsgs are included in the cold store
if hdr.Height < inclMsgs && hdr.Height > 0 {
if err := s.walkObjectIncomplete(hdr.Messages, visitor, fCold, stopWalk); err != nil {
sz, err := s.walkObjectIncomplete(hdr.Messages, visitor, fCold, stopWalk)
if err != nil {
return xerrors.Errorf("error walking messages (cid: %s): %w", hdr.Messages, err)
}
if err := s.walkObjectIncomplete(hdr.ParentMessageReceipts, visitor, fCold, stopWalk); err != nil {
atomic.AddInt64(szWalk, sz)
sz, err = s.walkObjectIncomplete(hdr.ParentMessageReceipts, visitor, fCold, stopWalk)
if err != nil {
return xerrors.Errorf("error walking messages receipts (cid: %s): %w", hdr.ParentMessageReceipts, err)
}
atomic.AddInt64(szWalk, sz)
}
// state is only retained if within the inclState boundary, with the exception of genesis
if hdr.Height >= inclState || hdr.Height == 0 {
if err := s.walkObject(hdr.ParentStateRoot, visitor, fHot); err != nil {
sz, err := s.walkObject(hdr.ParentStateRoot, visitor, fHot)
if err != nil {
return xerrors.Errorf("error walking state root (cid: %s): %w", hdr.ParentStateRoot, err)
}
atomic.AddInt64(szWalk, sz)
atomic.AddInt64(scanCnt, 1)
}
@@ -1001,13 +1076,15 @@ func (s *SplitStore) walkChain(ts *types.TipSet, inclState, inclMsgs abi.ChainEp
if err != nil {
return xerrors.Errorf("error computing cid reference to parent tipset")
}
if err := s.walkObjectIncomplete(hRef, visitor, fHot, stopWalk); err != nil {
sz, err := s.walkObjectIncomplete(hRef, visitor, fHot, stopWalk)
if err != nil {
return xerrors.Errorf("error walking parent tipset cid reference")
}
atomic.AddInt64(szWalk, sz)
for len(toWalk) > 0 {
// walking can take a while, so check this with every opportunity
if err := s.checkClosing(); err != nil {
if err := s.checkYield(); err != nil {
return err
}
@@ -1047,123 +1124,129 @@ func (s *SplitStore) walkChain(ts *types.TipSet, inclState, inclMsgs abi.ChainEp
}
}
log.Infow("chain walk done", "walked", *walkCnt, "scanned", *scanCnt)
log.Infow("chain walk done", "walked", *walkCnt, "scanned", *scanCnt, "walk size", szWalk)
s.szWalk = atomic.LoadInt64(szWalk)
return nil
}
func (s *SplitStore) walkObject(c cid.Cid, visitor ObjectVisitor, f func(cid.Cid) error) error {
func (s *SplitStore) walkObject(c cid.Cid, visitor ObjectVisitor, f func(cid.Cid) error) (int64, error) {
var sz int64
visit, err := visitor.Visit(c)
if err != nil {
return xerrors.Errorf("error visiting object: %w", err)
return 0, xerrors.Errorf("error visiting object: %w", err)
}
if !visit {
return nil
return sz, nil
}
if err := f(c); err != nil {
if err == errStopWalk {
return nil
return sz, nil
}
return err
return 0, err
}
if c.Prefix().Codec != cid.DagCBOR {
return nil
return sz, nil
}
// check this before recursing
if err := s.checkClosing(); err != nil {
return err
if err := s.checkYield(); err != nil {
return 0, err
}
var links []cid.Cid
err = s.view(c, func(data []byte) error {
sz += int64(len(data))
return cbg.ScanForLinks(bytes.NewReader(data), func(c cid.Cid) {
links = append(links, c)
})
})
if err != nil {
return xerrors.Errorf("error scanning linked block (cid: %s): %w", c, err)
return 0, xerrors.Errorf("error scanning linked block (cid: %s): %w", c, err)
}
for _, c := range links {
err := s.walkObject(c, visitor, f)
szLink, err := s.walkObject(c, visitor, f)
if err != nil {
return xerrors.Errorf("error walking link (cid: %s): %w", c, err)
return 0, xerrors.Errorf("error walking link (cid: %s): %w", c, err)
}
sz += szLink
}
return nil
return sz, nil
}
// like walkObject, but the object may be potentially incomplete (references missing)
func (s *SplitStore) walkObjectIncomplete(c cid.Cid, visitor ObjectVisitor, f, missing func(cid.Cid) error) error {
func (s *SplitStore) walkObjectIncomplete(c cid.Cid, visitor ObjectVisitor, f, missing func(cid.Cid) error) (int64, error) {
var sz int64
visit, err := visitor.Visit(c)
if err != nil {
return xerrors.Errorf("error visiting object: %w", err)
return 0, xerrors.Errorf("error visiting object: %w", err)
}
if !visit {
return nil
return sz, nil
}
// occurs check -- only for DAGs
if c.Prefix().Codec == cid.DagCBOR {
has, err := s.has(c)
if err != nil {
return xerrors.Errorf("error occur checking %s: %w", c, err)
return 0, xerrors.Errorf("error occur checking %s: %w", c, err)
}
if !has {
err = missing(c)
if err == errStopWalk {
return nil
return sz, nil
}
return err
return 0, err
}
}
if err := f(c); err != nil {
if err == errStopWalk {
return nil
return sz, nil
}
return err
return 0, err
}
if c.Prefix().Codec != cid.DagCBOR {
return nil
return sz, nil
}
// check this before recursing
if err := s.checkClosing(); err != nil {
return err
if err := s.checkYield(); err != nil {
return sz, err
}
var links []cid.Cid
err = s.view(c, func(data []byte) error {
sz += int64(len(data))
return cbg.ScanForLinks(bytes.NewReader(data), func(c cid.Cid) {
links = append(links, c)
})
})
if err != nil {
return xerrors.Errorf("error scanning linked block (cid: %s): %w", c, err)
return 0, xerrors.Errorf("error scanning linked block (cid: %s): %w", c, err)
}
for _, c := range links {
err := s.walkObjectIncomplete(c, visitor, f, missing)
szLink, err := s.walkObjectIncomplete(c, visitor, f, missing)
if err != nil {
return xerrors.Errorf("error walking link (cid: %s): %w", c, err)
return 0, xerrors.Errorf("error walking link (cid: %s): %w", c, err)
}
sz += szLink
}
return nil
return sz, nil
}
// internal version used during compaction and related operations
@@ -1226,7 +1309,7 @@ func (s *SplitStore) moveColdBlocks(coldr *ColdSetReader) error {
batch := make([]blocks.Block, 0, batchSize)
err := coldr.ForEach(func(c cid.Cid) error {
if err := s.checkClosing(); err != nil {
if err := s.checkYield(); err != nil {
return err
}
blk, err := s.hot.Get(s.ctx, c)
@@ -1429,8 +1512,9 @@ func (s *SplitStore) completeCompaction() error {
}
s.compactType = none
// Note: at this point we can start the splitstore; a compaction should run on
// the first head change, which will trigger gc on the hotstore.
// Note: at this point we can start the splitstore; base epoch is not
// incremented here so a compaction should run on the first head
// change, which will trigger gc on the hotstore.
// We don't mind the second (back-to-back) compaction as the head will
// have advanced during marking and coldset accumulation.
return nil
@@ -1488,6 +1572,13 @@ func (s *SplitStore) completePurge(coldr *ColdSetReader, checkpoint *Checkpoint,
return nil
}
func (s *SplitStore) clearSizeMeasurements() {
s.szKeys = 0
s.szMarkedLiveRefs = 0
s.szProtectedTxns = 0
s.szWalk = 0
}
// I really don't like having this code, but we seem to have some occasional DAG references with
// missing constituents. During testing in mainnet *some* of these references *sometimes* appeared
// after a little bit.
@@ -1528,7 +1619,7 @@ func (s *SplitStore) waitForMissingRefs(markSet MarkSet) {
missing = make(map[cid.Cid]struct{})
for c := range towalk {
err := s.walkObjectIncomplete(c, visitor,
_, err := s.walkObjectIncomplete(c, visitor,
func(c cid.Cid) error {
if isUnitaryObject(c) {
return errStopWalk
@@ -77,6 +77,10 @@ func (es *exposedSplitStore) GetSize(ctx context.Context, c cid.Cid) (int, error
return size, err
}
func (es *exposedSplitStore) Flush(ctx context.Context) error {
return es.s.Flush(ctx)
}
func (es *exposedSplitStore) Put(ctx context.Context, blk blocks.Block) error {
return es.s.Put(ctx, blk)
}
+70 -3
View File
@@ -7,23 +7,74 @@ import (
bstore "github.com/filecoin-project/lotus/blockstore"
)
func (s *SplitStore) gcHotstore() {
const (
// Fraction of garbage in badger vlog for online GC traversal to collect garbage
AggressiveOnlineGCThreshold = 0.0001
)
func (s *SplitStore) gcHotAfterCompaction() {
// Measure hotstore size, determine if we should do full GC, determine if we can do full GC.
// We should do full GC if
// FullGCFrequency is specified and compaction index matches frequency
// OR HotstoreMaxSpaceTarget is specified and total moving space is within 150 GB of target
// We can do full if
// HotstoreMaxSpaceTarget is not specified
// OR total moving space would not exceed 50 GB below target
//
// a) If we should not do full GC => online GC
// b) If we should do full GC and can => moving GC
// c) If we should do full GC and can't => aggressive online GC
getSize := func() int64 {
sizer, ok := s.hot.(bstore.BlockstoreSize)
if ok {
size, err := sizer.Size()
if err != nil {
log.Warnf("error getting hotstore size: %s, estimating empty hot store for targeting", err)
return 0
}
return size
}
log.Errorf("Could not measure hotstore size, assuming it is 0 bytes, which it is not")
return 0
}
hotSize := getSize()
copySizeApprox := s.szKeys + s.szMarkedLiveRefs + s.szProtectedTxns + s.szWalk
shouldTarget := s.cfg.HotstoreMaxSpaceTarget > 0 && hotSize+copySizeApprox > int64(s.cfg.HotstoreMaxSpaceTarget)-int64(s.cfg.HotstoreMaxSpaceThreshold)
shouldFreq := s.cfg.HotStoreFullGCFrequency > 0 && s.compactionIndex%int64(s.cfg.HotStoreFullGCFrequency) == 0
shouldDoFull := shouldTarget || shouldFreq
canDoFull := s.cfg.HotstoreMaxSpaceTarget == 0 || hotSize+copySizeApprox < int64(s.cfg.HotstoreMaxSpaceTarget)-int64(s.cfg.HotstoreMaxSpaceSafetyBuffer)
log.Debugw("approximating new hot store size", "key size", s.szKeys, "marked live refs", s.szMarkedLiveRefs, "protected txns", s.szProtectedTxns, "walked DAG", s.szWalk)
log.Infof("measured hot store size: %d, approximate new size: %d, should do full %t, can do full %t", hotSize, copySizeApprox, shouldDoFull, canDoFull)
var opts []bstore.BlockstoreGCOption
if s.cfg.HotStoreFullGCFrequency > 0 && s.compactionIndex%int64(s.cfg.HotStoreFullGCFrequency) == 0 {
if shouldDoFull && canDoFull {
opts = append(opts, bstore.WithFullGC(true))
} else if shouldDoFull && !canDoFull {
log.Warnf("Attention! Estimated moving GC size %d is not within safety buffer %d of target max %d, performing aggressive online GC to attempt to bring hotstore size down safely", copySizeApprox, s.cfg.HotstoreMaxSpaceSafetyBuffer, s.cfg.HotstoreMaxSpaceTarget)
log.Warn("If problem continues you can 1) temporarily allocate more disk space to hotstore and 2) reflect in HotstoreMaxSpaceTarget OR trigger manual move with `lotus chain prune hot-moving`")
log.Warn("If problem continues and you do not have any more disk space you can run continue to manually trigger online GC at aggressive thresholds (< 0.01) with `lotus chain prune hot`")
opts = append(opts, bstore.WithThreshold(AggressiveOnlineGCThreshold))
}
if err := s.gcBlockstore(s.hot, opts); err != nil {
log.Warnf("error garbage collecting hostore: %s", err)
}
log.Infof("measured hot store size after GC: %d", getSize())
}
func (s *SplitStore) gcBlockstore(b bstore.Blockstore, opts []bstore.BlockstoreGCOption) error {
if err := s.checkYield(); err != nil {
return err
}
if gc, ok := b.(bstore.BlockstoreGC); ok {
log.Info("garbage collecting blockstore")
startGC := time.Now()
if err := gc.CollectGarbage(opts...); err != nil {
opts = append(opts, bstore.WithCheckFreq(90*time.Second))
opts = append(opts, bstore.WithCheck(s.checkYield))
if err := gc.CollectGarbage(s.ctx, opts...); err != nil {
return err
}
@@ -33,3 +84,19 @@ func (s *SplitStore) gcBlockstore(b bstore.Blockstore, opts []bstore.BlockstoreG
return fmt.Errorf("blockstore doesn't support garbage collection: %T", b)
}
func (s *SplitStore) gcBlockstoreOnce(b bstore.Blockstore, opts []bstore.BlockstoreGCOption) error {
if gc, ok := b.(bstore.BlockstoreGCOnce); ok {
log.Debug("gc blockstore once")
startGC := time.Now()
if err := gc.GCOnce(s.ctx, opts...); err != nil {
return err
}
log.Debugw("gc blockstore once done", "took", time.Since(startGC))
return nil
}
return fmt.Errorf("blockstore doesn't support gc once: %T", b)
}
+17
View File
@@ -47,6 +47,23 @@ var (
PruneThreshold = 7 * build.Finality
)
// GCHotstore runs online GC on the chain state in the hotstore according the to options specified
func (s *SplitStore) GCHotStore(opts api.HotGCOpts) error {
if opts.Moving {
gcOpts := []bstore.BlockstoreGCOption{bstore.WithFullGC(true)}
return s.gcBlockstore(s.hot, gcOpts)
}
gcOpts := []bstore.BlockstoreGCOption{bstore.WithThreshold(opts.Threshold)}
var err error
if opts.Periodic {
err = s.gcBlockstore(s.hot, gcOpts)
} else {
err = s.gcBlockstoreOnce(s.hot, gcOpts)
}
return err
}
// PruneChain instructs the SplitStore to prune chain state in the coldstore, according to the
// options specified.
func (s *SplitStore) PruneChain(opts api.PruneOpts) error {
+1 -1
View File
@@ -101,7 +101,7 @@ func (s *SplitStore) doReify(c cid.Cid) {
defer s.txnLk.RUnlock()
count := 0
err := s.walkObjectIncomplete(c, newTmpVisitor(),
_, err := s.walkObjectIncomplete(c, newTmpVisitor(),
func(c cid.Cid) error {
if isUnitaryObject(c) {
return errStopWalk
+2
View File
@@ -757,6 +757,8 @@ func (b *mockStore) DeleteMany(_ context.Context, cids []cid.Cid) error {
return nil
}
func (b *mockStore) Flush(context.Context) error { return nil }
func (b *mockStore) AllKeysChan(ctx context.Context) (<-chan cid.Cid, error) {
return nil, errors.New("not implemented")
}
+2
View File
@@ -20,6 +20,8 @@ type SyncBlockstore struct {
bs MemBlockstore // specifically use a memStore to save indirection overhead.
}
func (*SyncBlockstore) Flush(context.Context) error { return nil }
func (m *SyncBlockstore) DeleteBlock(ctx context.Context, k cid.Cid) error {
m.mu.Lock()
defer m.mu.Unlock()
+10
View File
@@ -93,6 +93,16 @@ func (t *TimedCacheBlockstore) rotate() {
t.mu.Unlock()
}
func (t *TimedCacheBlockstore) Flush(ctx context.Context) error {
t.mu.Lock()
defer t.mu.Unlock()
if err := t.active.Flush(ctx); err != nil {
return err
}
return t.inactive.Flush(ctx)
}
func (t *TimedCacheBlockstore) Put(ctx context.Context, b blocks.Block) error {
// Don't check the inactive set here. We want to keep this block for at
// least one interval.
+9
View File
@@ -55,6 +55,15 @@ func (m unionBlockstore) GetSize(ctx context.Context, cid cid.Cid) (size int, er
return size, err
}
func (m unionBlockstore) Flush(ctx context.Context) (err error) {
for _, bs := range m {
if err = bs.Flush(ctx); err != nil {
break
}
}
return err
}
func (m unionBlockstore) Put(ctx context.Context, block blocks.Block) (err error) {
for _, bs := range m {
if err = bs.Put(ctx, block); err != nil {
-4
View File
@@ -69,10 +69,6 @@ var DrandConfigs = map[DrandEnum]dtypes.DrandConfig{
ChainInfoJSON: `{"public_key":"8cda589f88914aa728fd183f383980b35789ce81b274e5daee1f338b77d02566ef4d3fb0098af1f844f10f9c803c1827","period":25,"genesis_time":1595348225,"hash":"e73b7dc3c4f6a236378220c0dd6aa110eb16eed26c11259606e07ee122838d4f","groupHash":"567d4785122a5a3e75a9bc9911d7ea807dd85ff76b78dc4ff06b075712898607"}`,
},
DrandIncentinet: {
Servers: []string{
"https://dev1.drand.sh",
"https://dev2.drand.sh",
},
ChainInfoJSON: `{"public_key":"8cad0c72c606ab27d36ee06de1d5b2db1faf92e447025ca37575ab3a8aac2eaae83192f846fc9e158bc738423753d000","period":30,"genesis_time":1595873820,"hash":"80c8b872c714f4c00fdd3daa465d5514049f457f01f85a4caf68cdcd394ba039","groupHash":"d9406aaed487f7af71851b4399448e311f2328923d454e971536c05398ce2d9b"}`,
},
}
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
+1 -1
View File
@@ -37,7 +37,7 @@ func BuildTypeString() string {
}
// BuildVersion is the local build version
const BuildVersion = "1.21.0-dev"
const BuildVersion = "1.21.0-rc3"
func UserVersion() string {
if os.Getenv("LOTUS_VERSION_IGNORE_COMMIT") == "1" {
+12 -13
View File
@@ -4,7 +4,6 @@ import (
"bytes"
"fmt"
"go/format"
"io/ioutil"
"os"
"path/filepath"
"strconv"
@@ -66,7 +65,7 @@ func generateAdapters() error {
}
{
af, err := ioutil.ReadFile(filepath.Join(actDir, "actor.go.template"))
af, err := os.ReadFile(filepath.Join(actDir, "actor.go.template"))
if err != nil {
return xerrors.Errorf("loading actor template: %w", err)
}
@@ -90,7 +89,7 @@ func generateAdapters() error {
return err
}
if err := ioutil.WriteFile(filepath.Join(actDir, fmt.Sprintf("%s.go", act)), fmted, 0666); err != nil {
if err := os.WriteFile(filepath.Join(actDir, fmt.Sprintf("%s.go", act)), fmted, 0666); err != nil {
return err
}
}
@@ -100,7 +99,7 @@ func generateAdapters() error {
}
func generateState(actDir string, versions []int) error {
af, err := ioutil.ReadFile(filepath.Join(actDir, "state.go.template"))
af, err := os.ReadFile(filepath.Join(actDir, "state.go.template"))
if err != nil {
if os.IsNotExist(err) {
return nil // skip
@@ -123,7 +122,7 @@ func generateState(actDir string, versions []int) error {
return err
}
if err := ioutil.WriteFile(filepath.Join(actDir, fmt.Sprintf("v%d.go", version)), b.Bytes(), 0666); err != nil {
if err := os.WriteFile(filepath.Join(actDir, fmt.Sprintf("v%d.go", version)), b.Bytes(), 0666); err != nil {
return err
}
}
@@ -132,7 +131,7 @@ func generateState(actDir string, versions []int) error {
}
func generateMessages(actDir string) error {
af, err := ioutil.ReadFile(filepath.Join(actDir, "message.go.template"))
af, err := os.ReadFile(filepath.Join(actDir, "message.go.template"))
if err != nil {
if os.IsNotExist(err) {
return nil // skip
@@ -155,7 +154,7 @@ func generateMessages(actDir string) error {
return err
}
if err := ioutil.WriteFile(filepath.Join(actDir, fmt.Sprintf("message%d.go", version)), b.Bytes(), 0666); err != nil {
if err := os.WriteFile(filepath.Join(actDir, fmt.Sprintf("message%d.go", version)), b.Bytes(), 0666); err != nil {
return err
}
}
@@ -165,7 +164,7 @@ func generateMessages(actDir string) error {
func generatePolicy(policyPath string) error {
pf, err := ioutil.ReadFile(policyPath + ".template")
pf, err := os.ReadFile(policyPath + ".template")
if err != nil {
if os.IsNotExist(err) {
return nil // skip
@@ -187,7 +186,7 @@ func generatePolicy(policyPath string) error {
return err
}
if err := ioutil.WriteFile(policyPath, b.Bytes(), 0666); err != nil {
if err := os.WriteFile(policyPath, b.Bytes(), 0666); err != nil {
return err
}
@@ -196,7 +195,7 @@ func generatePolicy(policyPath string) error {
func generateBuiltin(builtinPath string) error {
bf, err := ioutil.ReadFile(builtinPath + ".template")
bf, err := os.ReadFile(builtinPath + ".template")
if err != nil {
if os.IsNotExist(err) {
return nil // skip
@@ -218,7 +217,7 @@ func generateBuiltin(builtinPath string) error {
return err
}
if err := ioutil.WriteFile(builtinPath, b.Bytes(), 0666); err != nil {
if err := os.WriteFile(builtinPath, b.Bytes(), 0666); err != nil {
return err
}
@@ -227,7 +226,7 @@ func generateBuiltin(builtinPath string) error {
func generateRegistry(registryPath string) error {
bf, err := ioutil.ReadFile(registryPath + ".template")
bf, err := os.ReadFile(registryPath + ".template")
if err != nil {
if os.IsNotExist(err) {
return nil // skip
@@ -248,7 +247,7 @@ func generateRegistry(registryPath string) error {
return err
}
if err := ioutil.WriteFile(registryPath, b.Bytes(), 0666); err != nil {
if err := os.WriteFile(registryPath, b.Bytes(), 0666); err != nil {
return err
}
@@ -83,6 +83,7 @@ type State interface {
GetAllocations(clientIdAddr address.Address) (map[AllocationId]Allocation, error)
GetClaim(providerIdAddr address.Address, claimId ClaimId) (*Claim, bool, error)
GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim, error)
GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error)
GetState() interface{}
}
@@ -170,6 +170,27 @@ func (s *state{{.v}}) GetClaims(providerIdAddr address.Address) (map[ClaimId]Cla
{{end}}
}
func (s *state{{.v}}) GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error) {
{{if (le .v 8)}}
return nil, xerrors.Errorf("unsupported in actors v{{.v}}")
{{else}}
v{{.v}}Map, err := s.LoadClaimsToMap(s.store, providerIdAddr)
retMap := make(map[abi.SectorNumber][]ClaimId)
for k, v := range v{{.v}}Map {
claims, ok := retMap[v.Sector]
if !ok {
retMap[v.Sector] = []ClaimId{ClaimId(k)}
} else {
retMap[v.Sector] = append(claims, ClaimId(k))
}
}
return retMap, err
{{end}}
}
func (s *state{{.v}}) ActorKey() string {
return manifest.VerifregKey
}
+6
View File
@@ -118,6 +118,12 @@ func (s *state0) GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim, e
}
func (s *state0) GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error) {
return nil, xerrors.Errorf("unsupported in actors v0")
}
func (s *state0) ActorKey() string {
return manifest.VerifregKey
}
+18
View File
@@ -134,6 +134,24 @@ func (s *state10) GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim,
}
func (s *state10) GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error) {
v10Map, err := s.LoadClaimsToMap(s.store, providerIdAddr)
retMap := make(map[abi.SectorNumber][]ClaimId)
for k, v := range v10Map {
claims, ok := retMap[v.Sector]
if !ok {
retMap[v.Sector] = []ClaimId{ClaimId(k)}
} else {
retMap[v.Sector] = append(claims, ClaimId(k))
}
}
return retMap, err
}
func (s *state10) ActorKey() string {
return manifest.VerifregKey
}
+6
View File
@@ -118,6 +118,12 @@ func (s *state2) GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim, e
}
func (s *state2) GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error) {
return nil, xerrors.Errorf("unsupported in actors v2")
}
func (s *state2) ActorKey() string {
return manifest.VerifregKey
}
+6
View File
@@ -119,6 +119,12 @@ func (s *state3) GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim, e
}
func (s *state3) GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error) {
return nil, xerrors.Errorf("unsupported in actors v3")
}
func (s *state3) ActorKey() string {
return manifest.VerifregKey
}
+6
View File
@@ -119,6 +119,12 @@ func (s *state4) GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim, e
}
func (s *state4) GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error) {
return nil, xerrors.Errorf("unsupported in actors v4")
}
func (s *state4) ActorKey() string {
return manifest.VerifregKey
}
+6
View File
@@ -119,6 +119,12 @@ func (s *state5) GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim, e
}
func (s *state5) GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error) {
return nil, xerrors.Errorf("unsupported in actors v5")
}
func (s *state5) ActorKey() string {
return manifest.VerifregKey
}
+6
View File
@@ -119,6 +119,12 @@ func (s *state6) GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim, e
}
func (s *state6) GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error) {
return nil, xerrors.Errorf("unsupported in actors v6")
}
func (s *state6) ActorKey() string {
return manifest.VerifregKey
}
+6
View File
@@ -118,6 +118,12 @@ func (s *state7) GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim, e
}
func (s *state7) GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error) {
return nil, xerrors.Errorf("unsupported in actors v7")
}
func (s *state7) ActorKey() string {
return manifest.VerifregKey
}
+6
View File
@@ -118,6 +118,12 @@ func (s *state8) GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim, e
}
func (s *state8) GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error) {
return nil, xerrors.Errorf("unsupported in actors v8")
}
func (s *state8) ActorKey() string {
return manifest.VerifregKey
}
+18
View File
@@ -133,6 +133,24 @@ func (s *state9) GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim, e
}
func (s *state9) GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error) {
v9Map, err := s.LoadClaimsToMap(s.store, providerIdAddr)
retMap := make(map[abi.SectorNumber][]ClaimId)
for k, v := range v9Map {
claims, ok := retMap[v.Sector]
if !ok {
retMap[v.Sector] = []ClaimId{ClaimId(k)}
} else {
retMap[v.Sector] = append(claims, ClaimId(k))
}
}
return retMap, err
}
func (s *state9) ActorKey() string {
return manifest.VerifregKey
}
+1
View File
@@ -131,6 +131,7 @@ type State interface {
GetAllocations(clientIdAddr address.Address) (map[AllocationId]Allocation, error)
GetClaim(providerIdAddr address.Address, claimId ClaimId) (*Claim, bool, error)
GetClaims(providerIdAddr address.Address) (map[ClaimId]Claim, error)
GetClaimIdsBySector(providerIdAddr address.Address) (map[abi.SectorNumber][]ClaimId, error)
GetState() interface{}
}
+4 -9
View File
@@ -3,14 +3,14 @@ package chain
import (
"fmt"
lru "github.com/hashicorp/golang-lru"
lru "github.com/hashicorp/golang-lru/v2"
"github.com/ipfs/go-cid"
"github.com/filecoin-project/lotus/build"
)
type BadBlockCache struct {
badBlocks *lru.ARCCache
badBlocks *lru.ARCCache[cid.Cid, BadBlockReason]
}
type BadBlockReason struct {
@@ -43,7 +43,7 @@ func (bbr BadBlockReason) String() string {
}
func NewBadBlockCache() *BadBlockCache {
cache, err := lru.NewARC(build.BadBlockCacheSize)
cache, err := lru.NewARC[cid.Cid, BadBlockReason](build.BadBlockCacheSize)
if err != nil {
panic(err) // ok
}
@@ -66,10 +66,5 @@ func (bts *BadBlockCache) Purge() {
}
func (bts *BadBlockCache) Has(c cid.Cid) (BadBlockReason, bool) {
rval, ok := bts.badBlocks.Get(c)
if !ok {
return BadBlockReason{}, false
}
return rval.(BadBlockReason), true
return bts.badBlocks.Get(c)
}
+22 -17
View File
@@ -8,14 +8,14 @@ import (
dchain "github.com/drand/drand/chain"
dclient "github.com/drand/drand/client"
hclient "github.com/drand/drand/client/http"
"github.com/drand/drand/common/scheme"
dlog "github.com/drand/drand/log"
gclient "github.com/drand/drand/lp2p/client"
"github.com/drand/kyber"
kzap "github.com/go-kit/kit/log/zap"
lru "github.com/hashicorp/golang-lru"
lru "github.com/hashicorp/golang-lru/v2"
logging "github.com/ipfs/go-log/v2"
pubsub "github.com/libp2p/go-libp2p-pubsub"
"go.uber.org/zap/zapcore"
"go.uber.org/zap"
"golang.org/x/xerrors"
"github.com/filecoin-project/go-state-types/abi"
@@ -61,7 +61,7 @@ type DrandBeacon struct {
filGenTime uint64
filRoundTime uint64
localCache *lru.Cache
localCache *lru.Cache[uint64, *types.BeaconEntry]
}
// DrandHTTPClient interface overrides the user agent used by drand
@@ -69,6 +69,18 @@ type DrandHTTPClient interface {
SetUserAgent(string)
}
type logger struct {
*zap.SugaredLogger
}
func (l *logger) With(args ...interface{}) dlog.Logger {
return &logger{l.SugaredLogger.With(args...)}
}
func (l *logger) Named(s string) dlog.Logger {
return &logger{l.SugaredLogger.Named(s)}
}
func NewDrandBeacon(genesisTs, interval uint64, ps *pubsub.PubSub, config dtypes.DrandConfig) (*DrandBeacon, error) {
if genesisTs == 0 {
panic("what are you doing this cant be zero")
@@ -79,9 +91,6 @@ func NewDrandBeacon(genesisTs, interval uint64, ps *pubsub.PubSub, config dtypes
return nil, xerrors.Errorf("unable to unmarshal drand chain info: %w", err)
}
dlogger := dlog.NewKitLoggerFrom(kzap.NewZapSugarLogger(
log.SugaredLogger.Desugar(), zapcore.InfoLevel))
var clients []dclient.Client
for _, url := range config.Servers {
hc, err := hclient.NewWithInfo(url, drandChain, nil)
@@ -96,7 +105,7 @@ func NewDrandBeacon(genesisTs, interval uint64, ps *pubsub.PubSub, config dtypes
opts := []dclient.Option{
dclient.WithChainInfo(drandChain),
dclient.WithCacheSize(1024),
dclient.WithLogger(dlogger),
dclient.WithLogger(&logger{&log.SugaredLogger}),
}
if ps != nil {
@@ -110,7 +119,7 @@ func NewDrandBeacon(genesisTs, interval uint64, ps *pubsub.PubSub, config dtypes
return nil, xerrors.Errorf("creating drand client: %w", err)
}
lc, err := lru.New(1024)
lc, err := lru.New[uint64, *types.BeaconEntry](1024)
if err != nil {
return nil, err
}
@@ -160,16 +169,12 @@ func (db *DrandBeacon) Entry(ctx context.Context, round uint64) <-chan beacon.Re
return out
}
func (db *DrandBeacon) cacheValue(e types.BeaconEntry) {
db.localCache.Add(e.Round, e)
db.localCache.Add(e.Round, &e)
}
func (db *DrandBeacon) getCachedValue(round uint64) *types.BeaconEntry {
v, ok := db.localCache.Get(round)
if !ok {
return nil
}
e, _ := v.(types.BeaconEntry)
return &e
v, _ := db.localCache.Get(round)
return v
}
func (db *DrandBeacon) VerifyEntry(curr types.BeaconEntry, prev types.BeaconEntry) error {
@@ -194,7 +199,7 @@ func (db *DrandBeacon) VerifyEntry(curr types.BeaconEntry, prev types.BeaconEntr
Round: curr.Round,
Signature: curr.Data,
}
err := dchain.VerifyBeacon(db.pubkey, b)
err := dchain.NewVerifier(scheme.GetSchemeFromEnv()).VerifyBeacon(*b, db.pubkey)
if err == nil {
db.cacheValue(curr)
}
+4 -3
View File
@@ -3,6 +3,7 @@
package drand
import (
"context"
"os"
"testing"
@@ -20,11 +21,11 @@ func TestPrintGroupInfo(t *testing.T) {
c, err := hclient.New(server, nil, nil)
assert.NoError(t, err)
cg := c.(interface {
FetchChainInfo(groupHash []byte) (*dchain.Info, error)
FetchChainInfo(ctx context.Context, groupHash []byte) (*dchain.Info, error)
})
chain, err := cg.FetchChainInfo(nil)
chain, err := cg.FetchChainInfo(context.Background(), nil)
assert.NoError(t, err)
err = chain.ToJSON(os.Stdout)
err = chain.ToJSON(os.Stdout, nil)
assert.NoError(t, err)
}
+5 -7
View File
@@ -5,7 +5,7 @@ import (
"sync"
"time"
lru "github.com/hashicorp/golang-lru"
lru "github.com/hashicorp/golang-lru/v2"
"github.com/libp2p/go-libp2p/core/peer"
"github.com/filecoin-project/lotus/build"
@@ -17,7 +17,7 @@ type blockReceiptTracker struct {
// using an LRU cache because i don't want to handle all the edge cases for
// manual cleanup and maintenance of a fixed size set
cache *lru.Cache
cache *lru.Cache[types.TipSetKey, *peerSet]
}
type peerSet struct {
@@ -25,7 +25,7 @@ type peerSet struct {
}
func newBlockReceiptTracker() *blockReceiptTracker {
c, _ := lru.New(512)
c, _ := lru.New[types.TipSetKey, *peerSet](512)
return &blockReceiptTracker{
cache: c,
}
@@ -46,20 +46,18 @@ func (brt *blockReceiptTracker) Add(p peer.ID, ts *types.TipSet) {
return
}
val.(*peerSet).peers[p] = build.Clock.Now()
val.peers[p] = build.Clock.Now()
}
func (brt *blockReceiptTracker) GetPeers(ts *types.TipSet) []peer.ID {
brt.lk.Lock()
defer brt.lk.Unlock()
val, ok := brt.cache.Get(ts.Key())
ps, ok := brt.cache.Get(ts.Key())
if !ok {
return nil
}
ps := val.(*peerSet)
out := make([]peer.ID, 0, len(ps.peers))
for p := range ps.peers {
out = append(out, p)
+28 -4
View File
@@ -3,6 +3,7 @@ package consensus
import (
"context"
"sync/atomic"
"time"
"github.com/ipfs/go-cid"
cbor "github.com/ipfs/go-ipld-cbor"
@@ -113,6 +114,8 @@ func (t *TipSetExecutor) ApplyBlocks(ctx context.Context,
return sm.VMConstructor()(ctx, vmopt)
}
var cronGas int64
runCron := func(vmCron vm.Interface, epoch abi.ChainEpoch) error {
cronMsg := &types.Message{
To: cron.Address,
@@ -130,6 +133,8 @@ func (t *TipSetExecutor) ApplyBlocks(ctx context.Context,
return xerrors.Errorf("running cron: %w", err)
}
cronGas += ret.GasUsed
if em != nil {
if err := em.MessageApplied(ctx, ts, cronMsg.Cid(), cronMsg, ret, true); err != nil {
return xerrors.Errorf("callback failed on cron message: %w", err)
@@ -181,7 +186,9 @@ func (t *TipSetExecutor) ApplyBlocks(ctx context.Context,
}
}
partDone()
vmEarly := partDone()
earlyCronGas := cronGas
cronGas = 0
partDone = metrics.Timer(ctx, metrics.VMApplyMessages)
vmi, err := makeVm(pstate, epoch, ts.MinTimestamp())
@@ -196,6 +203,8 @@ func (t *TipSetExecutor) ApplyBlocks(ctx context.Context,
processedMsgs = make(map[cid.Cid]struct{})
)
var msgGas int64
for _, b := range bms {
penalty := types.NewInt(0)
gasReward := big.Zero()
@@ -210,6 +219,8 @@ func (t *TipSetExecutor) ApplyBlocks(ctx context.Context,
return cid.Undef, cid.Undef, err
}
msgGas += r.GasUsed
receipts = append(receipts, &r.MessageReceipt)
gasReward = big.Add(gasReward, r.GasCosts.MinerTip)
penalty = big.Add(penalty, r.GasCosts.MinerPenalty)
@@ -235,18 +246,18 @@ func (t *TipSetExecutor) ApplyBlocks(ctx context.Context,
}
rErr := t.reward(ctx, vmi, em, epoch, ts, params)
if rErr != nil {
return cid.Undef, cid.Undef, xerrors.Errorf("error applying reward: %w", err)
return cid.Undef, cid.Undef, xerrors.Errorf("error applying reward: %w", rErr)
}
}
partDone()
vmMsg := partDone()
partDone = metrics.Timer(ctx, metrics.VMApplyCron)
if err := runCron(vmi, epoch); err != nil {
return cid.Cid{}, cid.Cid{}, err
}
partDone()
vmCron := partDone()
partDone = metrics.Timer(ctx, metrics.VMApplyFlush)
rectarr := blockadt.MakeEmptyArray(sm.ChainStore().ActorStore(ctx))
@@ -282,6 +293,11 @@ func (t *TipSetExecutor) ApplyBlocks(ctx context.Context,
return cid.Undef, cid.Undef, xerrors.Errorf("vm flush failed: %w", err)
}
vmFlush := partDone()
partDone = func() time.Duration { return time.Duration(0) }
log.Infow("ApplyBlocks stats", "early", vmEarly, "earlyCronGas", earlyCronGas, "vmMsg", vmMsg, "msgGas", msgGas, "vmCron", vmCron, "cronGas", cronGas, "vmFlush", vmFlush, "epoch", epoch, "tsk", ts.Key())
stats.Record(ctx, metrics.VMSends.M(int64(atomic.LoadUint64(&vm.StatSends))),
metrics.VMApplied.M(int64(atomic.LoadUint64(&vm.StatApplied))))
@@ -308,6 +324,14 @@ func (t *TipSetExecutor) ExecuteTipSet(ctx context.Context,
}
}
if ts.Height() == 0 {
// NB: This is here because the process that executes blocks requires that the
// block miner reference a valid miner in the state tree. Unless we create some
// magical genesis miner, this won't work properly, so we short circuit here
// This avoids the question of 'who gets paid the genesis block reward'
return blks[0].ParentStateRoot, blks[0].ParentMessageReceipts, nil
}
var parentEpoch abi.ChainEpoch
pstate := blks[0].ParentStateRoot
if blks[0].Height > 0 {
+10 -12
View File
@@ -517,12 +517,11 @@ func UpgradeFaucetBurnRecovery(ctx context.Context, sm *stmgr.StateManager, _ st
MessageReceipt: *stmgr.MakeFakeRct(),
ActorErr: nil,
ExecutionTrace: types.ExecutionTrace{
Msg: fakeMsg,
MsgRct: stmgr.MakeFakeRct(),
Error: "",
Duration: 0,
GasCharges: nil,
Subcalls: subcalls,
Msg: types.MessageTrace{
To: fakeMsg.To,
From: fakeMsg.From,
},
Subcalls: subcalls,
},
Duration: 0,
GasCosts: nil,
@@ -695,12 +694,11 @@ func splitGenesisMultisig0(ctx context.Context, em stmgr.ExecMonitor, addr addre
MessageReceipt: *stmgr.MakeFakeRct(),
ActorErr: nil,
ExecutionTrace: types.ExecutionTrace{
Msg: fakeMsg,
MsgRct: stmgr.MakeFakeRct(),
Error: "",
Duration: 0,
GasCharges: nil,
Subcalls: subcalls,
Msg: types.MessageTrace{
From: fakeMsg.From,
To: fakeMsg.To,
},
Subcalls: subcalls,
},
Duration: 0,
GasCosts: nil,
+9 -9
View File
@@ -4,7 +4,7 @@ import (
"context"
"sync"
lru "github.com/hashicorp/golang-lru"
lru "github.com/hashicorp/golang-lru/v2"
"github.com/ipfs/go-cid"
"github.com/filecoin-project/lotus/api"
@@ -14,14 +14,14 @@ type messageCache struct {
api EventAPI
blockMsgLk sync.Mutex
blockMsgCache *lru.ARCCache
blockMsgCache *lru.ARCCache[cid.Cid, *api.BlockMessages]
}
func newMessageCache(api EventAPI) *messageCache {
blsMsgCache, _ := lru.NewARC(500)
func newMessageCache(a EventAPI) *messageCache {
blsMsgCache, _ := lru.NewARC[cid.Cid, *api.BlockMessages](500)
return &messageCache{
api: api,
api: a,
blockMsgCache: blsMsgCache,
}
}
@@ -30,14 +30,14 @@ func (c *messageCache) ChainGetBlockMessages(ctx context.Context, blkCid cid.Cid
c.blockMsgLk.Lock()
defer c.blockMsgLk.Unlock()
msgsI, ok := c.blockMsgCache.Get(blkCid)
msgs, ok := c.blockMsgCache.Get(blkCid)
var err error
if !ok {
msgsI, err = c.api.ChainGetBlockMessages(ctx, blkCid)
msgs, err = c.api.ChainGetBlockMessages(ctx, blkCid)
if err != nil {
return nil, err
}
c.blockMsgCache.Add(blkCid, msgsI)
c.blockMsgCache.Add(blkCid, msgs)
}
return msgsI.(*api.BlockMessages), nil
return msgs, nil
}
+4 -4
View File
@@ -5,7 +5,7 @@ import (
"context"
"fmt"
"io"
"io/ioutil"
"os"
"sync/atomic"
"time"
@@ -167,7 +167,7 @@ func NewGeneratorWithSectorsAndUpgradeSchedule(numSectors int, us stmgr.UpgradeS
maddr1 := genesis2.MinerAddress(0)
m1temp, err := ioutil.TempDir("", "preseal")
m1temp, err := os.MkdirTemp("", "preseal")
if err != nil {
return nil, err
}
@@ -179,7 +179,7 @@ func NewGeneratorWithSectorsAndUpgradeSchedule(numSectors int, us stmgr.UpgradeS
maddr2 := genesis2.MinerAddress(1)
m2temp, err := ioutil.TempDir("", "preseal")
m2temp, err := os.MkdirTemp("", "preseal")
if err != nil {
return nil, err
}
@@ -256,7 +256,7 @@ func NewGeneratorWithSectorsAndUpgradeSchedule(numSectors int, us stmgr.UpgradeS
//return nil, xerrors.Errorf("creating drand beacon: %w", err)
//}
sm, err := stmgr.NewStateManager(cs, consensus.NewTipSetExecutor(filcns.RewardFunc), sys, us, beac)
sm, err := stmgr.NewStateManager(cs, consensus.NewTipSetExecutor(filcns.RewardFunc), sys, us, beac, ds)
if err != nil {
return nil, xerrors.Errorf("initing stmgr: %w", err)
}
+9 -5
View File
@@ -13,7 +13,11 @@ import (
)
var (
ReplaceByFeeRatioDefault = 1.25
ReplaceByFeePercentageMinimum types.Percent = 110
ReplaceByFeePercentageDefault types.Percent = 125
)
var (
MemPoolSizeLimitHiDefault = 30000
MemPoolSizeLimitLoDefault = 20000
PruneCooldownDefault = time.Minute
@@ -60,9 +64,9 @@ func (mp *MessagePool) getConfig() *types.MpoolConfig {
}
func validateConfg(cfg *types.MpoolConfig) error {
if cfg.ReplaceByFeeRatio < ReplaceByFeeRatioDefault {
return fmt.Errorf("'ReplaceByFeeRatio' is less than required %f < %f",
cfg.ReplaceByFeeRatio, ReplaceByFeeRatioDefault)
if cfg.ReplaceByFeeRatio < ReplaceByFeePercentageMinimum {
return fmt.Errorf("'ReplaceByFeeRatio' is less than required %s < %s",
cfg.ReplaceByFeeRatio, ReplaceByFeePercentageMinimum)
}
if cfg.GasLimitOverestimation < 1 {
return fmt.Errorf("'GasLimitOverestimation' cannot be less than 1")
@@ -91,7 +95,7 @@ func DefaultConfig() *types.MpoolConfig {
return &types.MpoolConfig{
SizeLimitHigh: MemPoolSizeLimitHiDefault,
SizeLimitLow: MemPoolSizeLimitLoDefault,
ReplaceByFeeRatio: ReplaceByFeeRatioDefault,
ReplaceByFeeRatio: ReplaceByFeePercentageDefault,
PruneCooldown: PruneCooldownDefault,
GasLimitOverestimation: GasLimitOverestimation,
}
+18 -19
View File
@@ -12,7 +12,7 @@ import (
"time"
"github.com/hashicorp/go-multierror"
lru "github.com/hashicorp/golang-lru"
lru "github.com/hashicorp/golang-lru/v2"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
"github.com/ipfs/go-datastore/namespace"
@@ -47,10 +47,8 @@ var log = logging.Logger("messagepool")
var futureDebug = false
var rbfNumBig = types.NewInt(uint64((ReplaceByFeeRatioDefault - 1) * RbfDenom))
var rbfDenomBig = types.NewInt(RbfDenom)
const RbfDenom = 256
var rbfNumBig = types.NewInt(uint64(ReplaceByFeePercentageMinimum))
var rbfDenomBig = types.NewInt(100)
var RepublishInterval = time.Duration(10*build.BlockDelaySecs+build.PropagationDelaySecs) * time.Second
@@ -161,7 +159,7 @@ type MessagePool struct {
// pruneCooldown is a channel used to allow a cooldown time between prunes
pruneCooldown chan struct{}
blsSigCache *lru.TwoQueueCache
blsSigCache *lru.TwoQueueCache[cid.Cid, crypto.Signature]
changes *lps.PubSub
@@ -169,9 +167,9 @@ type MessagePool struct {
netName dtypes.NetworkName
sigValCache *lru.TwoQueueCache
sigValCache *lru.TwoQueueCache[string, struct{}]
nonceCache *lru.Cache
nonceCache *lru.Cache[nonceCacheKey, uint64]
evtTypes [3]journal.EventType
journal journal.Journal
@@ -197,7 +195,13 @@ func newMsgSet(nonce uint64) *msgSet {
}
func ComputeMinRBF(curPrem abi.TokenAmount) abi.TokenAmount {
minPrice := types.BigAdd(curPrem, types.BigDiv(types.BigMul(curPrem, rbfNumBig), rbfDenomBig))
minPrice := types.BigDiv(types.BigMul(curPrem, rbfNumBig), rbfDenomBig)
return types.BigAdd(minPrice, types.NewInt(1))
}
func ComputeRBF(curPrem abi.TokenAmount, replaceByFeeRatio types.Percent) abi.TokenAmount {
rbfNumBig := types.NewInt(uint64(replaceByFeeRatio))
minPrice := types.BigDiv(types.BigMul(curPrem, rbfNumBig), rbfDenomBig)
return types.BigAdd(minPrice, types.NewInt(1))
}
@@ -365,9 +369,9 @@ func (ms *msgSet) toSlice() []*types.SignedMessage {
}
func New(ctx context.Context, api Provider, ds dtypes.MetadataDS, us stmgr.UpgradeSchedule, netName dtypes.NetworkName, j journal.Journal) (*MessagePool, error) {
cache, _ := lru.New2Q(build.BlsSignatureCacheSize)
verifcache, _ := lru.New2Q(build.VerifSigCacheSize)
noncecache, _ := lru.New(256)
cache, _ := lru.New2Q[cid.Cid, crypto.Signature](build.BlsSignatureCacheSize)
verifcache, _ := lru.New2Q[string, struct{}](build.VerifSigCacheSize)
noncecache, _ := lru.New[nonceCacheKey, uint64](256)
cfg, err := loadConfig(ctx, ds)
if err != nil {
@@ -1049,7 +1053,7 @@ func (mp *MessagePool) getStateNonce(ctx context.Context, addr address.Address,
n, ok := mp.nonceCache.Get(nk)
if ok {
return n.(uint64), nil
return n, nil
}
act, err := mp.api.GetActorAfter(addr, ts)
@@ -1469,15 +1473,10 @@ func (mp *MessagePool) MessagesForBlocks(ctx context.Context, blks []*types.Bloc
}
func (mp *MessagePool) RecoverSig(msg *types.Message) *types.SignedMessage {
val, ok := mp.blsSigCache.Get(msg.Cid())
sig, ok := mp.blsSigCache.Get(msg.Cid())
if !ok {
return nil
}
sig, ok := val.(crypto.Signature)
if !ok {
log.Errorf("value in signature cache was not a signature (got %T)", val)
return nil
}
return &types.SignedMessage{
Message: *msg,
+16 -6
View File
@@ -52,8 +52,8 @@ func (sm *StateManager) Call(ctx context.Context, msg *types.Message, ts *types.
}
// CallWithGas calculates the state for a given tipset, and then applies the given message on top of that state.
func (sm *StateManager) CallWithGas(ctx context.Context, msg *types.Message, priorMsgs []types.ChainMsg, ts *types.TipSet) (*api.InvocResult, error) {
return sm.callInternal(ctx, msg, priorMsgs, ts, cid.Undef, sm.GetNetworkVersion, true, true)
func (sm *StateManager) CallWithGas(ctx context.Context, msg *types.Message, priorMsgs []types.ChainMsg, ts *types.TipSet, applyTsMessages bool) (*api.InvocResult, error) {
return sm.callInternal(ctx, msg, priorMsgs, ts, cid.Undef, sm.GetNetworkVersion, true, applyTsMessages)
}
// CallAtStateAndVersion allows you to specify a message to execute on the given stateCid and network version.
@@ -117,12 +117,22 @@ func (sm *StateManager) callInternal(ctx context.Context, msg *types.Message, pr
if stateCid == cid.Undef {
stateCid = ts.ParentState()
}
tsMsgs, err := sm.cs.MessagesForTipset(ctx, ts)
if err != nil {
return nil, xerrors.Errorf("failed to lookup messages for parent tipset: %w", err)
}
if applyTsMessages {
tsMsgs, err := sm.cs.MessagesForTipset(ctx, ts)
if err != nil {
return nil, xerrors.Errorf("failed to lookup messages for parent tipset: %w", err)
}
priorMsgs = append(tsMsgs, priorMsgs...)
} else {
var filteredTsMsgs []types.ChainMsg
for _, tsMsg := range tsMsgs {
//TODO we should technically be normalizing the filecoin address of from when we compare here
if tsMsg.VMMessage().From == msg.VMMessage().From {
filteredTsMsgs = append(filteredTsMsgs, tsMsg)
}
}
priorMsgs = append(filteredTsMsgs, priorMsgs...)
}
// Technically, the tipset we're passing in here should be ts+1, but that may not exist.
+55 -1
View File
@@ -8,6 +8,7 @@ import (
"go.opencensus.io/trace"
"github.com/filecoin-project/lotus/api"
"github.com/filecoin-project/lotus/chain/store"
"github.com/filecoin-project/lotus/chain/types"
)
@@ -56,10 +57,18 @@ func (sm *StateManager) TipSetState(ctx context.Context, ts *types.TipSet) (st c
// NB: This is here because the process that executes blocks requires that the
// block miner reference a valid miner in the state tree. Unless we create some
// magical genesis miner, this won't work properly, so we short circuit here
// This avoids the question of 'who gets paid the genesis block reward'
// This avoids the question of 'who gets paid the genesis block reward'.
// This also makes us not attempt to lookup the tipset state with
// tryLookupTipsetState, which would cause a very long, very slow walk.
return ts.Blocks()[0].ParentStateRoot, ts.Blocks()[0].ParentMessageReceipts, nil
}
// First, try to find the tipset in the current chain. If found, we can avoid re-executing
// it.
if st, rec, found := tryLookupTipsetState(ctx, sm.cs, ts); found {
return st, rec, nil
}
st, rec, err = sm.tsExec.ExecuteTipSet(ctx, sm, ts, sm.tsExecMonitor, false)
if err != nil {
return cid.Undef, cid.Undef, err
@@ -68,6 +77,51 @@ func (sm *StateManager) TipSetState(ctx context.Context, ts *types.TipSet) (st c
return st, rec, nil
}
// Try to lookup a state & receipt CID for a given tipset by walking the chain instead of executing
// it. This will only successfully return the state/receipt CIDs if they're found in the state
// store.
//
// NOTE: This _won't_ recursively walk the receipt/state trees. It assumes that having the root
// implies having the rest of the tree. However, lotus generally makes that assumption anyways.
func tryLookupTipsetState(ctx context.Context, cs *store.ChainStore, ts *types.TipSet) (cid.Cid, cid.Cid, bool) {
nextTs, err := cs.GetTipsetByHeight(ctx, ts.Height()+1, nil, false)
if err != nil {
// Nothing to see here. The requested height may be beyond the current head.
return cid.Undef, cid.Undef, false
}
// Make sure we're on the correct fork.
if nextTs.Parents() != ts.Key() {
// Also nothing to see here. This just means that the requested tipset is on a
// different fork.
return cid.Undef, cid.Undef, false
}
stateCid := nextTs.ParentState()
receiptCid := nextTs.ParentMessageReceipts()
// Make sure we have the parent state.
if hasState, err := cs.StateBlockstore().Has(ctx, stateCid); err != nil {
log.Errorw("failed to lookup state-root in blockstore", "cid", stateCid, "error", err)
return cid.Undef, cid.Undef, false
} else if !hasState {
// We have the chain but don't have the state. It looks like we need to try
// executing?
return cid.Undef, cid.Undef, false
}
// Make sure we have the receipts.
if hasReceipts, err := cs.ChainBlockstore().Has(ctx, receiptCid); err != nil {
log.Errorw("failed to lookup receipts in blockstore", "cid", receiptCid, "error", err)
return cid.Undef, cid.Undef, false
} else if !hasReceipts {
// If we don't have the receipts, re-execute and try again.
return cid.Undef, cid.Undef, false
}
return stateCid, receiptCid, true
}
func (sm *StateManager) ExecutionTraceWithMonitor(ctx context.Context, ts *types.TipSet, em ExecMonitor) (cid.Cid, error) {
st, _, err := sm.tsExec.ExecuteTipSet(ctx, sm, ts, em, true)
return st, err
+18 -7
View File
@@ -174,9 +174,16 @@ func (us UpgradeSchedule) GetNtwkVersion(e abi.ChainEpoch) (network.Version, err
func (sm *StateManager) HandleStateForks(ctx context.Context, root cid.Cid, height abi.ChainEpoch, cb ExecMonitor, ts *types.TipSet) (cid.Cid, error) {
retCid := root
var err error
u := sm.stateMigrations[height]
if u != nil && u.upgrade != nil {
migCid, ok, err := u.migrationResultCache.Get(ctx, root)
if err == nil && ok {
log.Infow("CACHED migration", "height", height, "from", root, "to", migCid)
return migCid, nil
} else if err != nil {
log.Errorw("failed to lookup previous migration result", "err", err)
}
startTime := time.Now()
log.Warnw("STARTING migration", "height", height, "from", root)
// Yes, we clone the cache, even for the final upgrade epoch. Why? Reverts. We may
@@ -197,6 +204,11 @@ func (sm *StateManager) HandleStateForks(ctx context.Context, root cid.Cid, heig
"to", retCid,
"duration", time.Since(startTime),
)
// Only set if migration ran, we do not want a root => root mapping
if err := u.migrationResultCache.Store(ctx, root, retCid); err != nil {
log.Errorw("failed to store migration result", "err", err)
}
}
return retCid, nil
@@ -357,12 +369,11 @@ func DoTransfer(tree types.StateTree, from, to address.Address, amt abi.TokenAmo
// record the transfer in execution traces
cb(types.ExecutionTrace{
Msg: MakeFakeMsg(from, to, amt, 0),
MsgRct: MakeFakeRct(),
Error: "",
Duration: 0,
GasCharges: nil,
Subcalls: nil,
Msg: types.MessageTrace{
From: from,
To: to,
Value: amt,
},
})
}
+107 -5
View File
@@ -10,6 +10,7 @@ import (
"testing"
"github.com/ipfs/go-cid"
"github.com/ipfs/go-datastore"
ipldcbor "github.com/ipfs/go-ipld-cbor"
logging "github.com/ipfs/go-log/v2"
"github.com/stretchr/testify/require"
@@ -35,6 +36,7 @@ import (
"github.com/filecoin-project/lotus/chain/consensus"
"github.com/filecoin-project/lotus/chain/consensus/filcns"
"github.com/filecoin-project/lotus/chain/gen"
"github.com/filecoin-project/lotus/chain/stmgr"
. "github.com/filecoin-project/lotus/chain/stmgr"
"github.com/filecoin-project/lotus/chain/types"
"github.com/filecoin-project/lotus/chain/vm"
@@ -166,7 +168,7 @@ func TestForkHeightTriggers(t *testing.T) {
}
return st.Flush(ctx)
}}}, cg.BeaconSchedule())
}}}, cg.BeaconSchedule(), datastore.NewMapDatastore())
if err != nil {
t.Fatal(err)
}
@@ -284,7 +286,7 @@ func testForkRefuseCall(t *testing.T, nullsBefore, nullsAfter int) {
root cid.Cid, height abi.ChainEpoch, ts *types.TipSet) (cid.Cid, error) {
migrationCount++
return root, nil
}}}, cg.BeaconSchedule())
}}}, cg.BeaconSchedule(), datastore.NewMapDatastore())
if err != nil {
t.Fatal(err)
}
@@ -338,7 +340,7 @@ func testForkRefuseCall(t *testing.T, nullsBefore, nullsAfter int) {
currentHeight := ts.TipSet.TipSet().Height()
// CallWithGas calls on top of the given tipset.
ret, err := sm.CallWithGas(ctx, m, nil, ts.TipSet.TipSet())
ret, err := sm.CallWithGas(ctx, m, nil, ts.TipSet.TipSet(), true)
if parentHeight <= testForkHeight && currentHeight >= testForkHeight {
// If I had a fork, or I _will_ have a fork, it should fail.
require.Equal(t, ErrExpensiveFork, err)
@@ -359,7 +361,7 @@ func testForkRefuseCall(t *testing.T, nullsBefore, nullsAfter int) {
// Calls without a tipset should walk back to the last non-fork tipset.
// We _verify_ that the migration wasn't run multiple times at the end of the
// test.
ret, err = sm.CallWithGas(ctx, m, nil, nil)
ret, err = sm.CallWithGas(ctx, m, nil, nil, true)
require.NoError(t, err)
require.True(t, ret.MsgRct.ExitCode.IsSuccess())
@@ -502,7 +504,7 @@ func TestForkPreMigration(t *testing.T) {
return nil
},
}}},
}, cg.BeaconSchedule())
}, cg.BeaconSchedule(), datastore.NewMapDatastore())
if err != nil {
t.Fatal(err)
}
@@ -576,6 +578,7 @@ func TestDisablePreMigration(t *testing.T) {
}}},
},
cg.BeaconSchedule(),
datastore.NewMapDatastore(),
)
require.NoError(t, err)
require.NoError(t, sm.Start(context.Background()))
@@ -603,3 +606,102 @@ func TestDisablePreMigration(t *testing.T) {
require.Equal(t, 1, len(counter))
}
func TestMigrtionCache(t *testing.T) {
logging.SetAllLoggers(logging.LevelInfo)
cg, err := gen.NewGenerator()
require.NoError(t, err)
counter := make(chan struct{}, 10)
metadataDs := datastore.NewMapDatastore()
sm, err := NewStateManager(
cg.ChainStore(),
consensus.NewTipSetExecutor(filcns.RewardFunc),
cg.StateManager().VMSys(),
UpgradeSchedule{{
Network: network.Version1,
Height: testForkHeight,
Migration: func(_ context.Context, _ *StateManager, _ MigrationCache, _ ExecMonitor,
root cid.Cid, _ abi.ChainEpoch, _ *types.TipSet) (cid.Cid, error) {
counter <- struct{}{}
return root, nil
}},
},
cg.BeaconSchedule(),
metadataDs,
)
require.NoError(t, err)
require.NoError(t, sm.Start(context.Background()))
defer func() {
require.NoError(t, sm.Stop(context.Background()))
}()
inv := consensus.NewActorRegistry()
registry := builtin.MakeRegistryLegacy([]rtt.VMActor{testActor{}})
inv.Register(actorstypes.Version0, nil, registry)
sm.SetVMConstructor(func(ctx context.Context, vmopt *vm.VMOpts) (vm.Interface, error) {
nvm, err := vm.NewLegacyVM(ctx, vmopt)
require.NoError(t, err)
nvm.SetInvoker(inv)
return nvm, nil
})
cg.SetStateManager(sm)
for i := 0; i < 50; i++ {
_, err := cg.NextTipSet()
require.NoError(t, err)
}
ts, err := cg.ChainStore().GetTipsetByHeight(context.Background(), testForkHeight, nil, false)
require.NoError(t, err)
root, _, err := stmgr.ComputeState(context.Background(), sm, testForkHeight+1, []*types.Message{}, ts)
require.NoError(t, err)
t.Log(root)
require.Equal(t, 1, len(counter))
{
sm, err := NewStateManager(
cg.ChainStore(),
consensus.NewTipSetExecutor(filcns.RewardFunc),
cg.StateManager().VMSys(),
UpgradeSchedule{{
Network: network.Version1,
Height: testForkHeight,
Migration: func(_ context.Context, _ *StateManager, _ MigrationCache, _ ExecMonitor,
root cid.Cid, _ abi.ChainEpoch, _ *types.TipSet) (cid.Cid, error) {
counter <- struct{}{}
return root, nil
}},
},
cg.BeaconSchedule(),
metadataDs,
)
require.NoError(t, err)
sm.SetVMConstructor(func(ctx context.Context, vmopt *vm.VMOpts) (vm.Interface, error) {
nvm, err := vm.NewLegacyVM(ctx, vmopt)
require.NoError(t, err)
nvm.SetInvoker(inv)
return nvm, nil
})
ctx := context.Background()
base, _, err := sm.ExecutionTrace(ctx, ts)
require.NoError(t, err)
_, err = sm.HandleStateForks(context.Background(), base, ts.Height(), nil, ts)
require.NoError(t, err)
// Should not have increased as we should be using the cached results in the metadataDs
require.Equal(t, 1, len(counter))
}
}
+54 -8
View File
@@ -2,10 +2,13 @@ package stmgr
import (
"context"
"fmt"
"sync"
"github.com/ipfs/go-cid"
dstore "github.com/ipfs/go-datastore"
cbor "github.com/ipfs/go-ipld-cbor"
ipld "github.com/ipfs/go-ipld-format"
logging "github.com/ipfs/go-log/v2"
"golang.org/x/xerrors"
@@ -51,9 +54,47 @@ type versionSpec struct {
}
type migration struct {
upgrade MigrationFunc
preMigrations []PreMigration
cache *nv16.MemMigrationCache
upgrade MigrationFunc
preMigrations []PreMigration
cache *nv16.MemMigrationCache
migrationResultCache *migrationResultCache
}
type migrationResultCache struct {
ds dstore.Batching
keyPrefix string
}
func (m *migrationResultCache) keyForMigration(root cid.Cid) dstore.Key {
kStr := fmt.Sprintf("%s/%s", m.keyPrefix, root)
return dstore.NewKey(kStr)
}
func (m *migrationResultCache) Get(ctx context.Context, root cid.Cid) (cid.Cid, bool, error) {
k := m.keyForMigration(root)
bs, err := m.ds.Get(ctx, k)
if ipld.IsNotFound(err) {
return cid.Undef, false, nil
} else if err != nil {
return cid.Undef, false, xerrors.Errorf("error loading migration result: %w", err)
}
c, err := cid.Parse(bs)
if err != nil {
return cid.Undef, false, xerrors.Errorf("error parsing migration result: %w", err)
}
return c, true, nil
}
func (m *migrationResultCache) Store(ctx context.Context, root cid.Cid, resultCid cid.Cid) error {
k := m.keyForMigration(root)
if err := m.ds.Put(ctx, k, resultCid.Bytes()); err != nil {
return err
}
return nil
}
type Executor interface {
@@ -89,8 +130,7 @@ type StateManager struct {
postIgnitionVesting []msig0.State
postCalicoVesting []msig0.State
genesisPledge abi.TokenAmount
genesisMarketFunds abi.TokenAmount
genesisPledge abi.TokenAmount
tsExec Executor
tsExecMonitor ExecMonitor
@@ -103,7 +143,7 @@ type treeCache struct {
tree *state.StateTree
}
func NewStateManager(cs *store.ChainStore, exec Executor, sys vm.SyscallBuilder, us UpgradeSchedule, beacon beacon.Schedule) (*StateManager, error) {
func NewStateManager(cs *store.ChainStore, exec Executor, sys vm.SyscallBuilder, us UpgradeSchedule, beacon beacon.Schedule, metadataDs dstore.Batching) (*StateManager, error) {
// If we have upgrades, make sure they're in-order and make sense.
if err := us.Validate(); err != nil {
return nil, err
@@ -122,12 +162,18 @@ func NewStateManager(cs *store.ChainStore, exec Executor, sys vm.SyscallBuilder,
upgrade: upgrade.Migration,
preMigrations: upgrade.PreMigrations,
cache: nv16.NewMemMigrationCache(),
migrationResultCache: &migrationResultCache{
keyPrefix: fmt.Sprintf("/migration-cache/nv%d", upgrade.Network),
ds: metadataDs,
},
}
stateMigrations[upgrade.Height] = migration
}
if upgrade.Expensive {
expensiveUpgrades[upgrade.Height] = struct{}{}
}
networkVersions = append(networkVersions, versionSpec{
networkVersion: lastVersion,
atOrBelow: upgrade.Height,
@@ -155,8 +201,8 @@ func NewStateManager(cs *store.ChainStore, exec Executor, sys vm.SyscallBuilder,
}, nil
}
func NewStateManagerWithUpgradeScheduleAndMonitor(cs *store.ChainStore, exec Executor, sys vm.SyscallBuilder, us UpgradeSchedule, b beacon.Schedule, em ExecMonitor) (*StateManager, error) {
sm, err := NewStateManager(cs, exec, sys, us, b)
func NewStateManagerWithUpgradeScheduleAndMonitor(cs *store.ChainStore, exec Executor, sys vm.SyscallBuilder, us UpgradeSchedule, b beacon.Schedule, em ExecMonitor, metadataDs dstore.Batching) (*StateManager, error) {
sm, err := NewStateManager(cs, exec, sys, us, b, metadataDs)
if err != nil {
return nil, err
}
+10 -12
View File
@@ -51,17 +51,13 @@ func (sm *StateManager) setupGenesisVestingSchedule(ctx context.Context) error {
return xerrors.Errorf("loading state tree: %w", err)
}
gmf, err := getFilMarketLocked(ctx, sTree)
if err != nil {
return xerrors.Errorf("setting up genesis market funds: %w", err)
}
gp, err := getFilPowerLocked(ctx, sTree)
if err != nil {
return xerrors.Errorf("setting up genesis pledge: %w", err)
}
sm.genesisMarketFunds = gmf
sm.genesisMsigLk.Lock()
defer sm.genesisMsigLk.Unlock()
sm.genesisPledge = gp
totalsByEpoch := make(map[abi.ChainEpoch]abi.TokenAmount)
@@ -128,6 +124,8 @@ func (sm *StateManager) setupPostIgnitionVesting(ctx context.Context) error {
totalsByEpoch[sixYears] = big.NewInt(100_000_000)
totalsByEpoch[sixYears] = big.Add(totalsByEpoch[sixYears], big.NewInt(300_000_000))
sm.genesisMsigLk.Lock()
defer sm.genesisMsigLk.Unlock()
sm.postIgnitionVesting = make([]msig0.State, 0, len(totalsByEpoch))
for k, v := range totalsByEpoch {
ns := msig0.State{
@@ -178,6 +176,9 @@ func (sm *StateManager) setupPostCalicoVesting(ctx context.Context) error {
totalsByEpoch[sixYears] = big.Add(totalsByEpoch[sixYears], big.NewInt(300_000_000))
totalsByEpoch[sixYears] = big.Add(totalsByEpoch[sixYears], big.NewInt(9_805_053))
sm.genesisMsigLk.Lock()
defer sm.genesisMsigLk.Unlock()
sm.postCalicoVesting = make([]msig0.State, 0, len(totalsByEpoch))
for k, v := range totalsByEpoch {
ns := msig0.State{
@@ -198,21 +199,20 @@ func (sm *StateManager) setupPostCalicoVesting(ctx context.Context) error {
func (sm *StateManager) GetFilVested(ctx context.Context, height abi.ChainEpoch) (abi.TokenAmount, error) {
vf := big.Zero()
sm.genesisMsigLk.Lock()
defer sm.genesisMsigLk.Unlock()
// TODO: combine all this?
if sm.preIgnitionVesting == nil || sm.genesisPledge.IsZero() || sm.genesisMarketFunds.IsZero() {
if sm.preIgnitionVesting == nil || sm.genesisPledge.IsZero() {
err := sm.setupGenesisVestingSchedule(ctx)
if err != nil {
return vf, xerrors.Errorf("failed to setup pre-ignition vesting schedule: %w", err)
}
}
if sm.postIgnitionVesting == nil {
err := sm.setupPostIgnitionVesting(ctx)
if err != nil {
return vf, xerrors.Errorf("failed to setup post-ignition vesting schedule: %w", err)
}
}
if sm.postCalicoVesting == nil {
err := sm.setupPostCalicoVesting(ctx)
@@ -246,8 +246,6 @@ func (sm *StateManager) GetFilVested(ctx context.Context, height abi.ChainEpoch)
if height <= build.UpgradeAssemblyHeight {
// continue to use preIgnitionGenInfos, nothing changed at the Ignition epoch
vf = big.Add(vf, sm.genesisPledge)
// continue to use preIgnitionGenInfos, nothing changed at the Ignition epoch
vf = big.Add(vf, sm.genesisMarketFunds)
}
return vf, nil
+32 -24
View File
@@ -4,8 +4,8 @@ import (
"context"
"os"
"strconv"
"sync"
lru "github.com/hashicorp/golang-lru"
"golang.org/x/xerrors"
"github.com/filecoin-project/go-state-types/abi"
@@ -13,7 +13,7 @@ import (
"github.com/filecoin-project/lotus/chain/types"
)
var DefaultChainIndexCacheSize = 32 << 10
var DefaultChainIndexCacheSize = 32 << 15
func init() {
if s := os.Getenv("LOTUS_CHAIN_INDEX_CACHE"); s != "" {
@@ -27,7 +27,8 @@ func init() {
}
type ChainIndex struct {
skipCache *lru.ARCCache
indexCacheLk sync.Mutex
indexCache map[types.TipSetKey]*lbEntry
loadTipSet loadTipSetFunc
@@ -36,17 +37,14 @@ type ChainIndex struct {
type loadTipSetFunc func(context.Context, types.TipSetKey) (*types.TipSet, error)
func NewChainIndex(lts loadTipSetFunc) *ChainIndex {
sc, _ := lru.NewARC(DefaultChainIndexCacheSize)
return &ChainIndex{
skipCache: sc,
indexCache: make(map[types.TipSetKey]*lbEntry, DefaultChainIndexCacheSize),
loadTipSet: lts,
skipLength: 20,
}
}
type lbEntry struct {
ts *types.TipSet
parentHeight abi.ChainEpoch
targetHeight abi.ChainEpoch
target types.TipSetKey
}
@@ -58,25 +56,36 @@ func (ci *ChainIndex) GetTipsetByHeight(ctx context.Context, from *types.TipSet,
rounded, err := ci.roundDown(ctx, from)
if err != nil {
return nil, err
return nil, xerrors.Errorf("failed to round down: %w", err)
}
ci.indexCacheLk.Lock()
defer ci.indexCacheLk.Unlock()
cur := rounded.Key()
for {
cval, ok := ci.skipCache.Get(cur)
lbe, ok := ci.indexCache[cur]
if !ok {
fc, err := ci.fillCache(ctx, cur)
if err != nil {
return nil, err
return nil, xerrors.Errorf("failed to fill cache: %w", err)
}
cval = fc
lbe = fc
}
lbe := cval.(*lbEntry)
if lbe.ts.Height() == to || lbe.parentHeight < to {
return lbe.ts, nil
} else if to > lbe.targetHeight {
return ci.walkBack(ctx, lbe.ts, to)
if to == lbe.targetHeight {
ts, err := ci.loadTipSet(ctx, lbe.target)
if err != nil {
return nil, xerrors.Errorf("failed to load tipset: %w", err)
}
return ts, nil
}
if to > lbe.targetHeight {
ts, err := ci.loadTipSet(ctx, cur)
if err != nil {
return nil, xerrors.Errorf("failed to load tipset: %w", err)
}
return ci.walkBack(ctx, ts, to)
}
cur = lbe.target
@@ -87,16 +96,17 @@ func (ci *ChainIndex) GetTipsetByHeightWithoutCache(ctx context.Context, from *t
return ci.walkBack(ctx, from, to)
}
// Caller must hold indexCacheLk
func (ci *ChainIndex) fillCache(ctx context.Context, tsk types.TipSetKey) (*lbEntry, error) {
ts, err := ci.loadTipSet(ctx, tsk)
if err != nil {
return nil, err
return nil, xerrors.Errorf("failed to load tipset: %w", err)
}
if ts.Height() == 0 {
return &lbEntry{
ts: ts,
parentHeight: 0,
targetHeight: 0,
target: tsk,
}, nil
}
@@ -124,12 +134,10 @@ func (ci *ChainIndex) fillCache(ctx context.Context, tsk types.TipSetKey) (*lbEn
}
lbe := &lbEntry{
ts: ts,
parentHeight: parent.Height(),
targetHeight: skipTarget.Height(),
target: skipTarget.Key(),
}
ci.skipCache.Add(tsk, lbe)
ci.indexCache[tsk] = lbe
return lbe, nil
}
@@ -144,7 +152,7 @@ func (ci *ChainIndex) roundDown(ctx context.Context, ts *types.TipSet) (*types.T
rounded, err := ci.walkBack(ctx, ts, target)
if err != nil {
return nil, err
return nil, xerrors.Errorf("failed to walk back: %w", err)
}
return rounded, nil
@@ -164,7 +172,7 @@ func (ci *ChainIndex) walkBack(ctx context.Context, from *types.TipSet, to abi.C
for {
pts, err := ci.loadTipSet(ctx, ts.Parents())
if err != nil {
return nil, err
return nil, xerrors.Errorf("failed to load tipset: %w", err)
}
if to > pts.Height() {
+22 -4
View File
@@ -207,9 +207,7 @@ type mmCids struct {
}
func (cs *ChainStore) ReadMsgMetaCids(ctx context.Context, mmc cid.Cid) ([]cid.Cid, []cid.Cid, error) {
o, ok := cs.mmCache.Get(mmc)
if ok {
mmcids := o.(*mmCids)
if mmcids, ok := cs.mmCache.Get(mmc); ok {
return mmcids.bls, mmcids.secpk, nil
}
@@ -229,7 +227,7 @@ func (cs *ChainStore) ReadMsgMetaCids(ctx context.Context, mmc cid.Cid) ([]cid.C
return nil, nil, xerrors.Errorf("loading secpk message cids for block: %w", err)
}
cs.mmCache.Add(mmc, &mmCids{
cs.mmCache.Add(mmc, mmCids{
bls: blscids,
secpk: secpkcids,
})
@@ -237,6 +235,26 @@ func (cs *ChainStore) ReadMsgMetaCids(ctx context.Context, mmc cid.Cid) ([]cid.C
return blscids, secpkcids, nil
}
func (cs *ChainStore) ReadReceipts(ctx context.Context, root cid.Cid) ([]types.MessageReceipt, error) {
a, err := blockadt.AsArray(cs.ActorStore(ctx), root)
if err != nil {
return nil, err
}
receipts := make([]types.MessageReceipt, 0, a.Length())
var rcpt types.MessageReceipt
if err := a.ForEach(&rcpt, func(i int64) error {
if int64(len(receipts)) != i {
return xerrors.Errorf("missing receipt %d", i)
}
receipts = append(receipts, rcpt)
return nil
}); err != nil {
return nil, err
}
return receipts, nil
}
func (cs *ChainStore) MessagesForBlock(ctx context.Context, b *types.BlockHeader) ([]*types.Message, []*types.SignedMessage, error) {
blscids, secpkcids, err := cs.ReadMsgMetaCids(ctx, b.Messages)
if err != nil {
+7 -8
View File
@@ -11,7 +11,7 @@ import (
"sync"
"time"
lru "github.com/hashicorp/golang-lru"
lru "github.com/hashicorp/golang-lru/v2"
"github.com/ipfs/go-cid"
dstore "github.com/ipfs/go-datastore"
"github.com/ipfs/go-datastore/query"
@@ -120,8 +120,8 @@ type ChainStore struct {
reorgCh chan<- reorg
reorgNotifeeCh chan ReorgNotifee
mmCache *lru.ARCCache // msg meta cache (mh.Messages -> secp, bls []cid)
tsCache *lru.ARCCache
mmCache *lru.ARCCache[cid.Cid, mmCids]
tsCache *lru.ARCCache[types.TipSetKey, *types.TipSet]
evtTypes [1]journal.EventType
journal journal.Journal
@@ -133,8 +133,8 @@ type ChainStore struct {
}
func NewChainStore(chainBs bstore.Blockstore, stateBs bstore.Blockstore, ds dstore.Batching, weight WeightFunc, j journal.Journal) *ChainStore {
c, _ := lru.NewARC(DefaultMsgMetaCacheSize)
tsc, _ := lru.NewARC(DefaultTipSetCacheSize)
c, _ := lru.NewARC[cid.Cid, mmCids](DefaultMsgMetaCacheSize)
tsc, _ := lru.NewARC[types.TipSetKey, *types.TipSet](DefaultTipSetCacheSize)
if j == nil {
j = journal.NilJournal()
}
@@ -818,9 +818,8 @@ func (cs *ChainStore) GetBlock(ctx context.Context, c cid.Cid) (*types.BlockHead
}
func (cs *ChainStore) LoadTipSet(ctx context.Context, tsk types.TipSetKey) (*types.TipSet, error) {
v, ok := cs.tsCache.Get(tsk)
if ok {
return v.(*types.TipSet), nil
if ts, ok := cs.tsCache.Get(tsk); ok {
return ts, nil
}
// Fetch tipset block headers from blockstore in parallel
+3 -2
View File
@@ -196,7 +196,8 @@ func TestChainExportImportFull(t *testing.T) {
}
nbs := blockstore.NewMemorySync()
cs := store.NewChainStore(nbs, nbs, datastore.NewMapDatastore(), filcns.Weight, nil)
ds := datastore.NewMapDatastore()
cs := store.NewChainStore(nbs, nbs, ds, filcns.Weight, nil)
defer cs.Close() //nolint:errcheck
root, err := cs.Import(context.TODO(), buf)
@@ -213,7 +214,7 @@ func TestChainExportImportFull(t *testing.T) {
t.Fatal("imported chain differed from exported chain")
}
sm, err := stmgr.NewStateManager(cs, consensus.NewTipSetExecutor(filcns.RewardFunc), nil, filcns.DefaultUpgradeSchedule(), cg.BeaconSchedule())
sm, err := stmgr.NewStateManager(cs, consensus.NewTipSetExecutor(filcns.RewardFunc), nil, filcns.DefaultUpgradeSchedule(), cg.BeaconSchedule(), ds)
if err != nil {
t.Fatal(err)
}
+18 -23
View File
@@ -7,11 +7,12 @@ import (
"sync"
"time"
lru "github.com/hashicorp/golang-lru"
lru "github.com/hashicorp/golang-lru/v2"
bserv "github.com/ipfs/go-blockservice"
"github.com/ipfs/go-cid"
blocks "github.com/ipfs/go-libipfs/blocks"
logging "github.com/ipfs/go-log/v2"
"github.com/ipni/storetheindex/announce/message"
pubsub "github.com/libp2p/go-libp2p-pubsub"
"github.com/libp2p/go-libp2p/core/connmgr"
"github.com/libp2p/go-libp2p/core/peer"
@@ -20,7 +21,6 @@ import (
"golang.org/x/xerrors"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-legs/dtsync"
"github.com/filecoin-project/lotus/build"
"github.com/filecoin-project/lotus/chain"
@@ -217,7 +217,7 @@ func fetchCids(
type BlockValidator struct {
self peer.ID
peers *lru.TwoQueueCache
peers *lru.TwoQueueCache[peer.ID, int]
killThresh int
@@ -231,7 +231,7 @@ type BlockValidator struct {
}
func NewBlockValidator(self peer.ID, chain *store.ChainStore, cns consensus.Consensus, blacklist func(peer.ID)) *BlockValidator {
p, _ := lru.New2Q(4096)
p, _ := lru.New2Q[peer.ID, int](4096)
return &BlockValidator{
self: self,
peers: p,
@@ -244,21 +244,19 @@ func NewBlockValidator(self peer.ID, chain *store.ChainStore, cns consensus.Cons
}
func (bv *BlockValidator) flagPeer(p peer.ID) {
v, ok := bv.peers.Get(p)
val, ok := bv.peers.Get(p)
if !ok {
bv.peers.Add(p, int(1))
bv.peers.Add(p, 1)
return
}
val := v.(int)
if val >= bv.killThresh {
log.Warnf("blacklisting peer %s", p)
bv.blacklist(p)
return
}
bv.peers.Add(p, v.(int)+1)
bv.peers.Add(p, val+1)
}
func (bv *BlockValidator) Validate(ctx context.Context, pid peer.ID, msg *pubsub.Message) (res pubsub.ValidationResult) {
@@ -293,11 +291,11 @@ func (bv *BlockValidator) Validate(ctx context.Context, pid peer.ID, msg *pubsub
}
type blockReceiptCache struct {
blocks *lru.TwoQueueCache
blocks *lru.TwoQueueCache[cid.Cid, int]
}
func newBlockReceiptCache() *blockReceiptCache {
c, _ := lru.New2Q(8192)
c, _ := lru.New2Q[cid.Cid, int](8192)
return &blockReceiptCache{
blocks: c,
@@ -307,12 +305,12 @@ func newBlockReceiptCache() *blockReceiptCache {
func (brc *blockReceiptCache) add(bcid cid.Cid) int {
val, ok := brc.blocks.Get(bcid)
if !ok {
brc.blocks.Add(bcid, int(1))
brc.blocks.Add(bcid, 1)
return 0
}
brc.blocks.Add(bcid, val.(int)+1)
return val.(int)
brc.blocks.Add(bcid, val+1)
return val
}
type MessageValidator struct {
@@ -466,13 +464,13 @@ type peerMsgInfo struct {
type IndexerMessageValidator struct {
self peer.ID
peerCache *lru.TwoQueueCache
peerCache *lru.TwoQueueCache[address.Address, *peerMsgInfo]
chainApi full.ChainModuleAPI
stateApi full.StateModuleAPI
}
func NewIndexerMessageValidator(self peer.ID, chainApi full.ChainModuleAPI, stateApi full.StateModuleAPI) *IndexerMessageValidator {
peerCache, _ := lru.New2Q(8192)
peerCache, _ := lru.New2Q[address.Address, *peerMsgInfo](8192)
return &IndexerMessageValidator{
self: self,
@@ -497,7 +495,7 @@ func (v *IndexerMessageValidator) Validate(ctx context.Context, pid peer.ID, msg
return pubsub.ValidationIgnore
}
idxrMsg := dtsync.Message{}
idxrMsg := message.Message{}
err := idxrMsg.UnmarshalCBOR(bytes.NewBuffer(msg.Data))
if err != nil {
log.Errorw("Could not decode indexer pubsub message", "err", err)
@@ -515,15 +513,12 @@ func (v *IndexerMessageValidator) Validate(ctx context.Context, pid peer.ID, msg
return pubsub.ValidationReject
}
minerID := minerAddr.String()
msgCid := idxrMsg.Cid
var msgInfo *peerMsgInfo
val, ok := v.peerCache.Get(minerID)
msgInfo, ok := v.peerCache.Get(minerAddr)
if !ok {
msgInfo = &peerMsgInfo{}
} else {
msgInfo = val.(*peerMsgInfo)
}
// Lock this peer's message info.
@@ -544,7 +539,7 @@ func (v *IndexerMessageValidator) Validate(ctx context.Context, pid peer.ID, msg
// Check that the miner ID maps to the peer that sent the message.
err = v.authenticateMessage(ctx, minerAddr, originPeer)
if err != nil {
log.Warnw("cannot authenticate messsage", "err", err, "peer", originPeer, "minerID", minerID)
log.Warnw("cannot authenticate messsage", "err", err, "peer", originPeer, "minerID", minerAddr)
stats.Record(ctx, metrics.IndexerMessageValidationFailure.M(1))
return pubsub.ValidationReject
}
@@ -554,7 +549,7 @@ func (v *IndexerMessageValidator) Validate(ctx context.Context, pid peer.ID, msg
// messages from the same peer are handled concurrently, there is a
// small chance that one msgInfo could replace the other here when
// the info is first cached. This is OK, so no need to prevent it.
v.peerCache.Add(minerID, msgInfo)
v.peerCache.Add(minerAddr, msgInfo)
}
}
+2 -2
View File
@@ -9,12 +9,12 @@ import (
"github.com/golang/mock/gomock"
"github.com/ipfs/go-cid"
blocks "github.com/ipfs/go-libipfs/blocks"
"github.com/ipni/storetheindex/announce/message"
pubsub "github.com/libp2p/go-libp2p-pubsub"
pb "github.com/libp2p/go-libp2p-pubsub/pb"
"github.com/libp2p/go-libp2p/core/peer"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-legs/dtsync"
"github.com/filecoin-project/lotus/api/mocks"
"github.com/filecoin-project/lotus/chain/types"
@@ -105,7 +105,7 @@ func TestIndexerMessageValidator_Validate(t *testing.T) {
mc := gomock.NewController(t)
node := mocks.NewMockFullNode(mc)
subject := NewIndexerMessageValidator(peer.ID(tc.selfPID), node, node)
message := dtsync.Message{
message := message.Message{
Cid: validCid,
Addrs: nil,
ExtraData: tc.extraData,
+662 -4
View File
@@ -7,6 +7,7 @@ import (
"io"
"math"
"sort"
time "time"
cid "github.com/ipfs/go-cid"
cbg "github.com/whyrusleeping/cbor-gen"
@@ -15,6 +16,7 @@ import (
address "github.com/filecoin-project/go-address"
abi "github.com/filecoin-project/go-state-types/abi"
crypto "github.com/filecoin-project/go-state-types/crypto"
exitcode "github.com/filecoin-project/go-state-types/exitcode"
proof "github.com/filecoin-project/go-state-types/proof"
)
@@ -338,7 +340,7 @@ func (t *BlockHeader) UnmarshalCBOR(r io.Reader) (err error) {
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
@@ -616,7 +618,7 @@ func (t *ElectionProof) UnmarshalCBOR(r io.Reader) (err error) {
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
@@ -826,7 +828,7 @@ func (t *Message) UnmarshalCBOR(r io.Reader) (err error) {
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
@@ -1589,7 +1591,7 @@ func (t *ExpTipSet) UnmarshalCBOR(r io.Reader) (err error) {
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
@@ -2078,3 +2080,659 @@ func (t *EventEntry) UnmarshalCBOR(r io.Reader) (err error) {
}
return nil
}
var lengthBufGasTrace = []byte{133}
func (t *GasTrace) MarshalCBOR(w io.Writer) error {
if t == nil {
_, err := w.Write(cbg.CborNull)
return err
}
cw := cbg.NewCborWriter(w)
if _, err := cw.Write(lengthBufGasTrace); err != nil {
return err
}
// t.Name (string) (string)
if len(t.Name) > cbg.MaxLength {
return xerrors.Errorf("Value in field t.Name was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len(t.Name))); err != nil {
return err
}
if _, err := io.WriteString(w, string(t.Name)); err != nil {
return err
}
// t.TotalGas (int64) (int64)
if t.TotalGas >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.TotalGas)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.TotalGas-1)); err != nil {
return err
}
}
// t.ComputeGas (int64) (int64)
if t.ComputeGas >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.ComputeGas)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.ComputeGas-1)); err != nil {
return err
}
}
// t.StorageGas (int64) (int64)
if t.StorageGas >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.StorageGas)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.StorageGas-1)); err != nil {
return err
}
}
// t.TimeTaken (time.Duration) (int64)
if t.TimeTaken >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.TimeTaken)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.TimeTaken-1)); err != nil {
return err
}
}
return nil
}
func (t *GasTrace) UnmarshalCBOR(r io.Reader) (err error) {
*t = GasTrace{}
cr := cbg.NewCborReader(r)
maj, extra, err := cr.ReadHeader()
if err != nil {
return err
}
defer func() {
if err == io.EOF {
err = io.ErrUnexpectedEOF
}
}()
if maj != cbg.MajArray {
return fmt.Errorf("cbor input should be of type array")
}
if extra != 5 {
return fmt.Errorf("cbor input had wrong number of fields")
}
// t.Name (string) (string)
{
sval, err := cbg.ReadString(cr)
if err != nil {
return err
}
t.Name = string(sval)
}
// t.TotalGas (int64) (int64)
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.TotalGas = int64(extraI)
}
// t.ComputeGas (int64) (int64)
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.ComputeGas = int64(extraI)
}
// t.StorageGas (int64) (int64)
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.StorageGas = int64(extraI)
}
// t.TimeTaken (time.Duration) (int64)
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.TimeTaken = time.Duration(extraI)
}
return nil
}
var lengthBufMessageTrace = []byte{134}
func (t *MessageTrace) MarshalCBOR(w io.Writer) error {
if t == nil {
_, err := w.Write(cbg.CborNull)
return err
}
cw := cbg.NewCborWriter(w)
if _, err := cw.Write(lengthBufMessageTrace); err != nil {
return err
}
// t.From (address.Address) (struct)
if err := t.From.MarshalCBOR(cw); err != nil {
return err
}
// t.To (address.Address) (struct)
if err := t.To.MarshalCBOR(cw); err != nil {
return err
}
// t.Value (big.Int) (struct)
if err := t.Value.MarshalCBOR(cw); err != nil {
return err
}
// t.Method (abi.MethodNum) (uint64)
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.Method)); err != nil {
return err
}
// t.Params ([]uint8) (slice)
if len(t.Params) > cbg.ByteArrayMaxLen {
return xerrors.Errorf("Byte array in field t.Params was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajByteString, uint64(len(t.Params))); err != nil {
return err
}
if _, err := cw.Write(t.Params[:]); err != nil {
return err
}
// t.ParamsCodec (uint64) (uint64)
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.ParamsCodec)); err != nil {
return err
}
return nil
}
func (t *MessageTrace) UnmarshalCBOR(r io.Reader) (err error) {
*t = MessageTrace{}
cr := cbg.NewCborReader(r)
maj, extra, err := cr.ReadHeader()
if err != nil {
return err
}
defer func() {
if err == io.EOF {
err = io.ErrUnexpectedEOF
}
}()
if maj != cbg.MajArray {
return fmt.Errorf("cbor input should be of type array")
}
if extra != 6 {
return fmt.Errorf("cbor input had wrong number of fields")
}
// t.From (address.Address) (struct)
{
if err := t.From.UnmarshalCBOR(cr); err != nil {
return xerrors.Errorf("unmarshaling t.From: %w", err)
}
}
// t.To (address.Address) (struct)
{
if err := t.To.UnmarshalCBOR(cr); err != nil {
return xerrors.Errorf("unmarshaling t.To: %w", err)
}
}
// t.Value (big.Int) (struct)
{
if err := t.Value.UnmarshalCBOR(cr); err != nil {
return xerrors.Errorf("unmarshaling t.Value: %w", err)
}
}
// t.Method (abi.MethodNum) (uint64)
{
maj, extra, err = cr.ReadHeader()
if err != nil {
return err
}
if maj != cbg.MajUnsignedInt {
return fmt.Errorf("wrong type for uint64 field")
}
t.Method = abi.MethodNum(extra)
}
// t.Params ([]uint8) (slice)
maj, extra, err = cr.ReadHeader()
if err != nil {
return err
}
if extra > cbg.ByteArrayMaxLen {
return fmt.Errorf("t.Params: byte array too large (%d)", extra)
}
if maj != cbg.MajByteString {
return fmt.Errorf("expected byte array")
}
if extra > 0 {
t.Params = make([]uint8, extra)
}
if _, err := io.ReadFull(cr, t.Params[:]); err != nil {
return err
}
// t.ParamsCodec (uint64) (uint64)
{
maj, extra, err = cr.ReadHeader()
if err != nil {
return err
}
if maj != cbg.MajUnsignedInt {
return fmt.Errorf("wrong type for uint64 field")
}
t.ParamsCodec = uint64(extra)
}
return nil
}
var lengthBufReturnTrace = []byte{131}
func (t *ReturnTrace) MarshalCBOR(w io.Writer) error {
if t == nil {
_, err := w.Write(cbg.CborNull)
return err
}
cw := cbg.NewCborWriter(w)
if _, err := cw.Write(lengthBufReturnTrace); err != nil {
return err
}
// t.ExitCode (exitcode.ExitCode) (int64)
if t.ExitCode >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.ExitCode)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.ExitCode-1)); err != nil {
return err
}
}
// t.Return ([]uint8) (slice)
if len(t.Return) > cbg.ByteArrayMaxLen {
return xerrors.Errorf("Byte array in field t.Return was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajByteString, uint64(len(t.Return))); err != nil {
return err
}
if _, err := cw.Write(t.Return[:]); err != nil {
return err
}
// t.ReturnCodec (uint64) (uint64)
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.ReturnCodec)); err != nil {
return err
}
return nil
}
func (t *ReturnTrace) UnmarshalCBOR(r io.Reader) (err error) {
*t = ReturnTrace{}
cr := cbg.NewCborReader(r)
maj, extra, err := cr.ReadHeader()
if err != nil {
return err
}
defer func() {
if err == io.EOF {
err = io.ErrUnexpectedEOF
}
}()
if maj != cbg.MajArray {
return fmt.Errorf("cbor input should be of type array")
}
if extra != 3 {
return fmt.Errorf("cbor input had wrong number of fields")
}
// t.ExitCode (exitcode.ExitCode) (int64)
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative overflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.ExitCode = exitcode.ExitCode(extraI)
}
// t.Return ([]uint8) (slice)
maj, extra, err = cr.ReadHeader()
if err != nil {
return err
}
if extra > cbg.ByteArrayMaxLen {
return fmt.Errorf("t.Return: byte array too large (%d)", extra)
}
if maj != cbg.MajByteString {
return fmt.Errorf("expected byte array")
}
if extra > 0 {
t.Return = make([]uint8, extra)
}
if _, err := io.ReadFull(cr, t.Return[:]); err != nil {
return err
}
// t.ReturnCodec (uint64) (uint64)
{
maj, extra, err = cr.ReadHeader()
if err != nil {
return err
}
if maj != cbg.MajUnsignedInt {
return fmt.Errorf("wrong type for uint64 field")
}
t.ReturnCodec = uint64(extra)
}
return nil
}
var lengthBufExecutionTrace = []byte{132}
func (t *ExecutionTrace) MarshalCBOR(w io.Writer) error {
if t == nil {
_, err := w.Write(cbg.CborNull)
return err
}
cw := cbg.NewCborWriter(w)
if _, err := cw.Write(lengthBufExecutionTrace); err != nil {
return err
}
// t.Msg (types.MessageTrace) (struct)
if err := t.Msg.MarshalCBOR(cw); err != nil {
return err
}
// t.MsgRct (types.ReturnTrace) (struct)
if err := t.MsgRct.MarshalCBOR(cw); err != nil {
return err
}
// t.GasCharges ([]*types.GasTrace) (slice)
if len(t.GasCharges) > 1000000000 {
return xerrors.Errorf("Slice value in field t.GasCharges was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajArray, uint64(len(t.GasCharges))); err != nil {
return err
}
for _, v := range t.GasCharges {
if err := v.MarshalCBOR(cw); err != nil {
return err
}
}
// t.Subcalls ([]types.ExecutionTrace) (slice)
if len(t.Subcalls) > 1000000000 {
return xerrors.Errorf("Slice value in field t.Subcalls was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajArray, uint64(len(t.Subcalls))); err != nil {
return err
}
for _, v := range t.Subcalls {
if err := v.MarshalCBOR(cw); err != nil {
return err
}
}
return nil
}
func (t *ExecutionTrace) UnmarshalCBOR(r io.Reader) (err error) {
*t = ExecutionTrace{}
cr := cbg.NewCborReader(r)
maj, extra, err := cr.ReadHeader()
if err != nil {
return err
}
defer func() {
if err == io.EOF {
err = io.ErrUnexpectedEOF
}
}()
if maj != cbg.MajArray {
return fmt.Errorf("cbor input should be of type array")
}
if extra != 4 {
return fmt.Errorf("cbor input had wrong number of fields")
}
// t.Msg (types.MessageTrace) (struct)
{
if err := t.Msg.UnmarshalCBOR(cr); err != nil {
return xerrors.Errorf("unmarshaling t.Msg: %w", err)
}
}
// t.MsgRct (types.ReturnTrace) (struct)
{
if err := t.MsgRct.UnmarshalCBOR(cr); err != nil {
return xerrors.Errorf("unmarshaling t.MsgRct: %w", err)
}
}
// t.GasCharges ([]*types.GasTrace) (slice)
maj, extra, err = cr.ReadHeader()
if err != nil {
return err
}
if extra > 1000000000 {
return fmt.Errorf("t.GasCharges: array too large (%d)", extra)
}
if maj != cbg.MajArray {
return fmt.Errorf("expected cbor array")
}
if extra > 0 {
t.GasCharges = make([]*GasTrace, extra)
}
for i := 0; i < int(extra); i++ {
var v GasTrace
if err := v.UnmarshalCBOR(cr); err != nil {
return err
}
t.GasCharges[i] = &v
}
// t.Subcalls ([]types.ExecutionTrace) (slice)
maj, extra, err = cr.ReadHeader()
if err != nil {
return err
}
if extra > 1000000000 {
return fmt.Errorf("t.Subcalls: array too large (%d)", extra)
}
if maj != cbg.MajArray {
return fmt.Errorf("expected cbor array")
}
if extra > 0 {
t.Subcalls = make([]ExecutionTrace, extra)
}
for i := 0; i < int(extra); i++ {
var v ExecutionTrace
if err := v.UnmarshalCBOR(cr); err != nil {
return err
}
t.Subcalls[i] = v
}
return nil
}
-8
View File
@@ -44,14 +44,6 @@ type EthTx struct {
S EthBigInt `json:"s"`
}
func (tx *EthTx) Reward(blkBaseFee big.Int) EthBigInt {
availablePriorityFee := big.Sub(big.Int(tx.MaxFeePerGas), blkBaseFee)
if big.Cmp(big.Int(tx.MaxPriorityFeePerGas), availablePriorityFee) <= 0 {
return tx.MaxPriorityFeePerGas
}
return EthBigInt(availablePriorityFee)
}
type EthTxArgs struct {
ChainID int `json:"chainId"`
Nonce int `json:"nonce"`
+17 -6
View File
@@ -295,17 +295,21 @@ func EthAddressFromPubKey(pubk []byte) ([]byte, error) {
return ethAddr, nil
}
var maskedIDPrefix = [20 - 8]byte{0xff}
func IsEthAddress(addr address.Address) bool {
if addr.Protocol() != address.Delegated {
return false
}
payload := addr.Payload()
namespace, _, err := varint.FromUvarint(payload)
namespace, offset, err := varint.FromUvarint(payload)
if err != nil {
return false
}
return namespace == builtintypes.EthereumAddressManagerActorID
payload = payload[offset:]
return namespace == builtintypes.EthereumAddressManagerActorID && len(payload) == 20 && !bytes.HasPrefix(payload, maskedIDPrefix[:])
}
func EthAddressFromFilecoinAddress(addr address.Address) (EthAddress, error) {
@@ -326,9 +330,17 @@ func EthAddressFromFilecoinAddress(addr address.Address) (EthAddress, error) {
return EthAddress{}, xerrors.Errorf("invalid delegated address namespace in: %s", addr)
}
payload = payload[n:]
if namespace == builtintypes.EthereumAddressManagerActorID {
return CastEthAddress(payload)
if namespace != builtintypes.EthereumAddressManagerActorID {
return EthAddress{}, ErrInvalidAddress
}
ethAddr, err := CastEthAddress(payload)
if err != nil {
return EthAddress{}, err
}
if ethAddr.IsMaskedID() {
return EthAddress{}, xerrors.Errorf("f410f addresses cannot embed masked-ID payloads: %s", ethAddr)
}
return ethAddr, nil
}
return EthAddress{}, ErrInvalidAddress
}
@@ -376,8 +388,7 @@ func (ea *EthAddress) UnmarshalJSON(b []byte) error {
}
func (ea EthAddress) IsMaskedID() bool {
idmask := [12]byte{0xff}
return bytes.Equal(ea[:12], idmask[:])
return bytes.HasPrefix(ea[:], maskedIDPrefix[:])
}
func (ea EthAddress) ToFilecoinAddress() (address.Address, error) {
+15
View File
@@ -9,6 +9,7 @@ import (
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/big"
"github.com/filecoin-project/go-state-types/builtin"
)
type TestCase struct {
@@ -178,6 +179,20 @@ func TestParseEthAddr(t *testing.T) {
}
}
func TestMaskedIDInF4(t *testing.T) {
addr, err := address.NewIDAddress(100)
require.NoError(t, err)
eaddr, err := EthAddressFromFilecoinAddress(addr)
require.NoError(t, err)
badaddr, err := address.NewDelegatedAddress(builtin.EthereumAddressManagerActorID, eaddr[:])
require.NoError(t, err)
_, err = EthAddressFromFilecoinAddress(badaddr)
require.Error(t, err)
}
func TestUnmarshalEthCall(t *testing.T) {
data := `{"from":"0x4D6D86b31a112a05A473c4aE84afaF873f632325","to":"0xFe01CC39f5Ae8553D6914DBb9dC27D219fa22D7f","gas":"0x5","gasPrice":"0x6","value":"0x123","data":""}`
+29 -90
View File
@@ -2,44 +2,41 @@ package types
import (
"encoding/json"
"fmt"
"regexp"
"runtime"
"strings"
"time"
"github.com/filecoin-project/go-address"
"github.com/filecoin-project/go-state-types/abi"
"github.com/filecoin-project/go-state-types/exitcode"
)
type ExecutionTrace struct {
Msg *Message
MsgRct *MessageReceipt
Error string
Duration time.Duration
GasCharges []*GasTrace
Subcalls []ExecutionTrace
}
type GasTrace struct {
Name string
Location []Loc `json:"loc"`
TotalGas int64 `json:"tg"`
ComputeGas int64 `json:"cg"`
StorageGas int64 `json:"sg"`
TotalVirtualGas int64 `json:"vtg"`
VirtualComputeGas int64 `json:"vcg"`
VirtualStorageGas int64 `json:"vsg"`
TimeTaken time.Duration `json:"tt"`
Extra interface{} `json:"ex,omitempty"`
Callers []uintptr `json:"-"`
Name string
TotalGas int64 `json:"tg"`
ComputeGas int64 `json:"cg"`
StorageGas int64 `json:"sg"`
TimeTaken time.Duration `json:"tt"`
}
type Loc struct {
File string
Line int
Function string
type MessageTrace struct {
From address.Address
To address.Address
Value abi.TokenAmount
Method abi.MethodNum
Params []byte
ParamsCodec uint64
}
type ReturnTrace struct {
ExitCode exitcode.ExitCode
Return []byte
ReturnCodec uint64
}
type ExecutionTrace struct {
Msg MessageTrace
MsgRct ReturnTrace
GasCharges []*GasTrace `cborgen:"maxlen=1000000000"`
Subcalls []ExecutionTrace `cborgen:"maxlen=1000000000"`
}
func (et ExecutionTrace) SumGas() GasTrace {
@@ -52,71 +49,13 @@ func SumGas(charges []*GasTrace) GasTrace {
out.TotalGas += gc.TotalGas
out.ComputeGas += gc.ComputeGas
out.StorageGas += gc.StorageGas
out.TotalVirtualGas += gc.TotalVirtualGas
out.VirtualComputeGas += gc.VirtualComputeGas
out.VirtualStorageGas += gc.VirtualStorageGas
}
return out
}
func (l Loc) Show() bool {
ignorePrefix := []string{
"reflect.",
"github.com/filecoin-project/lotus/chain/vm.(*Invoker).transform",
"github.com/filecoin-project/go-amt-ipld/",
}
for _, pre := range ignorePrefix {
if strings.HasPrefix(l.Function, pre) {
return false
}
}
return true
}
func (l Loc) String() string {
file := strings.Split(l.File, "/")
fn := strings.Split(l.Function, "/")
var fnpkg string
if len(fn) > 2 {
fnpkg = strings.Join(fn[len(fn)-2:], "/")
} else {
fnpkg = l.Function
}
return fmt.Sprintf("%s@%s:%d", fnpkg, file[len(file)-1], l.Line)
}
var importantRegex = regexp.MustCompile(`github.com/filecoin-project/specs-actors/(v\d+/)?actors/builtin`)
func (l Loc) Important() bool {
return importantRegex.MatchString(l.Function)
}
func (gt *GasTrace) MarshalJSON() ([]byte, error) {
type GasTraceCopy GasTrace
if len(gt.Location) == 0 {
if len(gt.Callers) != 0 {
frames := runtime.CallersFrames(gt.Callers)
for {
frame, more := frames.Next()
if frame.Function == "github.com/filecoin-project/lotus/chain/vm.(*VM).ApplyMessage" {
break
}
l := Loc{
File: frame.File,
Line: frame.Line,
Function: frame.Function,
}
gt.Location = append(gt.Location, l)
if !more {
break
}
}
}
}
cpy := (*GasTraceCopy)(gt)
return json.Marshal(cpy)
}
+13
View File
@@ -219,4 +219,17 @@ func (m *Message) ValidForBlockInclusion(minGas int64, version network.Version)
return nil
}
// EffectiveGasPremium returns the effective gas premium claimable by the miner
// given the supplied base fee.
//
// Filecoin clamps the gas premium at GasFeeCap - BaseFee, if lower than the
// specified premium.
func (m *Message) EffectiveGasPremium(baseFee abi.TokenAmount) abi.TokenAmount {
available := big.Sub(m.GasFeeCap, baseFee)
if big.Cmp(m.GasPremium, available) <= 0 {
return m.GasPremium
}
return available
}
const TestGasLimit = 100e6
+1 -1
View File
@@ -10,7 +10,7 @@ type MpoolConfig struct {
PriorityAddrs []address.Address
SizeLimitHigh int
SizeLimitLow int
ReplaceByFeeRatio float64
ReplaceByFeeRatio Percent
PruneCooldown time.Duration
GasLimitOverestimation float64
}
+39
View File
@@ -0,0 +1,39 @@
package types
import (
"fmt"
"math"
"strconv"
"golang.org/x/xerrors"
)
// Percent stores a signed percentage as an int64. When converted to a string (or json), it's stored
// as a decimal with two places (e.g., 100% -> 1.00).
type Percent int64
func (p Percent) String() string {
abs := p
sign := ""
if abs < 0 {
abs = -abs
sign = "-"
}
return fmt.Sprintf(`%s%d.%d`, sign, abs/100, abs%100)
}
func (p Percent) MarshalJSON() ([]byte, error) {
return []byte(p.String()), nil
}
func (p *Percent) UnmarshalJSON(b []byte) error {
flt, err := strconv.ParseFloat(string(b)+"e2", 64)
if err != nil {
return xerrors.Errorf("unable to parse ratio %s: %w", string(b), err)
}
if math.Trunc(flt) != flt {
return xerrors.Errorf("ratio may only have two decimals: %s", string(b))
}
*p = Percent(flt)
return nil
}
+34
View File
@@ -0,0 +1,34 @@
package types
import (
"fmt"
"testing"
"github.com/stretchr/testify/require"
)
func TestPercent(t *testing.T) {
for _, tc := range []struct {
p Percent
s string
}{
{100, "1.0"},
{111, "1.11"},
{12, "0.12"},
{-12, "-0.12"},
{1012, "10.12"},
{-1012, "-10.12"},
{0, "0.0"},
} {
tc := tc
t.Run(fmt.Sprintf("%d <> %s", tc.p, tc.s), func(t *testing.T) {
m, err := tc.p.MarshalJSON()
require.NoError(t, err)
require.Equal(t, tc.s, string(m))
var p Percent
require.NoError(t, p.UnmarshalJSON([]byte(tc.s)))
require.Equal(t, tc.p, p)
})
}
}
+4
View File
@@ -234,6 +234,10 @@ func (ts *TipSet) MinTicketBlock() *BlockHeader {
return min
}
func (ts *TipSet) ParentMessageReceipts() cid.Cid {
return ts.blks[0].ParentMessageReceipts
}
func (ts *TipSet) ParentState() cid.Cid {
return ts.blks[0].ParentStateRoot
}
-391
View File
@@ -1,391 +0,0 @@
// Code generated by github.com/whyrusleeping/cbor-gen. DO NOT EDIT.
package vm
import (
"fmt"
"io"
"math"
"sort"
cid "github.com/ipfs/go-cid"
cbg "github.com/whyrusleeping/cbor-gen"
xerrors "golang.org/x/xerrors"
types "github.com/filecoin-project/lotus/chain/types"
)
var _ = xerrors.Errorf
var _ = cid.Undef
var _ = math.E
var _ = sort.Sort
var lengthBufFvmExecutionTrace = []byte{133}
func (t *FvmExecutionTrace) MarshalCBOR(w io.Writer) error {
if t == nil {
_, err := w.Write(cbg.CborNull)
return err
}
cw := cbg.NewCborWriter(w)
if _, err := cw.Write(lengthBufFvmExecutionTrace); err != nil {
return err
}
// t.Msg (types.Message) (struct)
if err := t.Msg.MarshalCBOR(cw); err != nil {
return err
}
// t.MsgRct (types.MessageReceipt) (struct)
if err := t.MsgRct.MarshalCBOR(cw); err != nil {
return err
}
// t.Error (string) (string)
if len(t.Error) > cbg.MaxLength {
return xerrors.Errorf("Value in field t.Error was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len(t.Error))); err != nil {
return err
}
if _, err := io.WriteString(w, string(t.Error)); err != nil {
return err
}
// t.GasCharges ([]vm.FvmGasCharge) (slice)
if len(t.GasCharges) > 1000000000 {
return xerrors.Errorf("Slice value in field t.GasCharges was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajArray, uint64(len(t.GasCharges))); err != nil {
return err
}
for _, v := range t.GasCharges {
if err := v.MarshalCBOR(cw); err != nil {
return err
}
}
// t.Subcalls ([]vm.FvmExecutionTrace) (slice)
if len(t.Subcalls) > 1000000000 {
return xerrors.Errorf("Slice value in field t.Subcalls was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajArray, uint64(len(t.Subcalls))); err != nil {
return err
}
for _, v := range t.Subcalls {
if err := v.MarshalCBOR(cw); err != nil {
return err
}
}
return nil
}
func (t *FvmExecutionTrace) UnmarshalCBOR(r io.Reader) (err error) {
*t = FvmExecutionTrace{}
cr := cbg.NewCborReader(r)
maj, extra, err := cr.ReadHeader()
if err != nil {
return err
}
defer func() {
if err == io.EOF {
err = io.ErrUnexpectedEOF
}
}()
if maj != cbg.MajArray {
return fmt.Errorf("cbor input should be of type array")
}
if extra != 5 {
return fmt.Errorf("cbor input had wrong number of fields")
}
// t.Msg (types.Message) (struct)
{
b, err := cr.ReadByte()
if err != nil {
return err
}
if b != cbg.CborNull[0] {
if err := cr.UnreadByte(); err != nil {
return err
}
t.Msg = new(types.Message)
if err := t.Msg.UnmarshalCBOR(cr); err != nil {
return xerrors.Errorf("unmarshaling t.Msg pointer: %w", err)
}
}
}
// t.MsgRct (types.MessageReceipt) (struct)
{
b, err := cr.ReadByte()
if err != nil {
return err
}
if b != cbg.CborNull[0] {
if err := cr.UnreadByte(); err != nil {
return err
}
t.MsgRct = new(types.MessageReceipt)
if err := t.MsgRct.UnmarshalCBOR(cr); err != nil {
return xerrors.Errorf("unmarshaling t.MsgRct pointer: %w", err)
}
}
}
// t.Error (string) (string)
{
sval, err := cbg.ReadString(cr)
if err != nil {
return err
}
t.Error = string(sval)
}
// t.GasCharges ([]vm.FvmGasCharge) (slice)
maj, extra, err = cr.ReadHeader()
if err != nil {
return err
}
if extra > 1000000000 {
return fmt.Errorf("t.GasCharges: array too large (%d)", extra)
}
if maj != cbg.MajArray {
return fmt.Errorf("expected cbor array")
}
if extra > 0 {
t.GasCharges = make([]FvmGasCharge, extra)
}
for i := 0; i < int(extra); i++ {
var v FvmGasCharge
if err := v.UnmarshalCBOR(cr); err != nil {
return err
}
t.GasCharges[i] = v
}
// t.Subcalls ([]vm.FvmExecutionTrace) (slice)
maj, extra, err = cr.ReadHeader()
if err != nil {
return err
}
if extra > 1000000000 {
return fmt.Errorf("t.Subcalls: array too large (%d)", extra)
}
if maj != cbg.MajArray {
return fmt.Errorf("expected cbor array")
}
if extra > 0 {
t.Subcalls = make([]FvmExecutionTrace, extra)
}
for i := 0; i < int(extra); i++ {
var v FvmExecutionTrace
if err := v.UnmarshalCBOR(cr); err != nil {
return err
}
t.Subcalls[i] = v
}
return nil
}
var lengthBufFvmGasCharge = []byte{132}
func (t *FvmGasCharge) MarshalCBOR(w io.Writer) error {
if t == nil {
_, err := w.Write(cbg.CborNull)
return err
}
cw := cbg.NewCborWriter(w)
if _, err := cw.Write(lengthBufFvmGasCharge); err != nil {
return err
}
// t.Name (string) (string)
if len(t.Name) > cbg.MaxLength {
return xerrors.Errorf("Value in field t.Name was too long")
}
if err := cw.WriteMajorTypeHeader(cbg.MajTextString, uint64(len(t.Name))); err != nil {
return err
}
if _, err := io.WriteString(w, string(t.Name)); err != nil {
return err
}
// t.TotalGas (int64) (int64)
if t.TotalGas >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.TotalGas)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.TotalGas-1)); err != nil {
return err
}
}
// t.ComputeGas (int64) (int64)
if t.ComputeGas >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.ComputeGas)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.ComputeGas-1)); err != nil {
return err
}
}
// t.StorageGas (int64) (int64)
if t.StorageGas >= 0 {
if err := cw.WriteMajorTypeHeader(cbg.MajUnsignedInt, uint64(t.StorageGas)); err != nil {
return err
}
} else {
if err := cw.WriteMajorTypeHeader(cbg.MajNegativeInt, uint64(-t.StorageGas-1)); err != nil {
return err
}
}
return nil
}
func (t *FvmGasCharge) UnmarshalCBOR(r io.Reader) (err error) {
*t = FvmGasCharge{}
cr := cbg.NewCborReader(r)
maj, extra, err := cr.ReadHeader()
if err != nil {
return err
}
defer func() {
if err == io.EOF {
err = io.ErrUnexpectedEOF
}
}()
if maj != cbg.MajArray {
return fmt.Errorf("cbor input should be of type array")
}
if extra != 4 {
return fmt.Errorf("cbor input had wrong number of fields")
}
// t.Name (string) (string)
{
sval, err := cbg.ReadString(cr)
if err != nil {
return err
}
t.Name = string(sval)
}
// t.TotalGas (int64) (int64)
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.TotalGas = int64(extraI)
}
// t.ComputeGas (int64) (int64)
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.ComputeGas = int64(extraI)
}
// t.StorageGas (int64) (int64)
{
maj, extra, err := cr.ReadHeader()
var extraI int64
if err != nil {
return err
}
switch maj {
case cbg.MajUnsignedInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 positive overflow")
}
case cbg.MajNegativeInt:
extraI = int64(extra)
if extraI < 0 {
return fmt.Errorf("int64 negative oveflow")
}
extraI = -1 - extraI
default:
return fmt.Errorf("wrong type for int64 field: %d", maj)
}
t.StorageGas = int64(extraI)
}
return nil
}
+2 -72
View File
@@ -51,57 +51,6 @@ type FvmExtern struct {
base cid.Cid
}
type FvmGasCharge struct {
Name string
TotalGas int64
ComputeGas int64
StorageGas int64
}
// This may eventually become identical to ExecutionTrace, but we can make incremental progress towards that
type FvmExecutionTrace struct {
Msg *types.Message
MsgRct *types.MessageReceipt
Error string
GasCharges []FvmGasCharge `cborgen:"maxlen=1000000000"`
Subcalls []FvmExecutionTrace `cborgen:"maxlen=1000000000"`
}
func (t *FvmExecutionTrace) ToExecutionTrace() types.ExecutionTrace {
if t == nil {
return types.ExecutionTrace{}
}
ret := types.ExecutionTrace{
Msg: t.Msg,
MsgRct: t.MsgRct,
Error: t.Error,
Subcalls: nil, // Should be nil when there are no subcalls for backwards compatibility
}
if len(t.GasCharges) > 0 {
ret.GasCharges = make([]*types.GasTrace, len(t.GasCharges))
for i, v := range t.GasCharges {
ret.GasCharges[i] = &types.GasTrace{
Name: v.Name,
TotalGas: v.TotalGas,
ComputeGas: v.ComputeGas,
StorageGas: v.StorageGas,
}
}
}
if len(t.Subcalls) > 0 {
ret.Subcalls = make([]types.ExecutionTrace, len(t.Subcalls))
for i, v := range t.Subcalls {
ret.Subcalls[i] = v.ToExecutionTrace()
}
}
return ret
}
func (x *FvmExtern) TipsetCid(ctx context.Context, epoch abi.ChainEpoch) (cid.Cid, error) {
tsk, err := x.tsGet(ctx, epoch)
if err != nil {
@@ -487,19 +436,9 @@ func (vm *FVM) ApplyMessage(ctx context.Context, cmsg types.ChainMsg) (*ApplyRet
var et types.ExecutionTrace
if len(ret.ExecTraceBytes) != 0 {
var fvmEt FvmExecutionTrace
if err = fvmEt.UnmarshalCBOR(bytes.NewReader(ret.ExecTraceBytes)); err != nil {
if err = et.UnmarshalCBOR(bytes.NewReader(ret.ExecTraceBytes)); err != nil {
return nil, xerrors.Errorf("failed to unmarshal exectrace: %w", err)
}
et = fvmEt.ToExecutionTrace()
}
// Set the top-level exectrace info from the message and receipt for backwards compatibility
et.Msg = vmMsg
et.MsgRct = &receipt
et.Duration = duration
if aerr != nil {
et.Error = aerr.Error()
}
applyRet := &ApplyRet{
@@ -562,18 +501,9 @@ func (vm *FVM) ApplyImplicitMessage(ctx context.Context, cmsg *types.Message) (*
var et types.ExecutionTrace
if len(ret.ExecTraceBytes) != 0 {
var fvmEt FvmExecutionTrace
if err = fvmEt.UnmarshalCBOR(bytes.NewReader(ret.ExecTraceBytes)); err != nil {
if err = et.UnmarshalCBOR(bytes.NewReader(ret.ExecTraceBytes)); err != nil {
return nil, xerrors.Errorf("failed to unmarshal exectrace: %w", err)
}
et = fvmEt.ToExecutionTrace()
} else {
et.Msg = vmMsg
et.MsgRct = &receipt
et.Duration = duration
if aerr != nil {
et.Error = aerr.Error()
}
}
applyRet := &ApplyRet{
+1 -19
View File
@@ -6,7 +6,6 @@ import (
"encoding/binary"
"fmt"
"os"
gruntime "runtime"
"time"
"github.com/ipfs/go-cid"
@@ -571,35 +570,18 @@ func (rt *Runtime) chargeGasFunc(skip int) func(GasCharge) {
func (rt *Runtime) chargeGasInternal(gas GasCharge, skip int) aerrors.ActorError {
toUse := gas.Total()
if EnableDetailedTracing {
var callers [10]uintptr
cout := gruntime.Callers(2+skip, callers[:])
now := build.Clock.Now()
if rt.lastGasCharge != nil {
rt.lastGasCharge.TimeTaken = now.Sub(rt.lastGasChargeTime)
}
gasTrace := types.GasTrace{
Name: gas.Name,
Extra: gas.Extra,
Name: gas.Name,
TotalGas: toUse,
ComputeGas: gas.ComputeGas,
StorageGas: gas.StorageGas,
VirtualComputeGas: gas.VirtualCompute,
VirtualStorageGas: gas.VirtualStorage,
Callers: callers[:cout],
}
if gasTrace.VirtualStorageGas == 0 {
gasTrace.VirtualStorageGas = gasTrace.StorageGas
}
if gasTrace.VirtualComputeGas == 0 {
gasTrace.VirtualComputeGas = gasTrace.ComputeGas
}
gasTrace.TotalVirtualGas = gasTrace.VirtualComputeGas + gasTrace.VirtualStorageGas
rt.executionTrace.GasCharges = append(rt.executionTrace.GasCharges, &gasTrace)
rt.lastGasChargeTime = now
+20 -9
View File
@@ -38,6 +38,7 @@ import (
)
const MaxCallDepth = 4096
const CborCodec = 0x51
var (
log = logging.Logger("vm")
@@ -125,6 +126,10 @@ func (bs *gasChargingBlocks) Put(ctx context.Context, blk block.Block) error {
}
func (vm *LegacyVM) makeRuntime(ctx context.Context, msg *types.Message, parent *Runtime) *Runtime {
paramsCodec := uint64(0)
if len(msg.Params) > 0 {
paramsCodec = CborCodec
}
rt := &Runtime{
ctx: ctx,
vm: vm,
@@ -140,7 +145,14 @@ func (vm *LegacyVM) makeRuntime(ctx context.Context, msg *types.Message, parent
pricelist: PricelistByEpoch(vm.blockHeight),
allowInternal: true,
callerValidated: false,
executionTrace: types.ExecutionTrace{Msg: msg},
executionTrace: types.ExecutionTrace{Msg: types.MessageTrace{
From: msg.From,
To: msg.To,
Value: msg.Value,
Method: msg.Method,
Params: msg.Params,
ParamsCodec: paramsCodec,
}},
}
if parent != nil {
@@ -369,15 +381,14 @@ func (vm *LegacyVM) send(ctx context.Context, msg *types.Message, parent *Runtim
return nil, nil
}()
mr := types.MessageReceipt{
ExitCode: aerrors.RetCode(err),
Return: ret,
GasUsed: rt.gasUsed,
retCodec := uint64(0)
if len(ret) > 0 {
retCodec = CborCodec
}
rt.executionTrace.MsgRct = &mr
rt.executionTrace.Duration = time.Since(start)
if err != nil {
rt.executionTrace.Error = err.Error()
rt.executionTrace.MsgRct = types.ReturnTrace{
ExitCode: aerrors.RetCode(err),
Return: ret,
ReturnCodec: retCodec,
}
return ret, err, rt

Some files were not shown because too many files have changed in this diff Show More