The correct name for this field is 'input' according to the Ethereum specs [0].
However, for the longest time, clients have been using 'data' and servers have been
lenient to accept both, preferring 'input' over 'data' when both appear.
Our lack of support for 'input' had gone unnoticed until go-ethereum decided
to adjust their ethclient implementation to issue eth_call and eth_estimateGas
requests with the 'input' field instead of 'data' [1]. This suddenly broke apps
using this client against Lotus' Eth API.
[0]: https://github.com/ethereum/execution-apis/blob/main/src/schemas/transaction.yaml#L33-L35
[1]: ethereum/go-ethereum#28078
Co-authored-by: raulk <raul.kripalani@gmail.com>
Previously, we weren't checking the "to" address. I've also re-ordered
the operations in this function to make it easier to reason about them.
It'll have a slight runtime cost (we _always_ ABI-encode the parameters,
then throw away the result if it turns out we're actually dealing with
an Ethereum transaction), but it's _much_ simpler.
Also explicitly limit how many bytes we're willing to read in one go
such that we're capable of reading a worst-case tipset (like, really,
never going to happen worst-case). Previously, this wasn't an issue.
However, we've bumped the max number of messages from 8,192 to 150,000
and need to limit allocations somewhere else.
Also explicitly limit how many bytes we're willing to read in one go
such that we're capable of reading a worst-case tipset (like, really,
never going to happen worst-case). Previously, this wasn't an issue.
However, we've bumped the max number of messages from 8,192 to 150,000
and need to limit allocations somewhere else.
Also explicitly limit how many bytes we're willing to read in one go
such that we're capable of reading a worst-case tipset (like, really,
never going to happen worst-case). Previously, this wasn't an issue.
However, we've bumped the max number of messages from 8,192 to 150,000
and need to limit allocations somewhere else.
Correctly handle "unresolvable" to/from addresses in top-level messages in the Ethereum API. Specifically:
1. Fail if we can't resolve the from address. As far as I can tell, this should be impossible (the message statically couldn't have been included in the block if the sender didn't exist).
2. If we can't resolve the "to" address to an ID, use "max uint64" as the ID (`0xff0000000000000000000000ffffffffffffffff`). This will only happen if the transaction was reverted. It'll be a little confusing, but the alternative is to (a) use an empty address (will look like a contract creation, which is definitely wrong) or (b) use a random/hashed address which will likely be more confusing as it won't be "obviously weird".
In #11212, we added the necessary logic to download the params
automatically. However, we're not taking advantage of CI caching.
Here, we configure CI to cache these params.
When translating "native" messages to Ethereum transactions, correctly handle parameters:
1. If the message looks like a valid "create external", treat it as a contract creation.
2. If it looks like a valid EVM invocation, decode it as such.
3. Otherwise, ABI-encode the parameters to make them look like a "handle_filecoin_method" call. This
will help chain explorers recognize these messages.
Part of #11355
We need to always use the state-tree from the tipset _after_ the message
executed. If we use any other state-tree, we might not find the address
we're trying to resolve.
This change also has some implication for pending messages: there's no
guarantee we'll be able to generate a 0x-style address for a pending
native message. So, instead of trying, I've removed support for pending
native messages from the Eth API. Messages from EthAccounts will still
work, and native messages will still show up in blocks/traces, they just
won't show up as "pending". Which should affect exactly nobody.
I'm also taking this opportunity to cleanup some edge-cases:
1. Pass contexts where appropriate.
2. Remove all state access from `ethTxHashFromSignedMessage`.
Part of #11355
We added this check as a self-test to make sure that the message indices
in our trace matched up with those in the block. Unfortunately, this
only works for blocks on the main-chain, and breaks tracing of uncle
blocks (and tracing of head).
* upgrade calibnet by removing move_partitions from miner actor in actor v12
* cids for buggy bundles
* revert changes to v12 tar
* upgrade system actor state
* update based on manifest
* nit: clean up some comments
* chore: rename param to oldBuggyMinerCID
* refactor, ensure both buggy bundles are loaded
* update to actors v12.0.0-rc.3
* fix: load second buggy bundle for UpgradeWatermelonFixHeight
* add calibration fix2 upgrade epcoh
* update mainnet upgrade epoch
---------
Co-authored-by: Aayush <arajasek94@gmail.com>
Co-authored-by: jennijuju <jiayingw703@gmail.com>
* upgrade calibnet by removing move_partitions from miner actor in actor v12
* cids for buggy bundles
* revert changes to v12 tar
* upgrade system actor state
* update based on manifest
* nit: clean up some comments
* chore: rename param to oldBuggyMinerCID
* refactor, ensure both buggy bundles are loaded
* update to actors v12.0.0-rc.3
* fix: load second buggy bundle for UpgradeWatermelonFixHeight
* add calibration fix2 upgrade epcoh
* update mainnet upgrade epoch
---------
Co-authored-by: Aayush <arajasek94@gmail.com>
Co-authored-by: jennijuju <jiayingw703@gmail.com>
* upgrade calibnet by removing move_partitions from miner actor in actor v12
* cids for buggy bundles
* revert changes to v12 tar
* upgrade system actor state
* update based on manifest
* nit: clean up some comments
* chore: rename param to oldBuggyMinerCID
* refactor, ensure both buggy bundles are loaded
* update to actors v12.0.0-rc.3
* fix: load second buggy bundle for UpgradeWatermelonFixHeight
* add calibration fix2 upgrade epcoh
* update mainnet upgrade epoch
---------
Co-authored-by: Aayush <arajasek94@gmail.com>
Co-authored-by: jennijuju <jiayingw703@gmail.com>
* chore: eth: move & rename input/output encoding functions
These are shared functions, so I'm moving them to the utils library.
* fix: eth: correctly encode and simplify native input/output encoding
When generating eth traces, we encode "native" message inputs/outputs
to "solidity ABI" by formatting the inputs/outputs the same way we do in
FEVM's "handle_native_method". However, we had quite a few bugs with the
implementation:
1. We were right-aligning 64bit values in 256bit words, instead of
left-aligning (as we should given that these values are big-endian).
2. The return-value encoding wasn't correctly handling lengths.
This patch:
1. Fixes those bugs.
2. Deduplicates the logic (we're doing _basically_ the same thing in
both cases).
3. Removes all error paths (these functions can't fail).
- Added a goroutine to listen for interrupt signals, which will cancel the current context when an interrupt signal is received. This allows for graceful shutdown of ongoing operations.
We were computing this based on the max block gas, but this is
incorrect. The new value isn't entirely correct either (we should
probably compute an average of the gas used in each block in the
tipset?), but it's good enough.
fixes#10515
We were computing this based on the max block gas, but this is
incorrect. The new value isn't entirely correct either (we should
probably compute an average of the gas used in each block in the
tipset?), but it's good enough.
fixes#10515
* refactor: make GetEmbeddedBuiltinActorsBundle take network bundle name
* update calibnet actor bundle to v12.0.0-rc.2, but include v12.0.0-rc.1 as calibrationnet-buggy.car
* wip: calibnet unbork migration
* calibnet: add buggy miner actor CID to actorMeta
* fix incorrect buggy calibnet manifest
* make UpgradeWatermelonFixHeight a build param
* calibnet patch: check whether network is calibration from init actor state
* add sanity checks to the v12 calibnet patch upgrade
* address review
* refactor: make GetEmbeddedBuiltinActorsBundle take network bundle name
* update calibnet actor bundle to v12.0.0-rc.2, but include v12.0.0-rc.1 as calibrationnet-buggy.car
* wip: calibnet unbork migration
* calibnet: add buggy miner actor CID to actorMeta
* fix incorrect buggy calibnet manifest
* make UpgradeWatermelonFixHeight a build param
* calibnet patch: check whether network is calibration from init actor state
* add sanity checks to the v12 calibnet patch upgrade
* address review
* refactor: make GetEmbeddedBuiltinActorsBundle take network bundle name
* update calibnet actor bundle to v12.0.0-rc.2, but include v12.0.0-rc.1 as calibrationnet-buggy.car
* wip: calibnet unbork migration
* calibnet: add buggy miner actor CID to actorMeta
* fix incorrect buggy calibnet manifest
* make UpgradeWatermelonFixHeight a build param
* calibnet patch: check whether network is calibration from init actor state
* add sanity checks to the v12 calibnet patch upgrade
* address review
1. Implement a pass-through for garbage collection on the idstore.
2. Fix the `lotus-shed state-prune` command.
NOTE: the new performance of running a full prune will be significantly
less as this version doesn't batch. However, it should now actually
_work_ and most users will be using the splitstore anyways.
* refactor: replace ClearLayerData with ClearCache
The `ClearLayerData` FFI call was accidentally introduced with the
Synthetic PoRep. The call does under the hood exactly what `ClearCache`
is doing. This is a first step to remove `ClearLayerData`t also from
the FFI again, in order to reduce the API surface.
* fix types
---------
Co-authored-by: Steven Allen <steven@stebalien.com>
We were computing this based on the max block gas, but this is
incorrect. The new value isn't entirely correct either (we should
probably compute an average of the gas used in each block in the
tipset?), but it's good enough.
fixes#10515
All these cases here are actually errors and returning `nil` makes this
hard to debug. We likely returned nil in the past to be "best effort"
but, as far as I can tell, we should only hit these error cases if
something is actually wrong.
part of #11325
* fix: networking: avoid dialing when trying to handshake peers
Also, avoid handshaking with peers that don't support the handshake
protocol.
* drop unused imports
---------
Co-authored-by: Steven Allen <steven@stebalien.com>
After changing in prev commit to use to ethereum addresses the
comparison does not make sense against builtin actors. This
fixes that by storing also the filecoin addresses in each trace
Also renamed filecoin related fields to Filecoin prefix.
Also remove requirement call to InvokeContract needed to come
from a evm actor
The sequence number used for replay detection was being updated before message validation confirmed that the message originated from the correct host. This would allow one host A to create a message with the ID of another host B that could then update the cached sequence number for B. While the message from A would fail validation and be ignored, the cached sequence number for B would get updated. This would lead to a temporary DoS for host B as its messages were incorrectly rejected as replays.
This fixes the issue by setting the cached sequence number after message validation.
Update the test-vectors repo to the "last" release. This release deletes
most of the code (that depended on an ancient version of lotus anyways)
as we now rely on the "fvm" test vectors for all but historical tests.
Unfortunately, we can't _yet_ merge the schema branch. But we will in a
moment. But this at least fixes go-state-types
These dependencies were based on branches which no longer exist.
Introduce nv21 skeleton for local testing:
- Use local go-state-types with actor_version_checklist changes: https://github.com/filecoin-project/go-state-types/blob/master/actors_version_checklist.md
- Imports mock v12-actors bundle
- Define upgrade heights
- Generate adapters
- Add upgrade schedule and migration
- Add actorstype to the NewActorRegistry in /chain/consensus/computestate.go
- Add upgrade field to api/types.go/ForkUpgradeParams
- Add upgrade to node/impl/full/state.go
- Add network version to chain/state/statetree.go
- make jen
- make docsgen-cli
This commit includes the following updates:
- Added a new 'pprof' flag to the command-line arguments in main.go. This flag allows the user to specify a file name for writing CPU profile information, enabling performance analysis.
- Implemented functionality to start CPU profiling in the 'Before' function. If a file name is provided via the 'pprof' flag, the function creates the specified file and begins CPU profiling.
- Added an 'After' function to stop CPU profiling once the program execution is finished.
These changes provide a way to analyze the CPU usage of the 'lotus-shed' application, which can be valuable for performance tuning and optimization.
This commit includes the following updates:
- Imported v12 of the filecoin state types in migrations.go. This allows the system to interpret and operate on the latest structure of filecoin state information, ensuring compatibility with the latest version of the Filecoin network.
- Updated the 'getMigrationFuncsForNetwork' function to include a case for network.Version21, enabling the use of the UpgradeActorsV12 and PreUpgradeActorsV12 functions for network migration. These changes ensure the system correctly handles migrations when the network upgrades to version 21.
- Implemented a new 'checkNv21Invariants' function to check the invariants for version 21 of the Filecoin network. This function:
1. Loads the new state root from the actor store.
2. Retrieves the actor code IDs for the current state version (v12).
3. Loads the actor tree from the state root and checks state invariants using the actor code IDs.
4. Logs any error messages generated during the invariant check.
5. Finally, prints a message with the time taken to complete the checks.
By checking these invariants, we can ensure the system's state is consistent and valid after migrating to network version 21.
1. Switch to go-multicodec as the source of multicodec code information.
This gives us a central, generated source of multicodec codes.
2. Use this library inside the VM and shapshot logic to consistently
allow CBOR, in addition to DagCBOR.
3. Remove the hard-coded CBOR constant.
Introduce nv21 skeleton for local testing:
- Use local go-state-types with actor_version_checklist changes: https://github.com/filecoin-project/go-state-types/blob/master/actors_version_checklist.md
- Imports mock v12-actors bundle
- Define upgrade heights
- Generate adapters
- Add upgrade schedule and migration
- Add actorstype to the NewActorRegistry in /chain/consensus/computestate.go
- Add upgrade field to api/types.go/ForkUpgradeParams
- Add upgrade to node/impl/full/state.go
- Add network version to chain/state/statetree.go
- make jen
- make docsgen-cli
- Updates to wasmtime 10.
- Unifies some dependencies between fvm v2 and v3 (slightly smaller
binary).
- Supports instance pool oversubscription when FVM concurrency is
enabled. This should allow users to specify high concurrency values
without running out of address space.
Remove existing chain on a snapshot-import
log info instead of print
log info instead of print
Remove defer lock
Remove defer lock, and explicitly close the lockedRepo instead
This commit refactors the code in `invariants.go` to support state invariant checks for different actor versions. The code now includes checks for versions 10 and 11 of the actors. The `CheckStateInvariants` function is invoked based on the actor version, passing the appropriate parameters. This change ensures that state invariant checks are performed correctly for different actor versions.
The changes made are as follows:
- Added imports for actor versions 10 and 11.
- Added `case` statements for versions 10 and 11 in the switch statement.
- Invoked `CheckStateInvariants` function with the correct parameters for each version.
- Handled errors returned by the `CheckStateInvariants` function.
These changes enhance the flexibility and compatibility of the code with different actor versions, improving the accuracy of state invariant checks.
Fixes: #10814
This PR updates the following RPC methods according to EIP-1898
specs.
The following RPC methods are affected:
- eth_getBalance
- eth_getStorageAt
- eth_getTransactionCount
- eth_getCode
- eth_call
Note that eth_getBlockByNumber was not included in this list in
the spec although it seems it should be affected also?
Currently these methods all accept a blkParam string which can be
one of "latest", "earliest", "pending", or a block number (decimal
or hex). The spec enables caller to additionally specify a json
hash which can include the following fields:
- blockNumber EthUint64: A block number (decimal or hex) which is
similar to the original use of the blkParam string
- blockHash EthHash: The block hash
- requireCanonical bool) If true we should make sure the block is
in the canonical chain
Since the blkParam needs to support both being a number/string and
a json hash then this to properly work we need to introduce a new
struct with pointer fields to check if they exist. This is done
in the EthBlockParamByNumberOrHash struct which first tries to
unmarshal as a json hash (according to eip-1898) and then fallback
to unmarshal as string/number.
This migrates everything except the `go-car` librairy: https://github.com/ipfs/boxo/issues/218#issuecomment-1529922103
I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups.
This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649
The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions.
Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent https://github.com/golang/go/issues/58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type.
Stubbing types were reverted https://github.com/ipfs/boxo/issues/218#issuecomment-1478650351
Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
Previously, we'd use the current head if not otherwise specified, even
when the user specified a epoch. Now:
1. If the user specifies nothing, we use head head's epoch.
2. If the user specifies a tipset and no epoch, we use that tipset and
the epoch of that tipset.
3. If the user specifies an epoch and no tipset, use the tipset at that
epoch (based on the current head).
4. Finally, if the user species both, use both (allowing the
epoch/tipset to disagree).
This PR introduces as sharded mutex within the ChainIndex#GetTipsetByHeight.
It also replaces a go map with xsync.Map which doesn't require locking.
The lock is taken when it appears that ChainIndex filling work should be
started. After claiming the lock, the status of the cache is rechecked,
if the entry is still missing, the fillCache is started.
Thanks to @snissn and @arajasek for debugging and taking initial stabs at this.
Supersedes #10866 and 10885
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
This benchmark is designed to stress test the rpc methods of a lotus node so that we can simulate real world usage and measure the performance of rpc methods on the node.
This benchmark has the following features:
* Can query each method both sequentially and concurrently
* Supports rate limiting
* Can query multiple different endpoints at once (supporting different concurrency level and rate limiting for each method)
* Gives a nice reporting summary of the stress testing of each method (including latency distribution, histogram and more)
* Easy to use
To use this benchmark you must specify the rpc methods you want to test using the --method options, the format of it is:
--method=NAME[:CONCURRENCY][:QPS][:PARAMS] where only METHOD is required.
Here are some real examples:
lotus-bench rpc --method='eth_chainId' // run eth_chainId with default concurrency and qps
lotus-bench rpc --method='eth_chainId:3' // override concurrency to 3
lotus-bench rpc --method='eth_chainId::100' // override to 100 qps while using default concurrency
lotus-bench rpc --method='eth_chainId:3:100' // run using 3 workers but limit to 100 qps
lotus-bench rpc --method='eth_getTransactionCount:::["0xd4c70007F3F502f212c7e6794b94C06F36173B36", "latest"]' // run using optional params while using default concurrency and qps
lotus-bench rpc --method='eth_chainId' --method='eth_getTransactionCount:10:0:["0xd4c70007F3F502f212c7e6794b94C06F36173B36", "latest"]' // run multiple methods at once`,
Fixes: https://github.com/filecoin-project/lotus/issues/10752
This PR introduces as sharded mutex within the ChainIndex#GetTipsetByHeight.
It also replaces a go map with xsync.Map which doesn't require locking.
The lock is taken when it appears that ChainIndex filling work should be
started. After claiming the lock, the status of the cache is rechecked,
if the entry is still missing, the fillCache is started.
Thanks to @snissn and @arajasek for debugging and taking initial stabs at this.
Supersedes #10866 and 10885
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
fix: types: error out on decoding BlockMsg with extraneous data
Fixes OSS-fuzz issue 48208: lotus:fuzz_block_msg
Signed-off-by: Yolan Romailler <anomalroil@users.noreply.github.com>
- Fix an issue where `lotus-miner storage redeclare --all` required an argument
- Actually implement the logic to redeclare a single sectors in a single storage path
* Fix 1.21 regression: GET_32G_MAX_CONCURRENT + mixed prepared/executing leads to stuck scheduler
If you have 12 GET tasks and GET_32G_MAX_CONCURRENT=1, sealing jobs will only show assigned tasks for GET of the miner
and is stuck.
I believe this to be a regression of 1.21 unifying the counters, in the case of GETs where PrepType and TaskType
both being seal/v0/fetch leading to a state where tasks are blocked since already counted towards the limit.
* itests: Repro issue from PR #10633
* make counters int (non-working)
* fix: worker sched: Send taskDone notifs after tasks are done
* itests: Make TestPledgeMaxConcurrentGet actually reproduce the issue
* make the linter happy
---------
Co-authored-by: Steffen Butzer <steffen.butzer@outlook.com>
* Fix 1.21 regression: GET_32G_MAX_CONCURRENT + mixed prepared/executing leads to stuck scheduler
If you have 12 GET tasks and GET_32G_MAX_CONCURRENT=1, sealing jobs will only show assigned tasks for GET of the miner
and is stuck.
I believe this to be a regression of 1.21 unifying the counters, in the case of GETs where PrepType and TaskType
both being seal/v0/fetch leading to a state where tasks are blocked since already counted towards the limit.
* itests: Repro issue from PR #10633
* make counters int (non-working)
* fix: worker sched: Send taskDone notifs after tasks are done
* itests: Make TestPledgeMaxConcurrentGet actually reproduce the issue
* make the linter happy
---------
Co-authored-by: Steffen Butzer <steffen.butzer@outlook.com>
This fixes the flakyness by:
1. Disconnecting the client from the miner before submitting the
message. That way, we force it to get stuck in the message pool.
2. Removing the logic that asks lotus for the "latest" block. We have
other tests that exercise "latest".
fixes#10824
* release the read lock earlier as it is not needed for chaincomputebasefee
* chain/messagepool/selection.go change to read lock in SelectMessages
* tighten up locks in chain/messagepool/repub.go and two questions on whether curTsLks are needed as comments
* include suggestion from @Jorropo to preallocate our msgs array so that we only need to make a single allocation
* mp.pending should not be accessed directly but through the getter
* from @arajasek: just check whether the sender is a robust address (anything except an ID address is robust) here, and return if so. That will:
be faster
reduce the size of this cache by half, because we can drop mp.keyCache.Add(ka, ka) on line 491.
* do not need curTslk and clean up code comments
* Make lotus-worker report GPU usage to miner during ReplicaUpdate task
* make gen
---------
Co-authored-by: Jacob Crowther <jcrowthe@users.noreply.github.com>
This reverts commit 8b2208fd9a, reversing
changes made to 2db6b12b78.
Unfortunately, this is rather tricky code. We've found several issues so
far and, while we've fixed a few, there are outstanding issues that
would require complex fixes we don't have time to tackle right now.
Luckily, this code isn't actually needed by the main Filecoin chain
which relies on consensus fault reporting to handle equivocation. So we
can just try again later.
- Increase epoch times to give the miners a chance to see each other's
blocks.
- Wait longer for a multi-block tipset.
- Reduce the initial wait (we're increasing the block times and I don't
really feel like waiting around).
* fix: sync: fail sync instead of logging if we sync the wrong chain
* fix: sync: write headers in the correct order
Just in case. This shouldn't be necessary, but we might as well.
* fix: minus minus
* fix: do put the tipset
Put != Persist
And fix the message to account for the fact that we now reject _old_
blocks along with new ones.
We frequently receive "out of date" blocks in hello messages from
syncing and/or out of sync nodes. This isn't an error.
This will reject blocks in pubsub validation if they're either:
1. Too far into the future (5 blocks beyond the expected head).
2. Too far into the past (before finality with respect to our current
head).
Specifically:
1. We were previously rejecting future blocks in the sync logic, but not
in pubsub itself.
2. We never used to check if a block was too _old_.
Motivation: Blocks that are too new/too old can cause us to perform
quite a bit of unnecessary work.
We have to save raw blocks to the snapshot, but we should not be scanning them
for additional links as if they were CBOR blocks.
This cleans the logic a bit (we were checking that the parent was a CBOR block
before queueing up the children, but then scanning the children... it was weird).
Additionally, more verbose logging is added for the next time ScanForLinks
fails (currently very little info was given).
Our ScanForLinks callback should only enqueue CBOR for further processing.
We have observed that EthGetTransactionCount is one of the hotspots
on Glif production notes, and we are seeing regular 10-20 second
latencies when calling this rpc method.
I tracked the high latency spikes and they were correlated when
we were running ExecuteTipSet while following the chain.
To address this, we should not rely on tipset computation to get
nounce and instead look at the parent tipset and then count the
messages sent from the 'addr'.
The function/parameter were poorly named and should never have been
exposed. "GC" confidence should always be the same, this parameter
doesn't let us actually set the _confidence_, just the point before
which we no longer support reverts.
fixes#10706
Technically, the block validator caught this panic. But it's pointless
because we have a _real_ mechanism to return the validation reason,
which we should have been using.
In general, panicing like this is a very bad idea because it's
non-obvious and, in this case, completely undocumented.
* Add feature to stagger sector prove commit submission
* make gen and docsgen as usual
* address comments and lint
* Update comment
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
* make gen for stupid comment
* make docsgen
* address comments
---------
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
We want to make the execution trace cache size configurable as SPs
may want to disable it while exchanges may want to crank it up.
We were also are going with intuition for this value, so having
ability to change it without a new build would help.
Fixes: https://github.com/filecoin-project/lotus/issues/10584
This PR adds a small cache to calls to ExecutionTrace which helps
improve performance for node operators like exchanges and block
explorers.
If items is in cache calls to this function will be 2-3x faster.
Fixes: https://github.com/filecoin-project/lotus/issues/10504
* have gas estimate call callInternal with applyTsMessages = false and other calls with applyTsMessages=true for gas caclulation optimization
* set applyTsMessages = true in CallWithGas call in shed
* update test with new callwithgas api optimization for eth call
* Update chain/stmgr/call.go
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change
* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change
* make sure that even if we arent apply ts messages we grab ts messages from the particular user who is requesting gas estimation
---------
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-10-0-4-29.us-east-2.compute.internal>
- drop the execution index; we don't need it
- it is inclusion tipset
- use MessagesForTipset
- hoist db sql stuffs on top for clarity
- add index for tipset on messages
- [ ] If the PR affects users (e.g., new feature, bug fix, system requirements change), update the CHANGELOG.md and add details to the UNRELEASED section.
- [ ] New features have usage guidelines and / or documentation updates in
This is an optional but **highly recommended feature release** of Lotus, as it includes fixes for synchronizations issues that users have experienced. The feature release also introduces `Lotus-Provider` in its alpha testing phase, as well as the ability to call external PC2-binaries during the sealing process.
## ☢️ Upgrade Warnings ☢️
There are no upgrade warnings for this feature release.
## ⭐️ Highlights ⭐️
### Lotus-Provider
The feature release ships the alpha release of the new Lotus-Provider binary, together with its initial features - High Availability of WindowPoSt and WinningPoSt.
So what is so exciting about Lotus-Provider:
**High Availability**
- You can run as many `Lotus-Provider` instances as you want for both WindowPoSt and WinningPOSt.
- You can connect them to as many clustered Yugabyte instances as you want to. This allows for an NxN configuration where all instances can communicate with all others.
- You have the option to connect different instances to different chain daemons.
**Simplicity**
- Once the configuration is in the database, setting up a new machine with Lotus-Provider is straightforward. Simply start the binary with the correct flags to find YugabyteDB and specify which configuration layers it should use.
**Durability**
- `Lotus-Provider` is designed with robustness in mind. Updates to the system are handled seamlessly, ensuring that performance and stability are maintained when taking down machines in your cluster for updates.
Read more about [`Lotus-Provider` in the documentation here](https://lotus.filecoin.io/storage-providers/lotus-provider/overview/). And check out the how you can migrate from [Lotus-Miner to Lotus-Provider here](https://lotus.filecoin.io/storage-providers/lotus-provider/setup/). **(Only recommended in testnets while its in Alpha)**
### External PC2-binaries
In this feature release, storage providers can call external PC2-binaries during the sealing process. This allows storage providers to leverage the SupraSeal PC2 binary, which has been shown to improve sealing speed in the PC2-phase. For instance, our current benchmarks show that an NVIDIA RTX A5000 card was able to complete PC2 in approximately 2.5 minutes.
We have verified that SupraSeal PC2 functions properly with Committed Capacity (CC) sectors, both SyntheticPoReps and non-Synthetic PoReps. However calling SupraSeal PC2 with deal sectors is not supported in this feature release.
For more information on how to use SupraSeal PC2 with your `lotus-worker`, as well as how to use feature, please [refer to the documentation](https://lotus.filecoin.io/tutorials/lotus-miner/supra-seal-pc2/).
## New features
- feat: sturdypost work branch ([filecoin-project/lotus#11405](https://github.com/filecoin-project/lotus/pull/11405))
- Adds the `Lotus-Provider` binary, and the HarmonyDB framework.
- feat: worker: Support delegating precommit2 to external binary ([filecoin-project/lotus#11185](https://github.com/filecoin-project/lotus/pull/11185))
- Allows for delegating PreCommit2 to an exteranl binary.
- Adds a script for building the SupraSeal-PC2 binary easily.
- Feat: daemon: Auto remove existing chain if importing chain file or snapshot ([filecoin-project/lotus#11277](https://github.com/filecoin-project/lotus/pull/11277))
- Auto removes the existing chain when importing a snapshot.
- feat: Add ETA to lotus sync wait (#11211) ([filecoin-project/lotus#11211](https://github.com/filecoin-project/lotus/pull/11211))
- Adds a ETA indicator to `lotus sync wait`, so you can get an estimate for how long until sync is completed.
- feat: Add lotus-bench cli option to stress test any binary ([filecoin-project/lotus#11270](https://github.com/filecoin-project/lotus/pull/11270))
- Enables the `Lotus-Bench` to run any binary and analyze their latency and histogram distribution, track most common errors, perform stress testing under different concurrency levels and see how it works under different QPS.
- feat: chain import: don't walk to genesis - 2-3x faster snapshot import (#11446) ([filecoin-project/lotus#11446](https://github.com/filecoin-project/lotus/pull/11446))
- Improves Snapshot import speed, by not walking back to genesis on import.
- feat: bench: flag to output GenerateWinningPoStWithVanilla params ([filecoin-project/lotus#11460](https://github.com/filecoin-project/lotus/pull/11460))
## Improvements
- feat: bootstrap: add glif bootstrap node on calibration ([filecoin-project/lotus#11175](https://github.com/filecoin-project/lotus/pull/11175))
- fix: bench: Set ticket and seed to a non-all zero value ([filecoin-project/lotus#11429](https://github.com/filecoin-project/lotus/pull/11429))
- feat: cli: sort actor CIDs alphabetically before printing (#11345) ([filecoin-project/lotus#11345](https://github.com/filecoin-project/lotus/pull/11345))
- fix: worker: Connect when --listen is not set ([filecoin-project/lotus#11294](https://github.com/filecoin-project/lotus/pull/11294))
- fix: miner info: Show correct sector state counts ([filecoin-project/lotus#11456](https://github.com/filecoin-project/lotus/pull/11456))
- feat: miner: defensive check for equivocation ([filecoin-project/lotus#11321](https://github.com/filecoin-project/lotus/pull/11321))
- feat: Instructions for setting up Grafana/Prometheus for monitoring local lotus node ([filecoin-project/lotus#11276](https://github.com/filecoin-project/lotus/pull/11276))
- fix: cli: Wrap error in wallet sign ([filecoin-project/lotus#11273](https://github.com/filecoin-project/lotus/pull/11273))
- fix: Add time slicing to splitstore purging to reduce lock congestion ([filecoin-project/lotus#11269](https://github.com/filecoin-project/lotus/pull/11269))
- feat: sealing: load SectorsSummary from sealing SectorStats instead of calling API each time ([filecoin-project/lotus#11353](https://github.com/filecoin-project/lotus/pull/11353))
- fix: shed: additional metrics in `mpool miner-select-messages` ([filecoin-project/lotus#11253](https://github.com/filecoin-project/lotus/pull/11253))
- storage: Return soft err when sector alloc fails in acquire ([filecoin-project/lotus#11338](https://github.com/filecoin-project/lotus/pull/11338))
- feat: miner: log detailed timing breakdown when mining takes longer than the block's timestamp ([filecoin-project/lotus#11228](https://github.com/filecoin-project/lotus/pull/11228))
- fix: shed: make invariants checker work with splitstore ([filecoin-project/lotus#11391](https://github.com/filecoin-project/lotus/pull/11391))
- feat: eth: encode eth tx input as solidity ABI (#11402) ([filecoin-project/lotus#11402](https://github.com/filecoin-project/lotus/pull/11402))
- fix: eth: use the correct state-tree when resolving addresses (#11387) ([filecoin-project/lotus#11387](https://github.com/filecoin-project/lotus/pull/11387))
- fix: chain: make failure to load the chain state fatal (#11426) ([filecoin-project/lotus#11426](https://github.com/filecoin-project/lotus/pull/11426))
- fix: build: an epoch is near an upgrade iff the upgrade is enabled (#11401) ([filecoin-project/lotus#11401](https://github.com/filecoin-project/lotus/pull/11401))
- fix: worker: listen for interrupt signals in GetStorageMinerAPI loop (#11309) ([filecoin-project/lotus#11309](https://github.com/filecoin-project/lotus/pull/11309))
- fix: sync: iterate over returned messages directly (#11373) ([filecoin-project/lotus#11373](https://github.com/filecoin-project/lotus/pull/11373))
- fix: miner: correct duration logs in mineOne ([filecoin-project/lotus#11241](https://github.com/filecoin-project/lotus/pull/11241))
- fix: cli: Add print to unseal cmd ([filecoin-project/lotus#11271](https://github.com/filecoin-project/lotus/pull/11271))
- fix: networking: avoid dialing when trying to handshake peers ([filecoin-project/lotus#11262](https://github.com/filecoin-project/lotus/pull/11262))
- metric milliseconds computation with golang original method (#11403) ([filecoin-project/lotus#11403](https://github.com/filecoin-project/lotus/pull/11403))
- refactor:ffi: replace ClearLayerData with ClearCache (#11352) ([filecoin-project/lotus#11352](https://github.com/filecoin-project/lotus/pull/11352))
- fix: api: compute gasUsedRatio based on max gas in the tipset (#11354) ([filecoin-project/lotus#11354](https://github.com/filecoin-project/lotus/pull/11354))
- fix: api: compute the effective gas cost with the correct base-fee (#11357) ([filecoin-project/lotus#11357](https://github.com/filecoin-project/lotus/pull/11357))
- fix: api: return errors on failure to lookup an eth txn receipt (#11329) ([filecoin-project/lotus#11329](https://github.com/filecoin-project/lotus/pull/11329))
- feat: exchange: change GetBlocks to always fetch the requested number of tipsets ([filecoin-project/lotus#11565](https://github.com/filecoin-project/lotus/pull/11565))
## Dependencies
- update go-libp2p to v0.31.0 ([filecoin-project/lotus#11225](https://github.com/filecoin-project/lotus/pull/11225))
- add support for v12 check invariants and also a default case to reduce future confusion (#11371) ([filecoin-project/lotus#11371](https://github.com/filecoin-project/lotus/pull/11371))
- feat: chain: light-weight patch to fix calibrationnet again by removing move_partitions from built-in actors (#11409) ([filecoin-project/lotus#11409](https://github.com/filecoin-project/lotus/pull/11409))
| Andrew Jackson (Ajax) | 161 | +24328/-12464 | 4148 |
| Łukasz Magiera | 99 | +5238/-2690 | 260 |
| Shrenuj Bansal | 27 | +3402/-1265 | 76 |
| Fridrik Asmundsson | 15 | +1148/-307 | 58 |
| Steven Allen | 15 | +674/-337 | 35 |
| Ian Norden | 1 | +625/-3 | 4 |
| Aarsh Shah | 4 | +227/-167 | 14 |
| Phi | 19 | +190/-183 | 32 |
| Aayush Rajasekaran | 3 | +291/-56 | 16 |
| Mikers | 2 | +76/-262 | 19 |
| Aayush | 14 | +111/-59 | 21 |
| Friðrik Ásmundsson | 1 | +101/-1 | 2 |
| Alejandro Criado-Pérez | 1 | +36/-36 | 27 |
| Jie Hou | 5 | +36/-10 | 5 |
| Florian RUEN | 2 | +24/-19 | 5 |
| Phi-rjan | 3 | +20/-8 | 3 |
| Icarus9913 | 1 | +11/-11 | 6 |
| Jiaying Wang | 3 | +8/-7 | 5 |
| guangwu | 1 | +3/-10 | 2 |
| Marten Seemann | 1 | +6/-6 | 2 |
| simlecode | 1 | +0/-6 | 2 |
| GlacierWalrus | 2 | +0/-5 | 2 |
| Anton Evangelatov | 1 | +2/-2 | 1 |
| Ales Dumikau | 3 | +2/-2 | 3 |
| renran | 1 | +2/-1 | 1 |
| Volker Mische | 1 | +1/-1 | 1 |
| Icarus Wu | 1 | +1/-1 | 1 |
| Hubert | 1 | +1/-1 | 1 |
| Aloxaf | 1 | +1/-1 | 1 |
| Alejandro | 1 | +1/-1 | 1 |
| lazavikmaria | 1 | +1/-0 | 1 |
# v1.25.1 / 2023-12-09
This is a **highly recommended PATCH RELEASE.** The patch release fixes the issue were node operators trying to catch up sync were unable to sync large message blocks/epochs due to an increased number of messages on the network.
This patch release allows for up to 10k messages per block. Additionally, it introduces a limit on the amount of data that can be read at once, ensuring the system can handle worst-case scenarios.
## Improvements
- fix: exchange: allow up to 10k messages per block ([filecoin-project/lotus#11506](https://github.com/filecoin-project/lotus/pull/11506))
# v 1.25.0 / 2023-11-22
This is a highly recommended feature release of Lotus. This optional release supports the Filecoin network version 21 upgrade, codenamed Watermelon 🍉, in addition to the numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.
**The Filecoin network upgrade v21, codenamed Watermelon 🍉, is at epoch 3469380 - 2023-12-12T13:30:00Z**
The full list of [protocol improvements delivered in the network upgrade can be found here](https://github.com/filecoin-project/core-devs/blob/master/Network%20Upgrades/v21.md).
## ☢️ Upgrade Warnings ☢️
- Read through the [changelog of the mandatory v1.24.0 release](https://github.com/filecoin-project/lotus/releases/tag/v1.24.0). Especially the `Migration` and `v12 Builtin Actor Bundle` sections.
- Please remove and clone a new Lotus repo (`git clone https://github.com/filecoin-project/lotus.git`) when upgrading to this release.
- This feature release requires a minimum Go version of v1.20.7 or higher to successfully build Lotus. Go version 1.21.x is not supported yet.
- EthRPC providers, please check out the [new tracing API to Lotus RPC](https://github.com/filecoin-project/lotus/pull/11100)
## ⭐️ Highlights ⭐️
**Unsealing bugfixes and enhancements**
This feature release introduces significant improvements and bugfixes with regards to unsealing, and ensures that unsealing operates as one would expect. Consequently, unsealing of all sector types (deal sectors, snap-sectors without sector keys, and snap-sectors with sector keys) now all function seamlessly.
Some additional unsealing improvements are:
- Unsealing on workers with only sealing paths works. :tada:
- Transferring unsealed files to long-term storage upon successful unsealing. :arrow_right:
- Ensuring no residual files in sealing paths post a successful unsealing operation. :broom:
**SupraSeal C2**
Lotus-workers can now be built to leverage the SupraSeal C2 sealing optimizations in your sealing pipeline. The code optimizations are currently behind the `FFI_USE_CUDA_SUPRASEAL` feature flag. We advice users to test this feature on a test-network, before trying to use it on the mainnet. Users can test out the feature by building their lotus-workers by exporting the `FFI_USE_CUDA_SUPRASEAL=1` enviroment variable, and building from source. For questions about the SupraSeal C2 sealing optimizations, reach out in the #fil-proofs or the #dsa-sealing slack channel.
## New features
- feat: add Eip155ChainID to StateGetNetworkParams ([filecoin-project/lotus#10987](https://github.com/filecoin-project/lotus/pull/10987))
- feat: profiling: state summary and visualization ([filecoin-project/lotus#11012](https://github.com/filecoin-project/lotus/pull/11012))
- feat: vm: allow raw "cbor" in state and use the new go-multicodec ([filecoin-project/lotus#11081](https://github.com/filecoin-project/lotus/pull/11081))
- Add new lotus-shed command for backfillling actor events ([filecoin-project/lotus#11088](https://github.com/filecoin-project/lotus/pull/11088))
- feat: Add new tracing API ([filecoin-project/lotus#11100](https://github.com/filecoin-project/lotus/pull/11100))
- feat: FVM: do not error on unsuccessful implicit messages ([filecoin-project/lotus#11120](https://github.com/filecoin-project/lotus/pull/11120))
- feat: chain node: Move consensus slasher to internal service ([filecoin-project/lotus#11126](https://github.com/filecoin-project/lotus/pull/11126))
- feat: chainstore: FRC-0051: Remove all equivocated blocks from tipsets ([filecoin-project/lotus#11104](https://github.com/filecoin-project/lotus/pull/11104))
- feat: miner: 2 minor refactors ([filecoin-project/lotus#11158](https://github.com/filecoin-project/lotus/pull/11158))
- feat: refactor: return randomness base to FVM without hashing ([filecoin-project/lotus#11167](https://github.com/filecoin-project/lotus/pull/11167))
- feat: Lotus Gateway: add allocation and claim related GET APIs to gateway ([filecoin-project/lotus#11183](https://github.com/filecoin-project/lotus/pull/11183))
- feat: shed: Add exec traces to `lotus-shed msg` ([filecoin-project/lotus#11188](https://github.com/filecoin-project/lotus/pull/11188))
- feat: miner: defensive check for equivocation ([filecoin-project/lotus#11328](https://github.com/filecoin-project/lotus/pull/11328))
## Improvements
- feat: daemon: improvemens to the consensus slasher ([filecoin-project/lotus#10979](https://github.com/filecoin-project/lotus/pull/10979))
- feat: daemon: improvemens to the consensus slasher ([filecoin-project/lotus#11063](https://github.com/filecoin-project/lotus/pull/11063))
- fix: events: Improve performance of event migration from V1 to V2 ([filecoin-project/lotus#11064](https://github.com/filecoin-project/lotus/pull/11064))
- fix: gateway: return an error when an Eth filter is not found ([filecoin-project/lotus#11152](https://github.com/filecoin-project/lotus/pull/11152))
- fix: chainstore: do not get stuck in unhappy equivocation cases ([filecoin-project/lotus#11159](https://github.com/filecoin-project/lotus/pull/11159))
- fix: sealing: Run unsealing in the background for better ux ([filecoin-project/lotus#11177](https://github.com/filecoin-project/lotus/pull/11177))
- fix: build: Allow lotus-wallet to be built independently ([filecoin-project/lotus#11187](https://github.com/filecoin-project/lotus/pull/11187))
- fix: wallet: Make import handle SIGINT/SIGTERM ([filecoin-project/lotus#11190](https://github.com/filecoin-project/lotus/pull/11190))
- fix: markets/dagstore: remove trace goroutine for dagstore wrapper ([filecoin-project/lotus#11191](https://github.com/filecoin-project/lotus/pull/11191))
- fix: chain: Do not update message info cache until after message validation ([filecoin-project/lotus#11202](https://github.com/filecoin-project/lotus/pull/11202))
- fix: chain: cancel long operations upon ctx cancelation ([filecoin-project/lotus#11206](https://github.com/filecoin-project/lotus/pull/11206))
- fix: worker: Convert `DC_[SectorSize]_[ResourceRestriction]` if set ([filecoin-project/lotus#11224](https://github.com/filecoin-project/lotus/pull/11224))
- chore: Extract stable release and post release portion outside of RC testing in template ([filecoin-project/lotus#11000](https://github.com/filecoin-project/lotus/pull/11000))
- fix: docs: include FFI steps in lotus RELEASE_ISSUE_TEMPLATE ([filecoin-project/lotus#11047](https://github.com/filecoin-project/lotus/pull/11047))
- chore: build: update to v1.23.4-dev ([filecoin-project/lotus#11049](https://github.com/filecoin-project/lotus/pull/11049))
- fix: deflake: Use MockProofs ([filecoin-project/lotus#11059](https://github.com/filecoin-project/lotus/pull/11059))
- fix: failing test: Tweak TestWindowPostV1P1NV20 test condition ([filecoin-project/lotus#11121](https://github.com/filecoin-project/lotus/pull/11121))
- fix: CI: make test-unit-rest actually be the rest of the tests ([filecoin-project/lotus#11147](https://github.com/filecoin-project/lotus/pull/11147))
- chore: merge releases into master ([filecoin-project/lotus#11154](https://github.com/filecoin-project/lotus/pull/11154))
- tests: mac seal test ([filecoin-project/lotus#11180](https://github.com/filecoin-project/lotus/pull/11180))
- tests: Take Download out of Sealer time ([filecoin-project/lotus#11182](https://github.com/filecoin-project/lotus/pull/11182))
- feat: test: Test that verified clients can directly transfer datacap, creating allocations ([filecoin-project/lotus#11169](https://github.com/filecoin-project/lotus/pull/11169))
- chore: merge feat/nv21 into master ([filecoin-project/lotus#11201](https://github.com/filecoin-project/lotus/pull/11201))
- ci: Use larger executor for cli tests ([filecoin-project/lotus#11212](https://github.com/filecoin-project/lotus/pull/11212))
- fix: dockerfile: Bump to Go 1.20.7 image ([filecoin-project/lotus#11221](https://github.com/filecoin-project/lotus/pull/11221))
- docs: Update PR template to callout remembering to update CHANGELOG ([filecoin-project/lotus#11232](https://github.com/filecoin-project/lotus/pull/11232))
- chore: add bootstrap node on calibration ([filecoin-project/lotus#11175](https://github.com/filecoin-project/lotus/pull/11175))
# 1.24.0 / 2023-11-22
This is the stable release for the upcoming **MANDATORY** Filecoin network upgrade v21, codenamed Watermelon 🍉, at **epoch 3469380 - 2023-12-12T13:30:00Z**.
The Filecoin network version 21 delivers the following FIPs:
- [FIP0052: Increase Max Sector Commitment to 3.5 years](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0052.md)
- [FIP0073: Remove beneficiary from the self_destruct syscall](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0073.md)
- [FIP0075: Improvements to FVM randomness syscalls](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0075.md)
Full list of the other protocol improvements we are delivering can be found [here](https://github.com/filecoin-project/core-devs/blob/master/Network%20Upgrades/v21.md).
## ☢️ Upgrade Warnings ☢️
This feature release requires a minimum Go version of v1.20.7 or higher to successfully build Lotus. Go version 1.21.x is not supported yet.
## v12 Builtin Actor Bundles
[Builtin actor v12.0.0](https://github.com/filecoin-project/builtin-actors/releases/tag/v12.0.0) is used for supporting this upgrade.
Make sure that your lotus actor bundle matches the v12 actors manifest by running the following cli after upgrading:
We are expecting a heavier than normal state migration for this upgrade due to the amount of state changes introduced for miner sector info. (This is a similar migration as the Shark upgrade, however, we have introduced a couple of migration performance optimizations since then for a smoother upgrade experience.)
All node operators, including storage providers, should be aware that ONE pre-migration is being scheduled 180 epochs before the upgrade, around 2023-12-12T12:00:00Z. It will take around 20-30 minutes for the pre-migration and less than 30 seconds for the final migration, depending on the amount of historical state in the node blockstore and the hardware specs the node is running on. During this time, expect slower block validation times, increased CPU and memory usage, and longer delays for API queries (during our testing, it topped out about 205 RAM(htop) on a 1TiB box).
We recommend node operators (who haven't enabled splitstore `discard` mode) that do not care about historical chain states, to prune the chain blockstore by syncing from a snapshot 1-2 days before the upgrade.
Note to full archival node operators: you may expect it takes some time for the node to complete the final migration, during this period your node will fall out of sync and your chain service may have some disruption. However, you can expect the node to catch up soon after the migration completes. You can test out the migration by running the following on your node in offline mode:
You can check out the [tutorial for benchmarking the network migration here.](https://lotus.filecoin.io/kb/test-migration/)
## BREAKING CHANGE
There is a new protocol limit on how many partition could be submited in one PoSt - if you have any customized tooling for batching PoSts, please update accordingly.
- feat: limit PoSted partitions to 3 ([filecoin-project/lotus#11327](https://github.com/filecoin-project/lotus/pull/11327))
## New features
- Implement and support [FIP0052: Increase Max Sector Commitment to 3.5 years](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0052.md)
- fix: docs: Update SectorLifetime to be in line with FIP-0052 ([filecoin-project/lotus#11314](https://github.com/filecoin-project/lotus/pull/11314))
- Implement and support [FIP0059: Synthetic PoRep](https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0059.md) - Check out the [Lotus documentation for Synthetic PoRep](https://lotus.filecoin.io/storage-providers/advanced-configurations/sealing/#synthetic-porep).
- chore: deps: update FFI, FVM, and actors ([filecoin-project/lotus#11310](https://github.com/filecoin-project/lotus/pull/11310))
- chore: deps: update to final actors ([filecoin-project/lotus#11330](https://github.com/filecoin-project/lotus/pull/11440))
- chore: deps: update to go-state-types v0.12.8 ([filecoin-project/lotus#11339](https://github.com/filecoin-project/lotus/pull/11437))
- chore: deps: update libp2p to v0.30.0 #11434
## Snapshots
The [Forest team](https://filecoinproject.slack.com/archives/C029LPZ5N73) at Chainsafe has launched a brand new lightweight snapshot service that is backed up by forest nodes! This is a great alternative service along with the fil-infra one, and it is compatible with lotus! We recommend lotus users to check it out [here](https://docs.filecoin.io/networks/mainnet#resources)!
# v1.23.3 / 2023-08-01
This feature release of Lotus includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.
This feature release requires a **minimum Go version of v1.19.12 or higher to successfully build Lotus**. Go version 1.20 is also supported, but 1.21 is NOT.
## Highlights
- [Lotus now includes a Slasher tool](https://github.com/filecoin-project/lotus/pull/10928) to monitor the network for Consensus Faults, and report them as appropriate
- The Slasher investigates all incoming blocks, and assesses whether they trigger any of the three Consensus Faults defined in the Filecoin protocol
- If any faults are detected, the Slasher sends a `ReportConsensusFault` message to the faulty miner
- For more information on the Slasher, including how to run it, please find the documentation [here](https://lotus.filecoin.io/lotus/manage/slasher-and-disputer/)
- The Ethereum-like RPC exposed by Lotus is now compatible with EIP-1898: https://github.com/filecoin-project/lotus/pull/10815
- The lotus-miner PieceReader now supports parallel reads: https://github.com/filecoin-project/lotus/pull/10913
- Added new environment variable `LOTUS_EXEC_TRACE_CACHE_SIZE` to configure execution trace cache size ([filecoin-project/lotus#10585](https://github.com/filecoin-project/lotus/pull/10585))
- If unset, we default to caching 16 most recent execution traces. Storage Providers may want to set this to 0, while exchanges may want to crank it up.
## New features
- feat: miner cli: sectors list upgrade-bounds tool ([filecoin-project/lotus#10923](https://github.com/filecoin-project/lotus/pull/10923))
- Add new RPC stress testing tool (lotus-bench rpc) with rich reporting ([filecoin-project/lotus#10761](https://github.com/filecoin-project/lotus/pull/10761))
- feat: Make sure we don't store duplidate actor events caused to reorgs in events.db ([filecoin-project/lotus#11015](https://github.com/filecoin-project/lotus/pull/11015))
- sealing: Use only non-assigned deals when selecting snap sectors ([filecoin-project/lotus#11002](https://github.com/filecoin-project/lotus/pull/11002))
- chore: not display privatekey ([filecoin-project/lotus#11006](https://github.com/filecoin-project/lotus/pull/11006))
- chore: shed: update actor version ([filecoin-project/lotus#11020](https://github.com/filecoin-project/lotus/pull/11020))
- chore: migrate to boxo ([filecoin-project/lotus#10921](https://github.com/filecoin-project/lotus/pull/10921))
- fix: eth: ensure that the event topics are non-nil ([filecoin-project/lotus#10971](https://github.com/filecoin-project/lotus/pull/10971))
- Add comment stating msgIndex is an experimental feature ([filecoin-project/lotus#10968](https://github.com/filecoin-project/lotus/pull/10968))
- feat: cli(compute-state) default to the tipset at the given epoch ([filecoin-project/lotus#10965](https://github.com/filecoin-project/lotus/pull/10965))
- Upgrade urfave dependency which now supports DisableSliceFlagSeparato… ([filecoin-project/lotus#10950](https://github.com/filecoin-project/lotus/pull/10950))
- Add new lotus-shed command for computing eth hash for a given message cid (#10961) ([filecoin-project/lotus#10961](https://github.com/filecoin-project/lotus/pull/10961))
- Prefill GetTipsetByHeight skiplist cache on lotus startup ([filecoin-project/lotus#10955](https://github.com/filecoin-project/lotus/pull/10955))
- Add lotus-shed command for backfilling txhash.db ([filecoin-project/lotus#10932](https://github.com/filecoin-project/lotus/pull/10932))
- chore: deps: update to go-libp2p 0.27.5 ([filecoin-project/lotus#10948](https://github.com/filecoin-project/lotus/pull/10948))
- Small improvement to make gen output ([filecoin-project/lotus#10951](https://github.com/filecoin-project/lotus/pull/10951))
- fix: improve perf of msgindex backfill ([filecoin-project/lotus#10941](https://github.com/filecoin-project/lotus/pull/10941))
- Add timing test for mpool select with a large mpool dump ([filecoin-project/lotus#10650](https://github.com/filecoin-project/lotus/pull/10650))
- feat: slashfilter: drop outdated near-upgrade check ([filecoin-project/lotus#10925](https://github.com/filecoin-project/lotus/pull/10925))
- opt: MinerInfo adds the PendingOwnerAddress field ([filecoin-project/lotus#10927](https://github.com/filecoin-project/lotus/pull/10927))
- feat: itest: force PoSt more aggressively around deadline closure ([filecoin-project/lotus#10926](https://github.com/filecoin-project/lotus/pull/10926))
- test: messagepool: gas rewards are negative if GasFeeCap too low ([filecoin-project/lotus#10649](https://github.com/filecoin-project/lotus/pull/10649))
- fix: types: error out on decoding BlockMsg with extraneous data ([filecoin-project/lotus#10863](https://github.com/filecoin-project/lotus/pull/10863))
- feat: cli: Make compact partitions cmd better ([filecoin-project/lotus#9070](https://github.com/filecoin-project/lotus/pull/9070))
- fix: include extra messages in ComputeState InvocResult output ([filecoin-project/lotus#10628](https://github.com/filecoin-project/lotus/pull/10628))
- feat: pubsub: treat ErrGasFeeCapTooLow as ignore, not reject ([filecoin-project/lotus#10652](https://github.com/filecoin-project/lotus/pull/10652))
- feat: run lotus-shed commands in context that is cancelled on sigterm ([filecoin-project/lotus#10877](https://github.com/filecoin-project/lotus/pull/10877))
- fix:lotus-fountain:set default data-cap same as MinVerifiedDealSize ([filecoin-project/lotus#10920](https://github.com/filecoin-project/lotus/pull/10920))
- pass the right g-recaptcha data
- fix: not call RUnlock ([filecoin-project/lotus#10912](https://github.com/filecoin-project/lotus/pull/10912))
- opt: cli: If present, print Events Root ([filecoin-project/lotus#10893](https://github.com/filecoin-project/lotus/pull/10893))
- Validate that FromBlock/ToBlock epoch is indeed a hex value (#10780) ([filecoin-project/lotus#10780](https://github.com/filecoin-project/lotus/pull/10780))
- fix: remove invalid field UpgradePriceListOopsHeight ([filecoin-project/lotus#10772](https://github.com/filecoin-project/lotus/pull/10772))
- fix: tests: Use mutex-wrapped datastore in storage tests ([filecoin-project/lotus#10846](https://github.com/filecoin-project/lotus/pull/10846))
- Make lotus-fountain UI slightly friendlier ([filecoin-project/lotus#10785](https://github.com/filecoin-project/lotus/pull/10785))
- Make (un)subscribe and filter RPC methods require only read perm ([filecoin-project/lotus#10825](https://github.com/filecoin-project/lotus/pull/10825))
- deps: Update go-jsonrpc to v0.3.1 ([filecoin-project/lotus#10845](https://github.com/filecoin-project/lotus/pull/10845))
- feat: make RunClientTest louder when deals fail ([filecoin-project/lotus#10817](https://github.com/filecoin-project/lotus/pull/10817))
- fix: cli: Change arg wording in change-beneficiary cmd ([filecoin-project/lotus#10823](https://github.com/filecoin-project/lotus/pull/10823))
- refactor: streamline error handling in CheckPendingMessages (#10818) ([filecoin-project/lotus#10818](https://github.com/filecoin-project/lotus/pull/10818))
- feat: Add tmp indices to events table while performing migration to V2
- fix: sync: iterate over returned messages directly #11373
- fix: api: compute the effective gas cost with the correct base-fee #11357
- fix: check invariants: v12 check #11371
- fix: api: compute gasUsedRatio based on max gas in the tipset #11354
# v1.23.2 / 2023-06-28
This is a patch release on top of 1.23.1 containing the fix for https://github.com/filecoin-project/lotus/issues/10906
This fixes the syncing issue seen by many node operators/SPs, usually when performing actions which would result in msgs staying in their mpool for longer periods of time (ex. PSD) resulting in these msgs being republished multiple times and possibly lowering your peer scores. Please refer to the above issue for more details.
We'd recommend everyone to accept this fix to better overall network health
## Improvements
- fix: pubsub: do not treat ErrExistingNonce as Reject
# v1.23.1 / 2023-06-20
This is an optional feature release of Lotus. This feature release includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.
**☢️ Upgrade Warnings ☢️**
If you are upgrading to this release candidate from Lotus v1.22.1, please make sure to read the upgrade warnings section in the [v1.23.0 release first.](https://github.com/filecoin-project/lotus/releases/tag/v1.23.0)
- *Storage providers:* The Lotus-Miner legacy-markets has been disbled by default in this feature release and will be removed in the near term future. Users are adviced to migrate to [Boost](https://boost.filecoin.io) or other SP markets systems.
## Highlights
**🛣 Execution Lanes 🛣**
This feature release introduces VM Execution Lanes! Execution lanes efficiently divide the workload between system processes (chainsync) and RPC requests. This way syncing the chain will not be at the mercy of responding to users' requests and RPC providers nodes should have less problems catching up.
To take advantage of VM Execution Lanes, you need to set up two environment variables:
- `LOTUS_FVM_CONCURRENCY` - read more about how this value should be set to [here](https://lotus.filecoin.io/lotus/configure/ethereum-rpc/#environment-variables)
- `LOTUS_FVM_CONCURRENCY_RESERVED = 4`
**🧱 Aggregation / Batching fixes 🔨**
Numerous aggregation and batching fixes has been included in the feature release. Large `ProveCommitAggregate` and `PreCommitBatching` messages that exceeds the block limit will now automatically be split into smaller messages when sent to the chain.
Additionally we have added a new feature that staggers the amount of ProveCommit messages sent simulatanously to the chain if a storage provider has been aggregating many sectors in ProveCommitAggregate message, but at the time of publishing the BaseFee is below the aggregation threshold. This stagger feature prevents issues where some of the ProveCommit messages fail with the SysErrorOutOfGas message. You can tweak how many messages will be staggered per epoch by changing `MaxSectorProveCommitsSubmittedPerEpoch` in the [sealing section of the config.toml file.](https://lotus.filecoin.io/storage-providers/advanced-configurations/sealing/#sealing-section)
*NB:* While these fixes are great for the reliability of aggregation and batching on the Lotus side, it has been uncovered that aggregating ProveCommit messages for sectors containing verified deals are currently more expensive then single messages due to an issue on the actors side. We therefore do not reccomend our users to aggregate ProveCommit messages when doing verified deals until that issue has been resolved. You can follow the discussion on resolving the issue on the [actors side here.](https://github.com/filecoin-project/FIPs/discussions/689)
**Unsealing CLI/API**
This feature release adds a dedicated `lotus-miner sectors unseal` command and API, allowing you to unseal specific sealed sectors easily.
## New features
- feat: VM Execution Lanes ([filecoin-project/lotus#10551](https://github.com/filecoin-project/lotus/pull/10551))
- Adds VM exections lanes, efficiently dividing the workload between system processes and RPC-requests.
- Add API and CLI to unseal sector (#10626) ([filecoin-project/lotus#10626](https://github.com/filecoin-project/lotus/pull/10626))
- Adds `lotus-miner sectors unseal` cmd, and a API-method to unseal a sector.
- feat: sealing: Split PCA/PCB batches if gas used exceeds block limit ([filecoin-project/lotus#10647](https://github.com/filecoin-project/lotus/pull/10647))
- Splits ProveCommitAggregate and PreCommitBatch messages into multiple messages if the message exceeds the block limit.
- Staggers the amount of ProveCommit messages sent simultanously if a storage provider has been aggregating many message, but at the moment of publishing the BaseFee is below the threshold for aggregation to prevent unwanted SysErrorOutOfGas issues.
- Set default for MaxSectorProveCommitsSubmittedPerEpoch ([filecoin-project/lotus#10728](https://github.com/filecoin-project/lotus/pull/10728))
- Sets the default amount of ProveCommits submitted per epoch to 20.
- feat: populate the index on snapshot import ([filecoin-project/lotus#10556](https://github.com/filecoin-project/lotus/pull/10556))
- feat:chain: Message Index (**HIGHLY EXPERIMENTAL**) ([filecoin-project/lotus#10452](https://github.com/filecoin-project/lotus/pull/10452))
- MVP of a message index that allows us to accelrate StateSearchMessage and related functionality, and eventually accelerate critical chain calls (follow up).
- feat: Add small cache to execution traces ([filecoin-project/lotus#10517](https://github.com/filecoin-project/lotus/pull/10517))
- fix: sealing: Make lotus-worker report GPU usage to miner during ReplicaUpdate task (#10806) ([filecoin-project/lotus#10806](https://github.com/filecoin-project/lotus/pull/10806))
- fix:splitstore:Don't block when potentially holding txnLk as writer ([filecoin-project/lotus#10811](https://github.com/filecoin-project/lotus/pull/10811))
- fix: prover: Propagate skipped sectors in local PoSt
- fix: unseal: check if sealed/update sector exists ([filecoin-project/lotus#10639](https://github.com/filecoin-project/lotus/pull/10639))
- fix: sealing pipeline: Allow nil message in TerminateWait ([filecoin-project/lotus#10696](https://github.com/filecoin-project/lotus/pull/10696))
- fix: cli: Check if the sectorID exists before removing ([filecoin-project/lotus#10611](https://github.com/filecoin-project/lotus/pull/10611))
- fix: cli: Make `net connect` to miner address work ([filecoin-project/lotus#10599](https://github.com/filecoin-project/lotus/pull/10599))
- fix: log: Stop logging `file does not exists` ([filecoin-project/lotus#10588](https://github.com/filecoin-project/lotus/pull/10588))
- Update config default value (#10605) ([filecoin-project/lotus#10605](https://github.com/filecoin-project/lotus/pull/10605))
- fix: cap the message gas limit at the block gas limit (#10637) ([filecoin-project/lotus#10637](https://github.com/filecoin-project/lotus/pull/10637))
- fix:mpool: prune excess messages before selection ([filecoin-project/lotus#10648](https://github.com/filecoin-project/lotus/pull/10648))
- fix: proving: Initialize slice with with same length as partition ([filecoin-project/lotus#10569](https://github.com/filecoin-project/lotus/pull/10569))
- perf: Address performance of EthGetTransactionCount ([filecoin-project/lotus#10700](https://github.com/filecoin-project/lotus/pull/10700))
- fix: sync: reduce log from error to info ([filecoin-project/lotus#10759](https://github.com/filecoin-project/lotus/pull/10759))
- fix: state: lotus-miner info should show deals info without admin permission ([filecoin-project/lotus#10323](https://github.com/filecoin-project/lotus/pull/10323))
- fix: tvx: make extract-multiple support the FVM ([filecoin-project/lotus#10714](https://github.com/filecoin-project/lotus/pull/10714))
- feat: badger: add a has check before writing to reduce duplicates ([filecoin-project/lotus#10680](https://github.com/filecoin-project/lotus/pull/10680))
- fix: chain: record heaviest tipset before notifying (#10694) ([filecoin-project/lotus#10694](https://github.com/filecoin-project/lotus/pull/10694))
- fix: Eth JSON-RPC api: handle messages with gasFeeCap less than baseFee (#10614) ([filecoin-project/lotus#10614](https://github.com/filecoin-project/lotus/pull/10614))
- fix: events: don't set GC confidence to 1 ([filecoin-project/lotus#10713](https://github.com/filecoin-project/lotus/pull/10713))
- feat: sync: validate (early) that blocks fall within range (#10691) ([filecoin-project/lotus#10691](https://github.com/filecoin-project/lotus/pull/10691))
- chainstore: Fix raw blocks getting scanned for links during snapshots (#10684) ([filecoin-project/lotus#10684](https://github.com/filecoin-project/lotus/pull/10684))
- fix: check for nil bcastDict (#10646) ([filecoin-project/lotus#10646](https://github.com/filecoin-project/lotus/pull/10646))
- fix: make state compute --html work with unknown methods ([filecoin-project/lotus#10619](https://github.com/filecoin-project/lotus/pull/10619))
- shed: get balances of evm accounts ([filecoin-project/lotus#10489](https://github.com/filecoin-project/lotus/pull/10489))
- feat: Use MessageIndex in WaitForMessage ([filecoin-project/lotus#10587](https://github.com/filecoin-project/lotus/pull/10587))
- fix: searchForIndexedMsg always returns an error ([filecoin-project/lotus#10586](https://github.com/filecoin-project/lotus/pull/10586))
- Fix: export-range: Ignore ipld Blocks not found in Receipts. ([filecoin-project/lotus#10535](https://github.com/filecoin-project/lotus/pull/10535))
- feat: stmgr: speed up calculation of genesis circ supply ([filecoin-project/lotus#10553](https://github.com/filecoin-project/lotus/pull/10553))
- fix: gas estimation: don't special case paych collects ([filecoin-project/lotus#10549](https://github.com/filecoin-project/lotus/pull/10549))
- fix: tracer: emit raw peer ids for compatibility with libp2p tracer ([filecoin-project/lotus#10271](https://github.com/filecoin-project/lotus/pull/10271))
- Merge branch 'feat/new-gw-methods'
## Dependencies
- chore: deps: update to go-libp2p 0.27.5
- devs: update libp2p #10937
- chore: deps: update to FVM 3.3.1 ([filecoin-project/lotus#10786](https://github.com/filecoin-project/lotus/pull/10786))
- chore: boxo: migrate from go-libipfs to boxo ([filecoin-project/lotus#10562](https://github.com/filecoin-project/lotus/pull/10562))
- chore: deps: update to go-state-types v0.11.0-alpha-3 ([filecoin-project/lotus#10606](https://github.com/filecoin-project/lotus/pull/10606))
This is the stable feature release for the upcoming MANDATORY network upgrade at `2023-04-27T13:00:00Z`, epoch `2809800`. This feature release delivers the nv19 Lighting and nv20 Thunder network upgrade for mainnet, and includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.
@@ -73,7 +73,7 @@ All releases under an odd minor version number indicate **feature releases**. Th
Feature releases include new development and bug fixes. They are not mandatory, but still highly recommended, **as they may contain critical security fixes**. Note that some of these releases may be very small patch releases that include critical hotfixes. There is no way to distinguish between a bug fix release and a feature release on the "feature" version. Both cases will use the "patch" version number.
We aim to ship a new feature release of the Lotus software from our development (master) branch every 3 weeks, so users can expect a regular cadence of Lotus feature releases. Note that mandatory releases for network upgrades may disrupt this schedule. For more, see the Release Cycle section (TODO: Link).
We aim to ship a new feature release of the Lotus software from our development (master) branch every 3 weeks, so users can expect a regular cadence of Lotus feature releases. Note that mandatory releases for network upgrades may disrupt this schedule. For more, see the [Release Cycle section](#release-cycle).
@echo "lotus-miner service installed. Don't forget to run 'sudo systemctl start lotus-miner' to start it and 'sudo systemctl enable lotus-miner' for it to be enabled on startup."
@echo "lotus-provider service installed. Don't forget to run 'sudo systemctl start lotus-provider' to start it and 'sudo systemctl enable lotus-provider' for it to be enabled on startup."
@@ -133,6 +133,8 @@ Note: The default branch `master` is the dev branch where the latest new feature
6. You should now have Lotus installed. You can now [start the Lotus daemon and sync the chain](https://lotus.filecoin.io/lotus/install/linux/#start-the-lotus-daemon-and-sync-the-chain).
7. (Optional) Follow the [Setting Up Prometheus and Grafana](https://github.com/filecoin-project/lotus/tree/master/metrics/README.md) guide for detailed instructions on setting up a working monitoring system running against a local running lotus node.
## License
Dual-licensed under [MIT](https://github.com/filecoin-project/lotus/blob/master/LICENSE-MIT) + [Apache 2.0](https://github.com/filecoin-project/lotus/blob/master/LICENSE-APACHE)
Some files were not shown because too many files have changed in this diff
Show More
Reference in New Issue
Block a user
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
jonathanface
Phi-rjan
Łukasz Magiera
Andrew Jackson (Ajax)
Aayush Rajasekaran
Jiaying Wang
jennijuju
Łukasz Magiera
Steven Allen
Rob Quist
Aarsh Shah
Friðrik Ásmundsson
renran
Ales Dumikau
lazavikmaria
GlacierWalrus
Icarus Wu
Florian RUEN
Anton Evangelatov
Mikers
Jiaying Wang
Alejandro Criado-Pérez
Hubert
Volker MischeandSteven Allen
Ian Norden
guangwu
Shrenuj Bansal
Jie Hou
Jie Hou
hunjixin
Aloxaf
Marten Seemann
Rod Vagg
Yolan Romailler
gammazero
Shrenuj Bansal
LexLuthr
simlecode
nemo
Mladen Trišić
ZenGround0
zenground0
mx
beck
hannahhoward
Adin Schmahmann
Jorropo
huangxinheng@storswift.com
swift-mx
Jakub Sztandera
PARTHIBAN
Jianhui Xie
Yolan Romailler
Maciej Witowski
cortze
Jacob Heun
Ian Davis
ychiao
Hector Sanjuan
vyzo
Zeng Li
adlrochaandAayush Rajasekaran
Arsenii Petrovich
dumikau
Ian Davis
Marcel Telka
Leng Bo