cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
f929ae17d7
lotus/CHANGELOG.md
Rod Vagg e5ccf1915d feat(config): move Fevm.Events->Events, implement soft deprecation 
Closes: https://github.com/filecoin-project/lotus/issues/11679

* Introduce a `moved:"To.New.Config"` tag which prints a stderr warning when
  you use one of these, but will move any set value to the new location if the
	new location isn't already set itself.
* Look for `X is DEPRECATED` to hold certain fields back from documentation.
* Use `toml:"omitempty"` to prevent the default config output from having these
  deprecated values.
2024-03-11 13:53:09 +11:00

469 KiB
Raw Blame History

Lotus changelog

UNRELEASED

New features

Improvements

v1.26.0-rc2 / 2024-03-0y

This is a release candidate of the upcoming MANDATORY Lotus v1.26.0 release, which will deliver the Filecoin network version 22, codenamed Dragon 🐉.

This release candidate sets the calibration network to upgrade at epoch 1427974, which is 2024-03-11T14:00:00Z This release does NOT set the mainnet upgrade epoch yet, in which will be updated in the final release. The Filecoin network version 22 delivers the following FIPs:

v13 Builtin Actor Bundle

The actor bundles for the calibration network can be checked as follows:

lotus state actor-cids --network-version=22
Network Version: 22
Actor Version: 13
Manifest CID: bafy2bzacea4firkyvt2zzdwqjrws5pyeluaesh6uaid246tommayr4337xpmi

Actor             CID  
account           bafk2bzaceb3j36ri5y5mfklgp5emlvrms6g4733ss2j3l7jismrxq6ng3tcc6
cron              bafk2bzaceaz6rocamdxehgpwcbku6wlapwpgzyyvkrploj66mlqptsulf52bs
datacap           bafk2bzacea22nv5g3yngpxvonqfj4r2nkfk64y6yw2malicm7odk77x7zuads
eam               bafk2bzaceatqtjzj7623i426noaslouvluhz6e3md3vvquqzku5qj3532uaxg
ethaccount        bafk2bzacean3hs7ga5csw6g3uu7watxfnqv5uvxviebn3ba6vg4sagwdur5pu
evm               bafk2bzacec5ibmbtzuzjgwjmksm2n6zfq3gkicxqywwu7tsscqgdzajpfctxk
init              bafk2bzaced5sq72oemz6qwi6yssxwlos2g54zfprslrx5qfhhx2vlgsbvdpcs
multisig          bafk2bzacedbgei6jkx36fwdgvoohce4aghvpohqdhoco7p4thszgssms7olv2
paymentchannel    bafk2bzaceasmgmfsi4mjanxlowsub65fmevhzky4toeqbtw4kp6tmu4kxjpgq
placeholder       bafk2bzacedfvut2myeleyq67fljcrw4kkmn5pb5dpyozovj7jpoez5irnc3ro
reward            bafk2bzacedjyp6ll5ez27dfgldjj4tntxfvyp4pa5zkk7s5uhipzqjyx2gmuc
storagemarket     bafk2bzaceabolct6qdnefwcrtati2us3sxtxfghyqk6aamfhl6byyefmtssqi
storageminer      bafk2bzaceckzw3v7wqliyggvjvihz4wywchnnsie4frfvkm3fm5znb64mofri
storagepower      bafk2bzacea7t4wynzjajl442mpdqbnh3wusjusqtnzgpvefvweh4n2tgzgqhu
system            bafk2bzacedjnrb5glewazsxpcx6rwiuhl4kwrfcqolyprn6rrjtlzmthlhdq6
verifiedregistry  bafk2bzacednskl3bykz5qpo54z2j2p4q44t5of4ktd6vs6ymmg2zebsbxazkm

Migration

We are expecting a bit heavier than normal state migration for this upgrade due to the amount of state changes introduced with Direct Data Onboarding.

All node operators, including storage providers, should be aware that ONE pre-migration is being scheduled 120 epochs before the upgrade. It will take around 10-20 minutes for the pre-migration and less than 30 seconds for the final migration, depending on the amount of historical state in the node blockstore and the hardware specs the node is running on. During this time, expect slower block validation times, increased CPU and memory usage, and longer delays for API queries

We recommend node operators (who haven't enabled splitstore discard mode) that do not care about historical chain states, to prune the chain blockstore by syncing from a snapshot 1-2 days before the upgrade.

You can test out the migration by running running the benchmarking a network migration tutorial.

For certain node operators, such as full archival nodes or systems that need to keep large amounts of state (RPC providers), completing the pre-migration in time before the network upgrade might not be achievable. For those node operators, it is recommended to skip the pre-migration and run the non-cached migration (i.e., just running the migration at the exact upgrade epoch), and schedule for some downtime during the upgrade epoch. Operators of such nodes can read the How to disable premigration in network upgrade tutorial.

New features

Improvements

Tracing API

Replace the CodeCid field in the message trace (added in 1.23.4) with an InvokedActor field.

Before:

{
    "Msg": {
        "From": ...,
        "To": ...,
        ...
        "CodeCid": ... // The actor's code CID.
    }
    "MsgRct": ...,
    "GasCharges": [],
    "Subcalls": [],
}

After:

{
    "Msg": {
        "From": ...,
        "To": ...
    }
    "InvokedActor": {         // The invoked actor (ommitted if the actor wasn't invoked).
        "Id": 1234,           // The ID of the actor.
        "State": {            // The actor's state object (may change between network versions).
           "Code": ...,       // The actor's code CID.
           "Head": ...,       // The actor's state-root (when invoked).
           "CallSeqNum": ..., // The actor's nonce.
           "Balance": ...,    // The actor's balance (when invoked).
           "Address": ...,    // Delegated address (FEVM only).
        }
    }
    "MsgRct": ...,
    "GasCharges": [],
    "Subcalls": [],
}

This means the trace now contains an accurate "snapshot" of the actor at the time of the call, information that may not be present in the final state-tree (e.g., due to reverts). This will hopefully improve the performance and accuracy of indexing services.

Ethereum Tracing API (trace_block and trace_replayBlockTransactions)

For those with the Ethereum JSON-RPC API enabled, the experimental Ethereum Tracing API has been improved significantly and should be considered "functional". However, it's still new and should be tested extensively before relying on it. This API translates FVM traces to Ethereum-style traces, implementing the OpenEthereum trace_block and trace_replayBlockTransactions APIs.

This release fixes numerous bugs with this API and now ABI-encodes non-EVM inputs/outputs as if they were explicit EVM calls to handle_filecoin_method for better block explorer compatibility.

However, there are some significant limitations:

  1. The Geth APIs are not implemented, only the OpenEthereum (Erigon, etc.) APIs.
  2. Block rewards are not (yet) included in the trace.
  3. Selfdestruct operations are not included in the trace.
  4. EVM smart contract "create" events always specify 0xfe as the "code" for newly created EVM smart contracts.

Additionally, Filecoin is not Ethereum no matter how much we try to provide API/tooling compatibility. This API attempts to translate Filecoin semantics into Ethereum semantics as accurately as possible, but it's hardly the best source of data unless you need Filecoin to look like an Ethereum compatible chain. If you're trying to build a new integration with Filecoin, please use the native StateCompute method instead.

GetActorEvents and SubscribeActorEvents

FIP-0049 introduced Actor Events that can be emitted by user programmed actors. FIP-0083 introduces new events emitted by the builtin Verified Registry, Miner and Market Actors. These new events for builtin actors are being activated with network version 22 to coincide with Direct Data Onboarding as defined in FIP-0076 which introduces additional flexibility for data onboarding. Sector, Deal and DataCap lifecycles can be tracked with these events, providing visibility and options for programmatic responses to changes in state.

Actor events are available on message receipts, but can now be retrieved from a node using the new GetActorEvents and SubscribeActorEvents methods. These methods allow for querying and subscribing to actor events, respectively. They depend on the Lotus node both collecting events (with Events.RealTimeFilterAPI and Events.HistoricFilterAPI) and being enabled with the new configuration option Events.EnableActorEventsAPI. Note that a Lotus node can only respond to requests for historic events that it retains in its event store.

Both GetActorEvents and SubscribeActorEvents take a filter parameter which can optionally filter events on:

  • Addresses of the actor(s) emitting the event
  • Specific Fields within the event
  • FromHeight and ToHeight to filter events by block height
  • TipSetKey to restrict events contained within a specific tipset

GetActorEvents provides a one-time query for actor events, while SubscribeActorEvents provides a long-lived connection (via websockets) to the Lotus node, allowing for real-time updates on actor events. The subscription can be cancelled by the client at any time.

Events Configuration Changes

All configuration options previously under Fevm.Events are now in the top-level Events section along with the new Events.EnableActorEventsAPI option mentioned above. If you have non-default options in [Events] under [Fevm] in your configuration file, please move them to the top-level [Events].

While Fevm.Events.* options are deprecated and replaced by Events.*, any existing custom values will be respected if their new form isn't set, but a warning will be printed to standard error upon startup. Support for these deprecated options will be removed in a future Lotus release, so please migrate your configuration promptly.

GetAllClaims and GetAllAlocations

Additionally the methods GetAllAllocations and GetAllClaims has been added to the Lotus API. These methods lists all the available allocations and claims available in the actor state.

Lotus CLI

The filplus commands used for listing allocations and claims have been updated. If no argument is provided to the either command, they will list out all the allocations and claims in the verified registry actor. The output list columns have been modified to AllocationID and ClaimID instead of ID.

lotus filplus list-allocations --help
NAME:
   lotus filplus list-allocations - List allocations available in verified registry actor or made by a client if specified

USAGE:
   lotus filplus list-allocations [command options] clientAddress

OPTIONS:
   --expired   list only expired allocations (default: false)
   --json      output results in json format (default: false)
   --help, -h  show help


lotus filplus list-claims --help     
NAME:
   lotus filplus list-claims - List claims available in verified registry actor or made by provider if specified

USAGE:
   lotus filplus list-claims [command options] providerAddress

OPTIONS:
   --expired   list only expired claims (default: false)
   --help, -h  show help

Dependencies

Others

v1.25.2 / 2024-01-11

This is an optional but highly recommended feature release of Lotus, as it includes fixes for synchronizations issues that users have experienced. The feature release also introduces Lotus-Provider in its alpha testing phase, as well as the ability to call external PC2-binaries during the sealing process.

☢️ Upgrade Warnings ☢️

There are no upgrade warnings for this feature release.

⭐️ Highlights ⭐️

Lotus-Provider

The feature release ships the alpha release of the new Lotus-Provider binary, together with its initial features - High Availability of WindowPoSt and WinningPoSt.

So what is so exciting about Lotus-Provider:

High Availability

  • You can run as many Lotus-Provider instances as you want for both WindowPoSt and WinningPOSt.
  • You can connect them to as many clustered Yugabyte instances as you want to. This allows for an NxN configuration where all instances can communicate with all others.
  • You have the option to connect different instances to different chain daemons.

Simplicity

  • Once the configuration is in the database, setting up a new machine with Lotus-Provider is straightforward. Simply start the binary with the correct flags to find YugabyteDB and specify which configuration layers it should use.

Durability

  • Lotus-Provider is designed with robustness in mind. Updates to the system are handled seamlessly, ensuring that performance and stability are maintained when taking down machines in your cluster for updates.

Read more about Lotus-Provider in the documentation here. And check out the how you can migrate from Lotus-Miner to Lotus-Provider here. (Only recommended in testnets while its in Alpha)

External PC2-binaries

In this feature release, storage providers can call external PC2-binaries during the sealing process. This allows storage providers to leverage the SupraSeal PC2 binary, which has been shown to improve sealing speed in the PC2-phase. For instance, our current benchmarks show that an NVIDIA RTX A5000 card was able to complete PC2 in approximately 2.5 minutes.

We have verified that SupraSeal PC2 functions properly with Committed Capacity (CC) sectors, both SyntheticPoReps and non-Synthetic PoReps. However calling SupraSeal PC2 with deal sectors is not supported in this feature release.

For more information on how to use SupraSeal PC2 with your lotus-worker, as well as how to use feature, please refer to the documentation.

New features

  • feat: sturdypost work branch (filecoin-project/lotus#11405)
    • Adds the Lotus-Provider binary, and the HarmonyDB framework.
  • feat: worker: Support delegating precommit2 to external binary (filecoin-project/lotus#11185)
    • Allows for delegating PreCommit2 to an exteranl binary.
  • feat: build: Add SupraSeal-PC2 binary script (filecoin-project/lotus#11430)
    • Adds a script for building the SupraSeal-PC2 binary easily.
  • Feat: daemon: Auto remove existing chain if importing chain file or snapshot (filecoin-project/lotus#11277)
    • Auto removes the existing chain when importing a snapshot.
  • feat: Add ETA to lotus sync wait (#11211) (filecoin-project/lotus#11211)
    • Adds a ETA indicator to lotus sync wait, so you can get an estimate for how long until sync is completed.
  • feat: mpool/wdpost: Maximize feecap config (filecoin-project/lotus#9746)
    • Adds a Maximixe FeeCap Config
  • feat: Add lotus-bench cli option to stress test any binary (filecoin-project/lotus#11270)
    • Enables the Lotus-Bench to run any binary and analyze their latency and histogram distribution, track most common errors, perform stress testing under different concurrency levels and see how it works under different QPS.
  • feat: chain import: don't walk to genesis - 2-3x faster snapshot import (#11446) (filecoin-project/lotus#11446)
    • Improves Snapshot import speed, by not walking back to genesis on import.
  • feat: metric: export Mpool message count (filecoin-project/lotus#11361)
    • Adds the mpool count as a prometheus metric.
  • feat: bench: flag to output GenerateWinningPoStWithVanilla params (filecoin-project/lotus#11460)

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
Andrew Jackson (Ajax) 161 +24328/-12464 4148
Łukasz Magiera 99 +5238/-2690 260
Shrenuj Bansal 27 +3402/-1265 76
Fridrik Asmundsson 15 +1148/-307 58
Steven Allen 15 +674/-337 35
Ian Norden 1 +625/-3 4
Aarsh Shah 4 +227/-167 14
Phi 19 +190/-183 32
Aayush Rajasekaran 3 +291/-56 16
Mikers 2 +76/-262 19
Aayush 14 +111/-59 21
Friðrik Ásmundsson 1 +101/-1 2
Alejandro Criado-Pérez 1 +36/-36 27
Jie Hou 5 +36/-10 5
Florian RUEN 2 +24/-19 5
Phi-rjan 3 +20/-8 3
Icarus9913 1 +11/-11 6
Jiaying Wang 3 +8/-7 5
guangwu 1 +3/-10 2
Marten Seemann 1 +6/-6 2
simlecode 1 +0/-6 2
GlacierWalrus 2 +0/-5 2
Anton Evangelatov 1 +2/-2 1
Ales Dumikau 3 +2/-2 3
renran 1 +2/-1 1
Volker Mische 1 +1/-1 1
Icarus Wu 1 +1/-1 1
Hubert 1 +1/-1 1
Aloxaf 1 +1/-1 1
Alejandro 1 +1/-1 1
lazavikmaria 1 +1/-0 1

v1.25.1 / 2023-12-09

This is a highly recommended PATCH RELEASE. The patch release fixes the issue were node operators trying to catch up sync were unable to sync large message blocks/epochs due to an increased number of messages on the network.

This patch release allows for up to 10k messages per block. Additionally, it introduces a limit on the amount of data that can be read at once, ensuring the system can handle worst-case scenarios.

Improvements

releases

v 1.25.0 / 2023-11-22

This is a highly recommended feature release of Lotus. This optional release supports the Filecoin network version 21 upgrade, codenamed Watermelon 🍉, in addition to the numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

The Filecoin network upgrade v21, codenamed Watermelon 🍉, is at epoch 3469380 - 2023-12-12T13:30:00Z

The full list of protocol improvements delivered in the network upgrade can be found here.

☢️ Upgrade Warnings ☢️

  • Read through the changelog of the mandatory v1.24.0 release. Especially the Migration and v12 Builtin Actor Bundle sections.
  • Please remove and clone a new Lotus repo (git clone https://github.com/filecoin-project/lotus.git) when upgrading to this release.
  • This feature release requires a minimum Go version of v1.20.7 or higher to successfully build Lotus. Go version 1.21.x is not supported yet.
  • EthRPC providers, please check out the new tracing API to Lotus RPC

⭐️ Highlights ⭐️

Unsealing bugfixes and enhancements

This feature release introduces significant improvements and bugfixes with regards to unsealing, and ensures that unsealing operates as one would expect. Consequently, unsealing of all sector types (deal sectors, snap-sectors without sector keys, and snap-sectors with sector keys) now all function seamlessly.

Some additional unsealing improvements are:

  • Unsealing on workers with only sealing paths works. 🎉
  • Transferring unsealed files to long-term storage upon successful unsealing. ➡️
  • Ensuring no residual files in sealing paths post a successful unsealing operation. 🧹

SupraSeal C2

Lotus-workers can now be built to leverage the SupraSeal C2 sealing optimizations in your sealing pipeline. The code optimizations are currently behind the FFI_USE_CUDA_SUPRASEAL feature flag. We advice users to test this feature on a test-network, before trying to use it on the mainnet. Users can test out the feature by building their lotus-workers by exporting the FFI_USE_CUDA_SUPRASEAL=1 enviroment variable, and building from source. For questions about the SupraSeal C2 sealing optimizations, reach out in the #fil-proofs or the #dsa-sealing slack channel.

New features

Improvements

Dependencies

Others

1.24.0 / 2023-11-22

This is the stable release for the upcoming MANDATORY Filecoin network upgrade v21, codenamed Watermelon 🍉, at epoch 3469380 - 2023-12-12T13:30:00Z.

The Filecoin network version 21 delivers the following FIPs:

Full list of the other protocol improvements we are delivering can be found here.

☢️ Upgrade Warnings ☢️

This feature release requires a minimum Go version of v1.20.7 or higher to successfully build Lotus. Go version 1.21.x is not supported yet.

v12 Builtin Actor Bundles

Builtin actor v12.0.0 is used for supporting this upgrade. Make sure that your lotus actor bundle matches the v12 actors manifest by running the following cli after upgrading:

./lotus state actor-cids --network-version 21
Network Version: 21
Actor Version: 12
Manifest CID: bafy2bzaceapkgfggvxyllnmuogtwasmsv5qi2qzhc2aybockd6kag2g5lzaio

Actor             CID  
datacap           bafk2bzacebpiwb2ml4qbnnaayxumtk43ryhc63exdgnhivy3hwgmzemawsmpq
paymentchannel    bafk2bzacectv4cm47bnhga5febf3lo3fq47g72kmmp2xd5s6tcxz7hiqdywa4
storagemarket     bafk2bzacedylkg5am446lcuih4voyzdn4yjeqfsxfzh5b6mcuhx4mok5ph5c4
storagepower      bafk2bzacecsij5tpfzjpfuckxvccv2p3bdqjklkrfyyoei6lx5dyj5j4fvjm6
cron              bafk2bzacechxjkfe2cehx4s7skj3wzfpzf7zolds64khrrrs66bhazsemktls
eam               bafk2bzaceb3elj4hfbbjp7g5bptc7su7mptszl4nlqfedilxvstjo5ungm6oe
ethaccount        bafk2bzaceb4gkau2vgsijcxpfuq33bd7w3efr2rrhxrwiacjmns2ntdiamswq
reward            bafk2bzacealqnxn5lwzwexd6reav4dppypquklx2ujlnvaxiqk2tzstyvkp5u
verifiedregistry  bafk2bzacedudgflxc75c77c6zkmfyq4u2xuk7k6xw6dfdccarjrvxx453b77q
evm               bafk2bzacecmnyfiwb52tkbwmm2dsd7ysi3nvuxl3lmspy7pl26wxj4zj7w4wi
placeholder       bafk2bzacedfvut2myeleyq67fljcrw4kkmn5pb5dpyozovj7jpoez5irnc3ro
storageminer      bafk2bzacedo75pabe4i2l3hvhtsjmijrcytd2y76xwe573uku25fi7sugqld6
system            bafk2bzacebfqrja2hip7esf4eafxjmu6xcogoqu5xxtgdg7xa5szgvvdguchu
account           bafk2bzaceboftg75mdiba7xbo2i3uvgtca4brhnr3u5ptihonixgpnrvhpxoa
init              bafk2bzacebllyegx5r6lggf6ymyetbp7amacwpuxakhtjvjtvoy2bfkzk3vms

Migration

We are expecting a heavier than normal state migration for this upgrade due to the amount of state changes introduced for miner sector info. (This is a similar migration as the Shark upgrade, however, we have introduced a couple of migration performance optimizations since then for a smoother upgrade experience.)

All node operators, including storage providers, should be aware that ONE pre-migration is being scheduled 180 epochs before the upgrade, around 2023-12-12T12:00:00Z. It will take around 20-30 minutes for the pre-migration and less than 30 seconds for the final migration, depending on the amount of historical state in the node blockstore and the hardware specs the node is running on. During this time, expect slower block validation times, increased CPU and memory usage, and longer delays for API queries (during our testing, it topped out about 205 RAM(htop) on a 1TiB box).

We recommend node operators (who haven't enabled splitstore discard mode) that do not care about historical chain states, to prune the chain blockstore by syncing from a snapshot 1-2 days before the upgrade.

Note to full archival node operators: you may expect it takes some time for the node to complete the final migration, during this period your node will fall out of sync and your chain service may have some disruption. However, you can expect the node to catch up soon after the migration completes. You can test out the migration by running the following on your node in offline mode:

  1. lotus chain head | head -n1
  2. Stop Lotus daemon
  3. ./lotus-shed migrate-state --repo=[path-to-your-lotus-repo] 21 [output-of-step-1]

You can check out the tutorial for benchmarking the network migration here.

BREAKING CHANGE

There is a new protocol limit on how many partition could be submited in one PoSt - if you have any customized tooling for batching PoSts, please update accordingly.

New features

Improvements

Dependencies

Snapshots

The Forest team at Chainsafe has launched a brand new lightweight snapshot service that is backed up by forest nodes! This is a great alternative service along with the fil-infra one, and it is compatible with lotus! We recommend lotus users to check it out here!

v1.23.3 / 2023-08-01

This feature release of Lotus includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

This feature release requires a minimum Go version of v1.19.12 or higher to successfully build Lotus. Go version 1.20 is also supported, but 1.21 is NOT.

Highlights

  • Lotus now includes a Slasher tool to monitor the network for Consensus Faults, and report them as appropriate
    • The Slasher investigates all incoming blocks, and assesses whether they trigger any of the three Consensus Faults defined in the Filecoin protocol
    • If any faults are detected, the Slasher sends a ReportConsensusFault message to the faulty miner
    • For more information on the Slasher, including how to run it, please find the documentation here
  • The Ethereum-like RPC exposed by Lotus is now compatible with EIP-1898: https://github.com/filecoin-project/lotus/pull/10815
  • The lotus-miner PieceReader now supports parallel reads: https://github.com/filecoin-project/lotus/pull/10913
  • Added new environment variable LOTUS_EXEC_TRACE_CACHE_SIZE to configure execution trace cache size (filecoin-project/lotus#10585)
    • If unset, we default to caching 16 most recent execution traces. Storage Providers may want to set this to 0, while exchanges may want to crank it up.

New features

Improvements && Bug Fixes

v1.23.2 / 2023-06-28

This is a patch release on top of 1.23.1 containing the fix for https://github.com/filecoin-project/lotus/issues/10906 This fixes the syncing issue seen by many node operators/SPs, usually when performing actions which would result in msgs staying in their mpool for longer periods of time (ex. PSD) resulting in these msgs being republished multiple times and possibly lowering your peer scores. Please refer to the above issue for more details. We'd recommend everyone to accept this fix to better overall network health

Improvements

  • fix: pubsub: do not treat ErrExistingNonce as Reject

v1.23.1 / 2023-06-20

This is an optional feature release of Lotus. This feature release includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

☢️ Upgrade Warnings ☢️

If you are upgrading to this release candidate from Lotus v1.22.1, please make sure to read the upgrade warnings section in the v1.23.0 release first.

  • Storage providers: The Lotus-Miner legacy-markets has been disbled by default in this feature release and will be removed in the near term future. Users are adviced to migrate to Boost or other SP markets systems.

Highlights

🛣 Execution Lanes 🛣 This feature release introduces VM Execution Lanes! Execution lanes efficiently divide the workload between system processes (chainsync) and RPC requests. This way syncing the chain will not be at the mercy of responding to users' requests and RPC providers nodes should have less problems catching up.

To take advantage of VM Execution Lanes, you need to set up two environment variables:

  • LOTUS_FVM_CONCURRENCY - read more about how this value should be set to here
  • LOTUS_FVM_CONCURRENCY_RESERVED = 4

🧱 Aggregation / Batching fixes 🔨

Numerous aggregation and batching fixes has been included in the feature release. Large ProveCommitAggregate and PreCommitBatching messages that exceeds the block limit will now automatically be split into smaller messages when sent to the chain.

Additionally we have added a new feature that staggers the amount of ProveCommit messages sent simulatanously to the chain if a storage provider has been aggregating many sectors in ProveCommitAggregate message, but at the time of publishing the BaseFee is below the aggregation threshold. This stagger feature prevents issues where some of the ProveCommit messages fail with the SysErrorOutOfGas message. You can tweak how many messages will be staggered per epoch by changing MaxSectorProveCommitsSubmittedPerEpoch in the sealing section of the config.toml file.

NB: While these fixes are great for the reliability of aggregation and batching on the Lotus side, it has been uncovered that aggregating ProveCommit messages for sectors containing verified deals are currently more expensive then single messages due to an issue on the actors side. We therefore do not reccomend our users to aggregate ProveCommit messages when doing verified deals until that issue has been resolved. You can follow the discussion on resolving the issue on the actors side here.

Unsealing CLI/API

This feature release adds a dedicated lotus-miner sectors unseal command and API, allowing you to unseal specific sealed sectors easily.

New features

  • feat: VM Execution Lanes (filecoin-project/lotus#10551)
    • Adds VM exections lanes, efficiently dividing the workload between system processes and RPC-requests.
  • Add API and CLI to unseal sector (#10626) (filecoin-project/lotus#10626)
    • Adds lotus-miner sectors unseal cmd, and a API-method to unseal a sector.
  • feat: sealing: Split PCA/PCB batches if gas used exceeds block limit (filecoin-project/lotus#10647)
    • Splits ProveCommitAggregate and PreCommitBatch messages into multiple messages if the message exceeds the block limit.
  • Add feature to stagger sector prove commit submission (#10543) (filecoin-project/lotus#10543)
    • Staggers the amount of ProveCommit messages sent simultanously if a storage provider has been aggregating many message, but at the moment of publishing the BaseFee is below the threshold for aggregation to prevent unwanted SysErrorOutOfGas issues.
  • Set default for MaxSectorProveCommitsSubmittedPerEpoch (filecoin-project/lotus#10728)
    • Sets the default amount of ProveCommits submitted per epoch to 20.
  • feat: worker: Ensure tempdir exists (#10433) (filecoin-project/lotus#10433)
    • Ensures that a temporary directory exists on start of a lotus-worker with a custom TMPDIR set.
  • feat: sync: harden chain sync (#10756) (filecoin-project/lotus#10756)
  • feat: populate the index on snapshot import (filecoin-project/lotus#10556)
  • feat:chain: Message Index (HIGHLY EXPERIMENTAL) (filecoin-project/lotus#10452)
    • MVP of a message index that allows us to accelrate StateSearchMessage and related functionality, and eventually accelerate critical chain calls (follow up).
  • feat: Add small cache to execution traces (filecoin-project/lotus#10517)
  • feat: shed: incoming block-sub chainwatch tool (filecoin-project/lotus#10513)

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
vyzo 70 +1990/-429 135
Alfonso de la Rocha 25 +814/-299 56
Steven Allen 14 +125/-539 28
Shrenuj Bansal 13 +482/-138 52
Aayush 17 +317/-301 90
Łukasz Magiera 13 +564/-26 16
Jennifer Wang 7 +401/-140 10
Fridrik Asmundsson 14 +315/-84 20
Jorropo 2 +139/-137 74
Mikers 6 +114/-43 14
Hector Sanjuan 5 +92/-44 5
Ales Dumikau 1 +117/-0 10
Mike Seiler 4 +51/-51 6
zenground0 6 +33/-25 8
Phi 8 +32/-10 10
Aayush Rajasekaran 1 +1/-32 2
Ian Davis 2 +7/-10 3
Marcel Telka 1 +5/-7 1
ychiao 1 +8/-3 2
jennijuju 1 +4/-4 8
adlrocha 2 +2/-2 2
Jiaying Wang 1 +0/-4 1
ZenGround0 1 +2/-1 2
Zeng Li 1 +1/-1 1

v1.23.0 / 2023-04-21

This is the stable feature release for the upcoming MANDATORY network upgrade at 2023-04-27T13:00:00Z, epoch 2809800. This feature release delivers the nv19 Lighting and nv20 Thunder network upgrade for mainnet, and includes numerous improvements and enhancements for node operators, ETH RPC-providers and storage providers.

☢️ Upgrade Warnings ☢️

Please read carefully through the upgrade warnings section if you are upgrading from a v1.20.X release, or the v1.22.0 release. If you are upgrading from a v1.21.0-rcX these warnings should be familiar to you.

  • Starting from this release, the SplitStore feature is automatically activated on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!
  • This feature release requires a minimum Go version of v1.19.7 or higher to successfully build Lotus. Additionally, Go version v1.20 and higher is now also supported.
  • Storage Providers: The proofs libraries now have CUDA enabled by default, which requires you to install (CUDA)[https://lotus.filecoin.io/tutorials/lotus-miner/cuda/] if you haven't already done so. If you prefer to use OpenCL on your GPUs instead, you can use the FFI_USE_OPENCL=1 flag when building from source. On the other hand, if you want to disable GPUs altogether, you can use the FFI_NO_GPU=1 environment variable when building from source.
  • Storage Providers: The lotus-miner sectors extend command has been refactored to the functionality of lotus-miner sectors renew.
  • Exchanges/Node operators/RPC-providers:: Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the by the FVM. Please make sure to read the Execution trace format change section carefully, as these are interface breaking changes
  • Syncing issues: If you have been struggling with syncing issues in normal operations you can try to adjust the amount of threads used for more concurrent FMV execution through via the LOTUS_FVM_CONCURRENCY enviroment variable. It is set to 4 threads by default. Recommended formula for concurrency == YOUR_RAM/4 , but max during a network upgrade is 24. If you are a Storage Provider and are pushing many messages within a short period of time, exporting LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS=1 will also help with keeping in sync.
  • Catching up from a Snapshot: Users have noticed that catching up sync from a snapshot is taking a lot longer these day. This is largely related to the built-in market actor consuming a lot of computational demand for block validation. A FIP for a short-term mitigation for this is currently in Last Call and will be included network version 19 upgrade if accepted. You can read the FIP here.

Highlights

Execution Trace Format Changes

Execution traces (returned from lotus state exec-trace, lotus state replay, etc.), has changed to account for changes introduced by the FVM. Specifically:

  • The Msg field no longer matches the Filecoin message format as many of the message fields didn't make sense in on-chain sub-calls. Instead, it now has the fields To, From, Value, Method, Params, and ParamsCodec where ParamsCodec is a new field indicating the IPLD codec of the parameters.
    • Importantly, the Msg.CID field has been removed. This field is still present in top-level invocation results, just not inside the execution trace itself.
  • The MsgRct field no longer includes a GasUsed field and now has a ReturnCodec field to indicating the IPLD codec of the return value.
  • The Error and Duration fields have been removed as these are not set by the FVM. The top-level message "invocation result" retains the Error and Duration fields, they've only been removed from the trace itself.
  • Gas Charges no longer include "virtual" gas fields (those starting with v...) or source location information (loc) as neither field is set by the FVM.

A note on "codecs": FVM parameters and return values are IPLD blocks where the "codec" specifies the data encoding. The codec will generally be one of:

  • 0x51, 0x71 - CBOR or DagCBOR. You should generally treat these as equivalent.
  • 0x55 - Raw bytes.
  • 0x00 - Nothing. If the codec is 0x00, the parameter and/or return value should be empty and should be treated as "void" (not specified).
Old ExecutionTrace: 
{
  "Msg": {
    "Version": 0,
    "To": "f01234",
    "From": "f04321",
    "Nonce": 1,
    "Value": "0",
    "GasLimit": 0,
    "GasFeeCap": "1234",
    "GasPremium": "1234",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "CID": {
        "/": "bafyxyz....."
    },
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "GasUsed": 12345,
  },
  "Error": "",
  "Duration": 568191845,
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "vtg": 0,
      "vcg": 0,
      "vsg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "Error": "",
      "Duration": 1235,
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}
New ExecutionTrace: 
{
  "Msg": {
    "To": "f01234",
    "From": "f04321",
    "Value": "0",
    "Method": 42,
    "Params": "<base64-data-or-null>",
    "ParamsCodec": 81
  },
  "MsgRct": {
    "ExitCode": 0,
    "Return": "<base64-data-or-null>",
    "ReturnCodec": 81
  },
  "GasCharges": [
    {
      "Name": "OnMethodInvocation",
      "loc": null,
      "tg": 23856,
      "cg": 23856,
      "tt": 0
    },
    {
      "Name": "wasm_exec",
      "loc": null,
      "tg": 1764,
      "cg": 1764,
      "sg": 0,
      "tt": 0
    },
    {
      "Name": "OnSyscall",
      "loc": null,
      "tg": 14000,
      "cg": 14000,
      "sg": 0,
      "tt": 0
    },
  ],
  "Subcalls": [
    {
      "Msg": { },
      "MsgRct": { },
      "GasCharges": [],
      "Subcalls": [],
    },
  ]
}

SplitStore

This feature release introduces numerous improvements and fixes to tackle SplitStore related issues that has been reported. With this feature release SplitStore is automatically activated by default on new nodes. However, for existing Lotus users, you need to explicitly configure SplitStore by uncommenting the EnableSplitstore option in your config.toml file. To enable SplitStore, set EnableSplitstore=true, and to disable it, set EnableSplitstore=false. It's important to note that your Lotus node will not start unless this configuration is properly set. Set it to false if you are running a full archival node!

SplitStore also has some new configuration settings that you can set in your config.toml file:

  • HotstoreMaxSpaceTarget suggests the max allowed space (in bytes) the hotstore can take.
  • HotstoreMaxSpaceThreshold a moving GC will be triggered when total moving size exceeds this threshold (in bytes).
  • HotstoreMaxSpaceSafetyBuffer a safety buffer to prevent moving GC from an overflowing disk.

The SplitStore also has two new commands:

  • lotus chain prune hot is a much less resource-intensive GC and is best suited for situations where you don't have the spare disk space for a full GC.
  • lotus chain prune hot-moving will run a full moving garbage collection of the hotstore. This commands create a new hotstore before deleting the old one so you need working room in the hotstore directory. The current size of a fully GC'd hotstore is around 295 GiB so you need to make sure you have at least that available.

You can read more about the new SplitStore commands in the documentation.

RPC API improvements

This feature release includes all the RPC API improvements made in the Lotus v1.20.x patch releases. It includes an updated FFI that sets the FVM parallelism to 4 by default.

Node operators with higher memory specs can experiment with setting LOTUS_FVM_CONCURRENCY to higher values, up to 48, to allow for more concurrent FVM execution.

Experimental scheduler assigners

In this release there are four new expirmental scheduler assigners:

  • The experiment-spread-qcount - similar to the spread assigner but also takes into account task counts which are in running/preparing/queued states.
  • The experiment-spread-tasks - similar to the spread assigner, but counts running tasks on a per-task-type basis
  • The experiment-spread-tasks-qcount - similar to the spread assigner, but also takes into account task counts which are in running/preparing/queued states, as well as counting running tasks on a per-task-type basis. Check the results for this assigner on (storage-only lotus-workers here).
  • The experiment-random - In each schedule loop the assinger figures a set of all workers which can handle the task and then picks a random one. Check the results for this assigner on (storage-only lotus-workers here).

Graceful shutdown of lotus-workers We have cleaned up some commands in the lotus-worker to make it less confusing how to gracefully shutting down a lotus-worker while there are incoming sealing tasks in the pipeline. To shut down a lotus-worker gracefully:

  1. lotus-worker tasks disable --all and wait for the worker to finish processing its current tasks.
  2. lotus-worker stop to detach it and do maintenance/upgrades.

CLI speedups

The lotus-miner sector list is now running in parallel - which should speed up the process from anywhere between 2x-10x+. You can tune it additionally with the check-parallelism option in the command. The Lotus-Miner info command also has a large speed improvement, as calls to the lotus legacy market has been removed.

New features

  • feat: splitstore: Pause compaction when out of sync (filecoin-project/lotus/#10641)
    • Pause the SplitStore compaction if the node is out of sync. Resumes the compation when its back in sync.
  • feat: splitstore: limit moving gc threads (#10621) (filecoin-project/lotus/#10621)
    • Makes moving gc less likely to cause node falling out of sync.
  • feat: splitstore: Update config default value (#10605) (filecoin-project/lotus/#10605)
    • Sets Splitstore HotStoreMaxSpaceTarget config to 650GB as default
  • feat: splitstore: Splitstore enabled by default (#10429) (filecoin-project/lotus#10429)
    • Enables SplitStore by default on new Lotus nodes. Existing Lotus users need to explicitly configure
  • feat: splitstore: Configure max space used by hotstore and GC makes best effort to respect (filecoin-project/lotus#10391)
    • Adds three new configs for setting the maximum allowed space the hotstore can take.
  • feat: splitstore: Badger GC of hotstore command (filecoin-project/lotus#10387)
    • Adds a lotus chain prune hot command, to run the garbage collection of the hotstore in a user driven way.
  • feat: sched: Assigner experiments (filecoin-project/lotus#10356)
    • Introduces experimental scheduler assigners that works better for setups that uses storage-only lotus-workers.
  • fix: wdpost: disabled post worker handling (filecoin-project/lotus#10394)
    • Improved scheduling logic for Proof-of-SpaceTime workers.
  • feat: cli: list claims and remove expired claims (filecoin-project/lotus#9875)
    • Adds a command to list claims made by a provider lotus filplus list-claims. And lotus filplus remove-expired-claims to remove expired claims.
  • feat: cli: make sectors list much faster (filecoin-project/lotus#10202)
    • Makes lotus-miner sector list checks run in parallel.
  • feat: cli: Add an EVM command to fetch a contract's bytecode (filecoin-project/lotus#10443)
    • Adds an lotus evm bytecode command to fetch a contract's bytecode.
  • feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (#10416) (filecoin-project/lotus#10416)
    • Reduces replacement message fee logic to help include update message replacements from developers using Ethereum tools like MetaMask.
  • feat: update renew-sectors with FIP-0045 logic (filecoin-project/lotus#10328)
    • Updates the lotus-miner sectors extend with FIP-0045 logic to include the ability to drop claims and set the maximum number of messages contained in a message.
  • feat: IPC: Abstract common consensus functions and consensus interface (filecoin-project/lotus#9481)
    • Add eudico's consensus interface to Lotus and implement EC behind that interface. This abstraction is the stepping-stone for Mir's integration.
  • fix: worker: add all tasks flag (filecoin-project/lotus#10232)
    • Adds an all flag for the lotus-worker tasks enable/disable cmds.
  • feat:shed:add cid to cbor serialization command (filecoin-project/lotus#10032)
    • Adds two lotus-shed commands, lotus-shed cid bytes and lotus-shed cid cbor to serialize cid to cbor and cid to bytes.
  • feat: add toolshed commands to inspect statetree size (filecoin-project/lotus#9982)
    • Adds two commands, lotus-shed stat-actor and lotus-shed stat-obj that work with an offline lotus repo to report dag size stats.
  • feat: shed: encode address to bytes (filecoin-project/lotus#10105)
    • Adds a lotus-shed address encode for encoding a filecoin address to hex bytes.
  • feat: chain: export-range (filecoin-project/lotus#10145)
    • Adds a lotus chain export-range command that can create archival-grade ranged exports of the chain as quickly as possible.
  • feat: stmgr: cache migrated stateroots (filecoin-project/lotus#10282)
    • Cache network migration results to avoid running migrations twice.
  • feat: shed: Add a tool to read data from sectors (filecoin-project/lotus#10169)
    • Adds a lotus-shed sectors read command that extract data from sectors from a running lotus-miner deployment.
  • feat: cli: Refactor renew and remove extend (filecoin-project/lotus#9920)
    • Refactors the lotus-miner sectors extend command to have the functionality of lotus-miner sectors renew. The lotus-miner sectors renew command has been deprecated.
  • feat: shed: Add beneficiary commands (filecoin-project/lotus#10037)
    • Adds the beneficiary address command to lotus-shed. You can now use lotus-shed actor propose-change-beneficiary and lotus-shed actor confirm-change-beneficiary to change beneficiary addresses.

Improvements

Dependencies

Others

Contributors

Contributor Commits Lines ± Files Changed
Hannah Howard 2 +2909/-6026 84
Łukasz Magiera 42 +2967/-1848 95
Steven Allen 20 +1703/-1345 88
Alfonso de la Rocha 17 +823/-1808 86
Peter Rabbitson 9 +1957/-219 34
Geoff Stuart 12 +818/-848 29
hannahhoward 5 +507/-718 36
Hector Sanjuan 6 +443/-726 35
Kevin Li 1 +1124/-14 22
zenground0 30 +791/-269 88
frrist 1 +992/-16 13
Travis Person 4 +837/-53 24
Phi 20 +622/-254 34
Ian Davis 7 +35/-729 20
Aayush 10 +378/-177 40
Raúl Kripalani 15 +207/-138 19
Arsenii Petrovich 7 +248/-94 30
ZenGround0 5 +238/-39 15
Neel Virdy 1 +109/-107 58
ychiao 1 +135/-39 3
Jorropo 2 +87/-82 67
Marten Seemann 8 +69/-64 17
Rod Vagg 1 +55/-16 3
Masih H. Derkani 3 +39/-27 12
raulk 2 +30/-29 5
dependabot[bot] 4 +37/-17 8
beck 2 +38/-2 2
Jennifer Wang 4 +20/-19 19
Richard Guan 3 +28/-8 5
omahs 7 +14/-14 7
dirkmc 2 +19/-7 6
David Choi 2 +16/-5 2
Mike Greenberg 1 +18/-1 1
Adin Schmahmann 1 +19/-0 2
Phi-rjan 5 +12/-4 5
Dirk McCormick 2 +6/-6 3
Aayush Rajasekaran 2 +9/-3 2
Jiaying Wang 5 +6/-4 5
Anjor Kanekar 1 +5/-5 1
vyzo 1 +3/-3 2
0x5459 1 +1/-1 1

v1.22.1 / 2023-04-23

Important Notice

This is a MANDATORY hotfix release that fixes a consensus-critical bug that was in v1.22.0 -- the necessary fix is https://github.com/filecoin-project/ref-fvm/pull/1750 and it is integrated into lotus via https://github.com/filecoin-project/lotus/pull/10735. You can NOT use 1.22.0 for the nv19 upgrade, you MUST be on 1.22.1 or higher.

About This Release

This is the stable release of Lotus v1.22.1 for the upcoming MANDATORY network upgrade at 2023-04-27T13:00:00Z, epoch 2809800. This release delivers the nv19 Lighting and nv20 Thunder network upgrade for mainnet.

Note that you must be on a go version higher than Go 1.18.8, but lower than Go v1.20.0. We would recommend Go 1.19.7.

The Lighting and Thunder upgrade introduces the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

Expedited nv19 Lightning ⚡️ rollout

In light of the recent degraded chain quality on the mainnet an expedited nv19 upgrade has been proposed and accepted to roll out the market cron mitigation (FIP0060) that will improve block validation times, and with that the delay in block production that is causing a decrease in the chain quality currently.

With this expedited roll out we want to inform you of some key changes and important dates:

The Lighting and Thunder upgrade now implements the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

v11 Builtin Actor Bundles

Make sure that your lotus actor bundle matches the v11 actors manifest by running after upgrading:

lotus state actor-cids --network-version 19
Network Version: 19
Actor Version: 11
Manifest CID: bafy2bzacecnhaiwcrpyjvzl4uv4q3jzoif26okl3m66q3cijp3dfwlcxwztwo

Actor             CID  
datacap           bafk2bzacebslykoyrb2hm7aacjngqgd5n2wmeii2goadrs5zaya3pvdf6pdnq
init              bafk2bzaceckwf3w6n2nw6eh77ktmsxqgsvshonvgnyk5q5syyngtetxvasfxg
reward            bafk2bzacebwjw2vxkobs7r2kwjdqqb42h2kucyuk6flbnyzw4odg5s4mogamo
cron              bafk2bzacebpewdvvgt6tk2o2u4rcovdgym67tadiis5usemlbejg7k3kt567o
ethaccount        bafk2bzaceclkmc4yidxc6lgcjpfypbde2eddnevcveo4j5kmh4ek6inqysz2k
evm               bafk2bzacediwh6etwzwmb5pivtclpdplewdjzphouwqpppce6opisjv2fjqfe
storagemarket     bafk2bzaceazu2j2zu4p24tr22btnqzkhzjvyjltlvsagaj6w3syevikeb5d7m
storagepower      bafk2bzaceaxgloxuzg35vu7l7tohdgaq2frsfp4ejmuo7tkoxjp5zqrze6sf4
system            bafk2bzaced7npe5mt5nh72jxr2igi2sofoa7gedt4w6kueeke7i3xxugqpjfm
account           bafk2bzacealnlr7st6lkwoh6wxpf2hnrlex5sknaopgmkr2tuhg7vmbfy45so
placeholder       bafk2bzacedfvut2myeleyq67fljcrw4kkmn5pb5dpyozovj7jpoez5irnc3ro
eam               bafk2bzaceaelwt4yfsfvsu3pa3miwalsvy3cfkcjvmt4sqoeopsppnrmj2mf2
multisig          bafk2bzaceafajceqwg5ybiz7xw6rxammuirkgtuv625gzaehsqfprm4bazjmk
paymentchannel    bafk2bzaceb4e6cnsnviegmqvsmoxzncruvhra54piq7bwiqfqevle6oob2gvo
storageminer      bafk2bzacec24okjqrp7c7rj3hbrs5ez5apvwah2ruka6haesgfngf37mhk6us
verifiedregistry  bafk2bzacedej3dnr62g2je2abmyjg3xqv4otvh6e26du5fcrhvw7zgcaaez3a

Changelog

Dependencies

github.com/filecoin-project/go-state-types (v0.11.0-rc1 -> v0.11.1):

v1.22.0 / 2023-04-21

EDIT: Do NOT use this release for nv19, you MUST use v1.22.1 or higher.

This is the stable release of Lotus v1.22.0 for the upcoming MANDATORY network upgrade at 2023-04-27T13:00:00Z, epoch 2809800. This release delivers the nv19 Lighting and nv20 Thunder network upgrade for mainnet.

Note that you must be on a go version higher then Go 1.18.8, but lower then Go v1.20.0. We would recommend Go 1.19.7.

The Lighting and Thunder upgrade introduces the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

Expedited nv19 Lightning ⚡️ rollout

In light of the recent degraded chain quality on the mainnet an expedited nv19 upgrade has been proposed and accepted to roll out the market cron mitigation (FIP0060) that will improve block validation times, and with that the delay in block production that is causing a decrease in the chain quality currently.

With this expedited roll out we want to inform you of some key changes and important dates:

The Lighting and Thunder upgrade now implements the following Filecoin Improvement Proposals (FIPs), delivered by builtin-actors v11 (see actors v11.0.0):

  • FIP 0060 - Thirty day market deal maintenance interval
  • FIP 0061 - WindowPoSt grindability fix
  • FIP 0062 - Fallback method handler for multisig actor

v11 Builtin Actor Bundles

Make sure that your lotus actor bundle matches the v11 actors manifest by running after upgrading:

lotus state actor-cids --network-version 19
Network Version: 19
Actor Version: 11
Manifest CID: bafy2bzacecnhaiwcrpyjvzl4uv4q3jzoif26okl3m66q3cijp3dfwlcxwztwo

Actor             CID  
datacap           bafk2bzacebslykoyrb2hm7aacjngqgd5n2wmeii2goadrs5zaya3pvdf6pdnq
init              bafk2bzaceckwf3w6n2nw6eh77ktmsxqgsvshonvgnyk5q5syyngtetxvasfxg
reward            bafk2bzacebwjw2vxkobs7r2kwjdqqb42h2kucyuk6flbnyzw4odg5s4mogamo
cron              bafk2bzacebpewdvvgt6tk2o2u4rcovdgym67tadiis5usemlbejg7k3kt567o
ethaccount        bafk2bzaceclkmc4yidxc6lgcjpfypbde2eddnevcveo4j5kmh4ek6inqysz2k
evm               bafk2bzacediwh6etwzwmb5pivtclpdplewdjzphouwqpppce6opisjv2fjqfe
storagemarket     bafk2bzaceazu2j2zu4p24tr22btnqzkhzjvyjltlvsagaj6w3syevikeb5d7m
storagepower      bafk2bzaceaxgloxuzg35vu7l7tohdgaq2frsfp4ejmuo7tkoxjp5zqrze6sf4
system            bafk2bzaced7npe5mt5nh72jxr2igi2sofoa7gedt4w6kueeke7i3xxugqpjfm
account           bafk2bzacealnlr7st6lkwoh6wxpf2hnrlex5sknaopgmkr2tuhg7vmbfy45so
placeholder       bafk2bzacedfvut2myeleyq67fljcrw4kkmn5pb5dpyozovj7jpoez5irnc3ro
eam               bafk2bzaceaelwt4yfsfvsu3pa3miwalsvy3cfkcjvmt4sqoeopsppnrmj2mf2
multisig          bafk2bzaceafajceqwg5ybiz7xw6rxammuirkgtuv625gzaehsqfprm4bazjmk
paymentchannel    bafk2bzaceb4e6cnsnviegmqvsmoxzncruvhra54piq7bwiqfqevle6oob2gvo
storageminer      bafk2bzacec24okjqrp7c7rj3hbrs5ez5apvwah2ruka6haesgfngf37mhk6us
verifiedregistry  bafk2bzacedej3dnr62g2je2abmyjg3xqv4otvh6e26du5fcrhvw7zgcaaez3a

Changelog

Dependencies

github.com/filecoin-project/go-state-types (v0.11.0-rc1 -> v0.11.1):

v1.20.4 / 2023-03-17

This is a patch release intended to alleviate performance issues reported by some users since the nv18 upgrade. The primary change is to update the FFI to allow for FVM parallelism of 4 by default, and make this user-configurable. through the LOTUS_FVM_CONCURRENCY env var.

Users with higher memory specs can experiment with setting LOTUS_FVM_CONCURRENCY to higher values, up to 48, to allow for more concurrent FVM execution.

Bug fixes

  • Splitstore: Don't enforce walking receipt tree during compaction #10505
  • fix: build: drop drand incentinet servers #10506

Improvement

  • chore: update ffi to increase execution parallelism #10503

v1.20.3 / 2023-03-09

A 🐈 stepped on the ⌨️ and made a mistake while resolving conflicts 😨. This releases only includes #10439 to fix that mistake. v1.20.2 is retracted - Please skip v1.20.2 and only update to v1.20.3!!!

Changelog

compare to v1.20.1

This is a HIGHLY RECOMMENDED patch release for node operators/API service providers that run ETH RPC service and an optional release for Storage Providers.

Bug fixes

  • fix: EthAPI: use StateCompute for feeHistory; apply minimum gas premium #10413
  • fix: eth API: return correct txIdx around null blocks #10419
  • fix: Eth API: make block parameter parsing sounder. #10427

Improvement

  • feat: Lotus Gateway: Add missing methods - master #10420
  • feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x #10416
  • We recommend storage providers to update your nodes to this patch, that will help improve developers who use Ethereum tooling's experience.

v1.20.2 / 2023-03-09

DO NOT USE: Use 1.20.3 instead!

This is a HIGHLY RECOMMENDED patch release for node operators/API service providers that run ETH RPC service and an optional release for Storage Providers.

Bug fixes

  • fix: EthAPI: use StateCompute for feeHistory; apply minimum gas premium #10413
  • fix: eth API: return correct txIdx around null blocks #10419
  • fix: Eth API: make block parameter parsing sounder. #10427

Improvement

  • feat: Lotus Gateway: Add missing methods - master #10420
  • feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x #10416
  • We recommend storage providers to update your nodes to this patch, that will help improve developers who use Ethereum tooling's experience.

v1.20.1 / 2023-03-06

This an optional patch releases for node operators/API service providers that run ETH RPC service.

Bug fixes

  • fix: EthAPI: Correctly get parent hash #10389
  • fix: EthAPI: Make newEthBlockFromFilecoinTipSet faster and correct #10380
  • fix: state: short-circuit genesis state computation

1.20.0 / 2023-02-28

This is a MANDATORY release of Lotus that delivers the Hygge network upgrade, introducing Filecoin network version 18. The centerpiece of the upgrade is the introduction of the Filecoin Virtual Machine (FVM)’s Milestone 2.1, which will allow for EVM-compatible contracts to be deployed on the Filecoin network. This upgrade delivers user-programmablity to the Filecoin network for the first time!

The Filecoin mainnet is scheduled to upgrade to nv18 at epoch 2683348, on March 14th at 2023-03-14T15:14:00Z. All node operators, including storage providers, must upgrade to this release before that time. Storage providers must update their daemons, miners, market and worker(s)/boost. At the upgrade, a short migration will run that converts code actors v9 code CIDs to v10 CIDs, and installs the new Ethereum Address Manager singleton (see below). This is expected to be a lightweight migration that causes no service disruption.

The Hygge upgrade introduces the following Filecoin Improvement Proposals (FIPs), delivered in FVM3 (see FVM v3.0.0) and builtin-actors v10 (see actors v10.0.0):

  • FIP-0048: f4 Address Class
  • FIP-0049: Actor events
  • FIP-0050: API between user-programmed actors and built-in actors
  • FIP-0054: Filecoin EVM runtime (FEVM)
  • FIP-0055: Supporting Ethereum Accounts, Addresses, and Transactions
  • FIP-0057: Update gas charging schedule and system limits for FEVM

Filecoin Ethereum Virtual Machine (FEVM)

The Filecoin Ethereum Virtual Machine (FEVM) is built on top of the WASM-based execution environment introduced in the Skyr v16 upgrade. The chief feature introduced is the ability for anyone participating in the Filecoin network to deploy their own EVM-compatible contracts onto the blockchain, and invoke them as appropriate.

New Built-in Actors

The FEVM is principally delivered through the introduction of the new EVM actor. This actor “represents” smart contracts on the Filecoin network, and includes an interpreter that implements all EVM opcodes as their Filecoin equivalents, and translates state I/O operations to be compatible with Filecoin’s IPLD-based data model. For more on the EVM actors, please see FIP-0054.

The creation of EVM actors is managed by the new Ethereum Address Manager actor (EAM), a singleton that is invoked in order to deploy EVM actors. In order to make usage of the FEVM as seamless as possible for users familiar with the Ethereum ecosystem, this upgrades also introduces a dedicated actor to serve as “Ethereum Accounts”. This actor exists to allow for secp keys to be used in the Ethereum addressing scheme. The last new built-in actor introduced is the Placeholder actor, a thin “shell” of an actor that can transform into either EVM or EthAccount actors. For more on the EAM, EthAccount, and Placeholder actors, please see FIP-0055.

v10 Built-in actor bundles

Bundles for all networks (mainnet, calibnet, etc.) are included in the lotus source tree (build/actors/) and embedded on build, for v10 actors you can find it here. Reminder: Lotus verifies that the bundle CIDs are the right ones upon build & upgrade against the values in build/builtin_actors_gen.go, according to the network you are building. You may also check the bundle manifest CID matches the bundle gen-ed values by running lotus state actor-cids --network-version 18.

The manifest CID & full list of actor code CIDs for nv18 using actor v10 is:

"_manifest":        "bafy2bzacecsuyf7mmvrhkx2evng5gnz5canlnz2fdlzu2lvcgptiq2pzuovos"
"account":          "bafk2bzaceampw4romta75hyz5p4cqriypmpbgnkxncgxgqn6zptv5lsp2w2bo"
"cron":             "bafk2bzacedcbtsifegiu432m5tysjzkxkmoczxscb6hqpmrr6img7xzdbbs2g"
"datacap":          "bafk2bzacealj5uk7wixhvk7l5tnredtelralwnceafqq34nb2lbylhtuyo64u"
"eam":             "bafk2bzacedrpm5gbleh4xkyo2jvs7p5g6f34soa6dpv7ashcdgy676snsum6g"
"ethaccount":   "bafk2bzaceaqoc5zakbhjxn3jljc4lxnthllzunhdor7sxhwgmskvc6drqc3fa"
"evm":                "bafk2bzaceahmzdxhqsm7cu2mexusjp6frm7r4kdesvti3etv5evfqboos2j4g"
"init":                "bafk2bzaced2f5rhir3hbpqbz5ght7ohv2kgj42g5ykxrypuo2opxsup3ykwl6"
"multisig":         "bafk2bzaceduf3hayh63jnl4z2knxv7cnrdenoubni22fxersc4octlwpxpmy4"
"paymentchannel":   "bafk2bzaceartlg4mrbwgzcwric6mtvyawpbgx2xclo2vj27nna57nxynf3pgc"
"placeholder": "bafk2bzacedfvut2myeleyq67fljcrw4kkmn5pb5dpyozovj7jpoez5irnc3ro"
"reward":           "bafk2bzacebnhtaejfjtzymyfmbdrfmo7vgj3zsof6zlucbmkhrvcuotw5dxpq"
"storagemarket":    "bafk2bzaceclejwjtpu2dhw3qbx6ow7b4pmhwa7ocrbbiqwp36sq5yeg6jz2bc"
"storageminer":     "bafk2bzaced4h7noksockro7glnssz2jnmo2rpzd7dvnmfs4p24zx3h6gtx47s"
"storagepower":     "bafk2bzacec4ay4crzo73ypmh7o3fjendhbqrxake46bprabw67fvwjz5q6ixq"
"system":           "bafk2bzacedakk5nofebyup4m7nvx6djksfwhnxzrfuq4oyemhpl4lllaikr64"
"verifiedregistry": "bafk2bzacedfel6edzqpe5oujno7fog4i526go4dtcs6vwrdtbpy2xq6htvcg6"

Node Operators

FVM has been running in lotus since v1.16.0 and up, and the new FEVM does not increase any node hardware spec requirement.

With FEVM on Filecoin, we aim to provide full compatibility with the existing EVM ecosystem and its tooling out of the box. Consequently, lotus now provides a full set of Ethereum-styled APIs for developers and token holders to interact with the Filecoin network as well. For full documentation on this new tooling, please see the Lotus docs website.

Enabling Ethereum JSON RPC API

Note that Ethereum APIs are only supported in the lotus v1 API, meaning that any node operator who wants to enable Eth API services must be using the v1 API, instead of the v0 API. To enable Eth RPC, simply set EnableEthRPC to true in your node config.toml file; or set env var LOTUS_FEVM_ENABLEETHRPC to 1 before starting your lotus node.

Eth tx hash and Filecoin message CID

Most of the Eth APIs take Eth accounts and tx has as an input, and they start with 0x , and that is what Ethereum tooling support. However, in Filecoin, we have Filecoin account formats where things start with f (f410 specifically for eth accounts on Filecoin) and the messages are in the format of CIDs. To enable a smooth developer experience, Lotus internally converts between Ethereum address and Filecoin account address as needed. In addition, lotus also keeps a Eth tx hash <> Filecoin message CID map and stores them in a SQLite database as node sees a FEVM messages. The database is initiated and the maps are populated automatically in ~/<lotus_repo>/sqlite/txhash.db for any node that as Eth RPC enabled. Node operators can configure how many historical mappings they wanna store by configuring EthTxHashMappingLifetimeDays .

Events

FIP-0049 introduces actor events that can be emitted and externally observable during message execution. An events.db is created automatically under ~/<lotus_repo>/sqlite to store these events if the node has Eth RPC enabled. Node operators can configure the events support base on their needs by configuration Events configurations.

Note: All three features are new, and we welcome user feedback, please create an issue if you have any enhancements that you’d like to see!

1.19.0 / 2022-12-07

This is an optional feature release of Lotus. This feature release includes the SplitStore beta, the experimental Lotus node cluster feature, as well as numerous enhancments and bugfixes.

Highlights

🟢 SplitStore v2（Beta) 🟢

Splitstore aims to reduce the node performance impact that's caused by the Filecoin's very large, and continuously growing chain datastore by having a hot and cold blockstore. You can find more about the Splitstore implementation here. Splitstore has three basic modes for node operators to configure according to your needs:

  • discard: hotstore only, automatically archive out-of-scope objects that are beyond 4 finalities(3600 epochs).
  • universal: stores all chain data that's beyond 4 finalities into coldstore.
  • messages: only stores on-chain messages into coldstore.

The EnableColdStoreAutoPrune= configuration is being deprecated in this release, as there is only ever one compaction running. We welcome all node operators to try the new feature and let us know here if you have any feedback! There are more configuration one may set, you can read the full documentation about the SplitStoreV2 here: https://lotus.filecoin.io/lotus/configure/splitstore/.

🧪 Node Cluster (EXPERIMENTAL.) 🧪

The Lotus HA node cluster feature allows you to run multiple Lotus daemons for the same lotus-miner increasing resiliency. We welcome all Lotus users to join the early testing for this feature and provide your feedback. Please note that this feature is targeted towards more enterprise users of Lotus and requires at least 3 lotus nodes to be set up in a cluster. Check out the documentation here: https://lotus.filecoin.io/lotus/configure/clusters/

⭐️ SnapDeals Enhancements ⭐️

Numerous SnapDeals related improvements and fixes made it into this release before the code freeze. Some the highlights of the issues that has been fixed in this feature release are:

  • Unable to snap-up a sector again if something went wrong. - This has now been fixed ✅
  • Error messages on loop during an open deadline. - This has now been fixed ✅

New features

  • feat:splitstore:single compaction that can handle prune aka two marksets one compaction (#9571) (filecoin-project/lotus#9571)
    • Introduces a new SplitStore-mode, messages, which will only store on-chain messages. Fixes previously issues with regards to AutoPrune not compacting the coldstore. Link to documentation
  • feat: Raft consensus for lotus nodes in a cluster (filecoin-project/lotus#9294)
    • Adds the experimental node cluster feature.
  • feat: storage: Force exit GenerateSingleVanillaProof on cancelled context (filecoin-project/lotus#9613)
    • GenerateSingleVanillaProof now respects context, which means that it will skip slow to read sectors 🐌 and return a context error. Instead of being blocked forever if storage reads where blocked (e.g disconnected NFS).
  • feat: wdpost: Configurable pre-check timeouts (filecoin-project/lotus#9680)
    • Adds configuration knobs for setting custom amount of time a proving pre-check can take before a sector and partition will be skipped. Link to documentation
  • feat: chain: future proof the from & to address protocols (filecoin-project/lotus#9515)
    • This lets us add new address protocols to go-address without implicitly accepting them in messages on the network.
  • feat: Retrieval into remote blockstores (filecoin-project/lotus#9565)
    • Makes it possible to point retrievals at a network-backed blockstore.
  • feat: Add node uptime rpc / output in info command (filecoin-project/lotus#9436)
    • Adds node uptime stats to the lotus-miner info and lotus info commands
  • feat: wdpost: Add ability to only have single partition per msg for partitions with… (filecoin-project/lotus#9413)
    • Adds a configuration option to have a single partition per PoSt Message for partitions containing recovering sectors.
  • feat: miner paramfetch: Don't fetch param files when not needed (filecoin-project/lotus#9391)
    • A Lotus-Miner processes that has disabled local PoSt / C2 / PR2 does not need the param-files. This makes node startup much faster, reducing downtime by a lot when restarts are needed.
  • feat: client: Add retrieval deal ID and bytes transferred to retrieval output (filecoin-project/lotus#9398)
    • Appends retrieval deal ID and bytes transferred to the retrieval output.
  • feat: dealpublisher: check for duplicate deals before adding (filecoin-project/lotus#9365)
  • feat: Drop active retrieval check (#764) (filecoin-project/go-fil-markets#764)
  • feat(retrievalmarkets): expose GetDynamicAsk (#748) (filecoin-project/go-fil-markets#748)
  • feat: handle retrieval queries for unindexed identity payload CIDs (#747) (filecoin-project/go-fil-markets#747)
  • feat: add a method for validating an address for a network version (#115) (filecoin-project/go-state-types#115)

Improvements

  • fix: miner-cli: Fix lotus-miner proving check (filecoin-project/lotus#9643)
    • Fixes the issue where the lotus-miner proving check command always outputted Error: rg is nil
  • fix: sealing pipeline: Clear CreationTime when starting sector upgrade (filecoin-project/lotus#9642)
    • Fixes the issue where an aborted SnapDeal upgrade could no longer be retried with SnapDeals.
  • fix:sealing-fsm:wait mutable fsm state for immutable sector upgrade error (filecoin-project/lotus#9598)
    • Creating a new WaitMutable state - now if the deadline is open and the sectors are trying finalize they will wait on the worker until the deadline has closed. Important to note that they will not finalize as soon as the deadline closes, they will wait 1h before continuing. Fixes the previous issue where upgraded Snap-sectors for an open deadline cause a lot of error-messages and p_aux issues
  • fix: cli: add beneficiary info to lotus-miner actor control list (filecoin-project/lotus#9632)
    • Adds the beneficiary address to the lotus-miner actor control list output.
  • fix: sealing pipeine: Release assigned deals on snapdeals abort (filecoin-project/lotus#9601)
  • fix: docker: make compatible with arm platform (filecoin-project/lotus#9363)
    • Makes the Dockerfile.lotus compatible with ARM-platforms (e.g Mac M1).
  • fix: post worker sched: Don't check worker session in a busy loop (filecoin-project/lotus#9495)
    • Fixes a looping pattern which could result in a flood of requests between lotus-miner<->lotus-worker, potentially exhausting resources needed to make http requests, that lead to all sorts of random RPC-related issues.
  • fix: miner: init miner's with 32GiB sectors by default (filecoin-project/lotus#9364)
    • Makes the lotus-miner init defualt to 32GiB sectors.
  • fix: store identity CIDs in CARs for online deals (#749) (filecoin-project/go-fil-markets#749)
  • fix: cliutil: Fix URL-based API endpoint parsing

Dependencies

Others

lotus-market EOL notice

As mentioned in lotus v1.17.0 release notes, markets related features, enhancements and fixes is now lower priority for Lotus. We recommend our users to migrate to other deal making focused software, like boost as soon as possible. That being said, the lotus maintainers will be:

  • Lotus maintainers will stop supporting lotus-market subcomponent/storage deal making related issues or enhancements on Jan 31, 2023.
  • In Q2 2023, we will be deprecating/removing lotus-market related code from this repository.

If you have any questions or concerns, please raise them in Lotus discussion!

Contributors

Contributor Commits Lines ± Files Changed
Geoff Stuart 69 +4745/-19478 405
Shrenuj Bansal 39 +5257/-2183 243
Łukasz Magiera 32 +2763/-730 169
Aayush 47 +1439/-1138 157
Ian Davis 21 +556/-1065 41
Rod Vagg 5 +657/-320 18
jennijuju 4 +632/-317 6
Aayush Rajasekaran 13 +700/-135 18
Jennifer Wang 14 +740/-54 25
ZenGround0 1 +193/-195 14
Hannah Howard 4 +138/-122 52
Steven Allen 4 +105/-24 11
zenground0 9 +109/-16 14
Peter Rabbitson 1 +27/-23 3
hannahhoward 2 +49/-0 2
Airenas Vaičiūnas 2 +31/-16 2
simlecode 6 +19/-10 12
Phi 5 +16/-10 7
sectrgt 2 +18/-0 2
Jiaying Wang 2 +4/-4 3
Rob Quist 1 +3/-1 1
Jakub Sztandera 1 +1/-1 1

1.18.2 / 2022-12-10

This is an OPTIONAL patch release that fixes a recently reported bug, where the miner process crashes due to a panic during an AddPiece process. More details can be found here.

1.18.1 / 2022-11-28

This is a small OPTIONAL patch release for the mandatory v1.18.0 release that supports the Filecoin nv17 Shark Upgrade. We highly recommend you to read the full v1.18.0 release note if you haven't already.

Note to SPs: If you are running into issue with updating your miner node from an earlier release and is failing to restart your miner, check your journalctl and see if you noticed the following error:

New sector storage: <%d>
Nov 19 15:03:43 g0lotus01 lotus-miner[<id>]: ERROR: creating node: starting node: could not build arguments for function "reflect".makeFuncStub (/usr/local/go/src/reflect/asm_amd64.s:28): failed to build *paths.Local: received non-nil error from function "reflect".makeFuncStub (/usr/local/go/src/reflect/asm_amd64.s:28): opening path /media/data1/lotusstorage: path with ID <%d> already opened: '<path>'

If so, this check is introduced via #9032, precisely this line. It's added to prevents double-attaching paths given it's now possible to manipulate paths at runtime. Verify storage.json configs if you encounter this error or remove the undesired depulicated path as you see fit accordingly.

Bug Fixes

  • fix: cli: check found before dereferencing SectorInfo #9703

1.18.0 / 2022-11-15

⚠️ Please note that from Lotus v1.17.2&^ will require a Go-version of v1.18.1&^

This is the final release of the upcoming MANDATORY release of Lotus that introduces Filecoin network v17, codenamed the Shark upgrade. Shark upgrade delivers a wave of protocol refinements that will allow for useful smart contracts to be written using the FVM (eg. programmable markets, lending contracts, etc.).

The Filecoin mainnet is scheduled to upgrade to nv17 at epoch 2383680, on Nov 30th on 2022-11-30T14:00:00Z. All node operators, including storage providers, must upgrade to this release before that time. Storage providers must update their daemons, miners, market and worker(s)/boost.

The Shark upgrade introduces the following FIPs, delivered in actors v9:

  • FIP0029 Beneficiary Address for Storage Providers: step towards better lending market for SP
  • FIP0034 Fix PreCommit Deposit Independent of Sector Content: resolves a significant weakening of Filecoin PoRep’s security guarantees
    • ❗Pre-commit deposit will be calculated as the 20-day projection of expected reward earned by a sector with a sector quality of 10 (i.e. full of verified deals), regardless of sector content. The Initial Pledge value, due when the sector is proven, is left unchanged.
  • FIP0041 Forward Compatibility for PreCommit: enables a cleaner and easier transition to Programmable Storage Markets
  • FIP0044 Standard Message Authentication: enable metadata authentication for user defined actor
  • FIP0045 Decoupling Fil+ from Marketplace: DataCap and the 10x QAP is now only associated with how long DATA is wanted to be stored on the network.
    • This is a transitional state to enabling far more efficient and dynamic storage markets on Filecoin network in the future.
    • ⭐️ First Fungible Token Contract - Datacap Actor, on Filecoin! (fungible token standard, token contract library).
    • For storage deal participants (clients and storage providers):
      • PublishStorageDeals/ProveCommit(Aggregate)/ProveReplicaUpdates message that includes verified deals will see a gas usage increase, more details can be found here
      • Term is introduced for defining how long the DataCap is assigned to a piece of data. Anyone who cares about that piece of data may extend the term, which incentives SPs to store the data longer on the network without a new deal/resealing.
      • There is no more diluted verified deal QAP due to deal/sector space time for new sectors that contains verified deals after this upgrade.
      • SPs may enjoy 90 days of extra QAP than deal duration by default, given term_max is always deal duration + 90 days. ❗ We highly recommend all lotus users, especially storage providers, developers and clients to read the FIPs in detail to understand the protocol changes and potential impact to network participants!

Snapshots

The #fil-infra team at PL has launched a brand new Lightweight Filecoin Chain Snapshots Service to support chain management needs for the node operators, check here for the full detail. We are planning to switch the snapshot service listed in lotus docs to the new Lightweight Filecoin Chain Snapshots Service by EOY, and deprecate public support of the current snapshots production. We recommend all users to test and switch the new service ASAP, and if you run into any issue, please report them here and the team would be happy to support you! For the main differences between the old & the new service, checkout the FAQ section here

Migration

We are expecting a heavier than normal state migration for this upgrade due to the amount of the state changes introduced. All node operators, including storage providers, should be aware that two pre-migrations are being scheduled. The first pre-migration will begin at 2022-11-30T12:00:00Z (120 minutes before the real upgrade), the second pre-migration will begin at 2022-11-30T13:45:00Z (7.5 minutes before the real upgrade). The first pre-migration will take up to 1.5hr, depending on the amount of the historical state in the node blockstore and the hardware specs the node is running on. During this time, expect slower block validation times, increased CPU and memory usage, and longer delays for API queries. We recommend node operators (who haven't enbabled splistore universal mode) that do not care about historical chain states, to prune the chain blockstore by syncing from a snapshot 1-2 days before the upgrade. Note to full archival node operators: you may expect a migration that takes up to 20 min upon the upgrade, during this period your node will fall out of sync and your chain service may have some disruption. However, you can expect the node to catch up soon after the migration completes.

v9 Built-in actor bundles

Bundles for all networks(mainnet, calibnet, and etc) are included in the lotus source tree (build/actors/) and embedded on build, for v9 actors you can find it here. Reminder: Lotus verifies that the bundle CIDs are the right ones upon build & upgrade against the values in build/builtin_actors_gen.go, according to the network you are building. You may also check the bundle manifest CID matches the bundle gen-ed values by running lotus state actor-cids --network-version 17.

The manifest CID & full list of actor code CIDs for nv17 using actor v9 is:

"_manifest":        "bafy2bzaceb6j6666h36xnhksu3ww4kxb6e25niayfgkdnifaqi6m6ooc66i6i"
"account":          "bafk2bzacect2p7urje3pylrrrjy3tngn6yaih4gtzauuatf2jllk3ksgfiw2y"
"cron":             "bafk2bzacebcec3lffmos3nawm5cvwehssxeqwxixoyyfvejy7viszzsxzyu26"
"datacap":          "bafk2bzacebb6uy2ys7tapekmtj7apnjg7oyj4ia5t7tlkvbmwtxwv74lb2pug"
"init":             "bafk2bzacebtdq4zyuxk2fzbdkva6kc4mx75mkbfmldplfntayhbl5wkqou33i"
"multisig":         "bafk2bzacec4va3nmugyqjqrs3lqyr2ij67jhjia5frvx7omnh2isha6abxzya"
"paymentchannel":   "bafk2bzacebhdvjbjcgupklddfavzef4e4gnkt3xk3rbmgfmk7xhecszhfxeds"
"reward":           "bafk2bzacebezgbbmcm2gbcqwisus5fjvpj7hhmu5ubd37phuku3hmkfulxm2o"
"storagemarket":    "bafk2bzacec3j7p6gklk64stax5px3xxd7hdtejaepnd4nw7s2adihde6emkcu"
"storageminer":     "bafk2bzacedyux5hlrildwutvvjdcsvjtwsoc5xnqdjl73ouiukgklekeuyfl4"
"storagepower":     "bafk2bzacedsetphfajgne4qy3vdrpyd6ekcmtfs2zkjut4r34cvnuoqemdrtw"
"system":           "bafk2bzaceagvlo2jtahj7dloshrmwfulrd6e2izqev32qm46eumf754weec6c"
"verifiedregistry": "bafk2bzacecf3yodlyudzukumehbuabgqljyhjt5ifiv4vetcfohnvsxzynwga"

New Features

Improvements

Dependencies

Others

lotus-market EOL notice

As mentioned in lotus v1.17.0 release notes, markets related features, enhancements and fixes is now lower priority for Lotus. We recommend our users to migrate to other deal making focused software, like boost as soon as possible. That being said, the lotus maintainers will be:

  • Lotus maintainers will stop supporting lotus-market subcomponent/storage deal making related issues or enhancements on Jan 31, 2023.
  • In Q2 2023, we will be deprecating/removing lotus-market related code from this repository. If you have any questions or concerns, please raise them in Lotus discussion!

Contributors

Contributor Commits Lines ± Files Changed
@geoff-vball 73 +14533/-19712 509
@arajasek 16 +2230/-303 49
@arajasek 29 +701/-297 117
@magik6k 5 +429/-135 45
@Frrist 1 +246/-203 25
@stebalien 2 +323/-2 6
@shrenujbansal 3 +176/-61 10
@ZenGround0 2 +78/-38 5
@jennijuju 8 +97/-18 16
@simlecode 5 +18/-9 11
Kevin Li 1 +7/-0 1
@zenground0 2 +3/-3 3
@jennijuju 1 +3/-3 2
Rod Vagg 1 +3/-2 2
@jennijuju 1 +2/-2 2
Peter Rabbitson 1 +3/-0 1
Jakub Sztandera 1 +1/-1 1

v1.17.2 / 2022-10-05

This is an OPTIONAL release of Lotus. This feature release introduces new sector number management APIs in Lotus that enables all the Sealing-as-a-Service and Lotus interactions needed to function. The default propagation delay setting for storage providers has also been changed, as well as numerous other features and enhancements. Check out the sub-bullet points in the feature and enhancement section to get a short description about each feature and enhancements.

Highlights

🦭 SaaS 🦭 New sector management APIs makes it possible to import partially sealed sectors into Lotus. This release implements all SaaS<->Lotus interactions needed for such services to work. Deep dive into the new APIs here: https://github.com/filecoin-project/lotus/discussions/9079#discussioncomment-3652044

⏳ Propagation delay ⌛️ In v1.17.2 the default PropagationDelay has been raised from 6 seconds -> 10 seconds, and you can tune this yourself with the PROPAGATION_DELAY_SECS environment variable. This means you will now wait for 10 seconds for other blocks to arrive from the network before computing a winningPoSt (if eligible). In your lotus-miner logs that means you will see this "baseDeltaSeconds": 10 as default.

⚠️ Please note that Lotus v1.17.2 will require a Go-version of v1.18.1 or higher!

New features

  • feat: sealing: Partially sealed sector import (filecoin-project/lotus#9210)
    • Implements support for importing (partially) sealed sectors which is needed for enabling external sealing services.
  • feat: sealing: Use bitfields to manage sector numbers (filecoin-project/lotus#9183)
    • Needed for Sealing-as-a-Service. Move sector number assigning logic to use stored bitfields instead of stored counters. Makes it possible to reserve ranges of sector numbers, see the sector assigner state and view sector number reservations.
  • feat: env: propagation delay (filecoin-project/lotus#9290)
    • The default propagation delay is raised to 10 seconds from 6 seconds. Ability to set it yourself with the PROPAGATION_DELAY_SECS environment variable.
  • feat: cli: lotus info cmd (filecoin-project/lotus#9233)
    • A new lotus info command that prints useful node information in one place.
  • feat: proving: Introduce manual sector fault recovery (#9144) (filecoin-project/lotus#9144)
    • Allow users to declare fault recovery messages manually with the lotus-miner proving recover-faults command, rather than waiting for it to happen automatically before windowPost.
  • feat: api: Reintroduce StateActorManifestCID (filecoin-project/lotus#9201)
    • Adds ability to retrieve the Actor Manifest CID through the api.
  • feat: message: Add uuid to mpool message sent to chain node from miner (filecoin-project/lotus#9174)
  • feat: message: Add retries to mpool push message from lotus-miner (filecoin-project/lotus#9177)
    • Retries to mpool push message API in case of unavailability of the lotus chain node.

Network 17 related features :

Enhancements

  • feat: sectors renew --only-cc (filecoin-project/lotus#9184)
    • Exlude extending deal-related sectors with the --only-cc option when using the lotus-miner sectors renew
  • feat: miner: display updated & update-cache for storage list (filecoin-project/lotus#9323)
    • Show amount of updated & update-cache sectors in each storage path in the lotus-miner storage list output
  • feat: add descriptive errors to markets event handler (filecoin-project/lotus#9326)
    • More descriptive market error logs
  • feat: cli: Add option to terminate sectors from worker address (filecoin-project/lotus#9291)
    • Adds a flag to allow either owner address or worker address to send terminate sectors message.
  • fix: cli: actor-cids cli command now defaults to current network (filecoin-project/lotus#9321)
    • Makes the command defaults to the current network.
  • fix: ux: Output bytes in lotus client commP cmd (filecoin-project/lotus#9189)
    • Adds an additional line that outputs bytes in the lotus client commP command.
  • fix: sealing: Add information on what worker a job was assigned to in logs (filecoin-project/lotus#9151)
    • Adds the worker hostname into the assignment logs.
  • refactor: sealing pipeline: Remove useless storage adapter code (filecoin-project/lotus#9142
    • Remove proxy code in storage/miner.go / storage/miner_sealing.go, call the pipeline directly instead.

Bug fixes

Dependency updates

Others

Contributors

Contributor Commits Lines ± Files Changed
Aayush Rajasekaran 8 +23010/-2122 109
Aayush 15 +6168/-2679 360
Łukasz Magiera 69 +6462/-2137 606
Geoff Stuart 19 +3080/-1177 342
Marco Munizaga 16 +543/-424 41
Shrenuj Bansal 30 +485/-419 88
LexLuthr 3 +498/-12 19
Phi 15 +330/-70 17
Jennifer Wang 7 +132/-12 11
TippyFlitsUK 1 +43/-45 12
Steven Allen 1 +18/-28 2
Frrist 1 +19/-11 2
Eng Zer Jun 1 +14/-11 6
Dirk McCormick 2 +23/-1 3
Ian Davis 3 +7/-9 3
Masih H. Derkani 1 +11/-0 1
Anton Evangelatov 1 +11/-0 1
Yu 2 +4/-4 5
Hannah Howard 1 +4/-4 1
Phi-rjan 1 +1/-2 1
Jiaying Wang 1 +3/-0 1
nujz 1 +1/-1 1
Rob Quist 1 +1/-1 1

v1.17.1 / 2022-09-06

This is an optional release of Lotus. This release introduces the Splitstore v2 - beta(beta). Splitstore aims to reduce the node performance impact that's caused by the Filecoin's very large, and continuously growing datastore. Splitstore v2 introduces the coldstore auto prune/GC feature & some improvements for the hotstore. We welcome all lotus users to join the early testers and try the new Splitstore out, you can leave any feedback or report issues in this discussion or create an issue. As always, multiple small bug fixes, new features & improvements are also included in this release.

New features

Improvements

Bug Fixes

Dependency Updates

  • github.com/multiformats/go-multibase (v0.0.3 -> v0.1.1)

Others

Contributors

Contributor Commits Lines ± Files Changed
Łukasz Magiera 34 +2329/-317 163
ZenGround0 2 +1527/-89 38
Ian Davis 14 +751/-232 30
LexLuthr 17 +480/-225 63
TheMenko 4 +323/-61 5
Aayush 10 +285/-92 30
beck 3 +143/-93 3
Steven Allen 4 +95/-75 9
zenground0 5 +44/-116 9
Shrenuj Bansal 7 +136/-7 16
Patrick Deuse 3 +76/-57 3
Jennifer Wang 3 +6/-52 11
zl 2 +20/-16 2
Aayush Rajasekaran 2 +6/-6 2
Clint Armstrong 1 +7/-3 1
Cory Schwartz 2 +9/-0 2
Jorropo 1 +3/-2 2
Geoff Stuart 1 +5/-0 1
Frank Y 1 +2/-2 2
Aloxaf 1 +2/-2 1

Lotus changelog

v1.17.0 / 2022-08-02

This is an optional release of Lotus. This feature release introduces a lot of new sealing and scheduler improvements, and many other functionalities and bug fixes.

PSA: Markets related features, enhancements and fixes is now lower priority for Lotus, and is going to be in the hands of Boost, built by the amazing Bedrock team. You can find Lotus mission scope here

New features

Improvements

Bug Fixes

Dependency Updates

  • github.com/filecoin-project/go-address (v0.0.6 -> v1.0.0)
  • github.com/filecoin-project/go-fil-markets (v1.20.1 -> v1.23.1)
  • github.com/filecoin-project/go-indexer-core (v0.2.8 -> v0.2.9)
  • github.com/filecoin-project/go-data-transfer (v1.15.1 -> v1.15.2)
  • github.com/filecoin-project/go-legs (v0.3.7 -> v0.4.4)
  • github.com/filecoin-project/go-state-types (v0.1.8 -> v0.1.10)
  • github.com/filecoin-project/index-provider (v0.5.0 -> v0.8.1)
  • github.com/filecoin-project/specs-actors (v0.9.14 -> v0.9.15)
  • github.com/filecoin-project/specs-actors/v3 (v3.1.1 -> v3.1.2)
  • github.com/filecoin-project/specs-actors/v4 (v4.0.1 -> v4.0.2)
  • github.com/filecoin-project/specs-actors/v5 (v5.0.4 -> v5.0.6)
  • github.com/filecoin-project/specs-actors/v6 (v6.0.1 -> v6.0.2)
  • github.com/filecoin-project/specs-actors/v7 (v7.0.0 -> v7.0.1)
  • github.com/filecoin-project/specs-actors/v8 (null -> v8.0.1)
  • github.com/filecoin-project/specs-storage (v0.2.4 -> v0.4.1)
  • github.com/filecoin-project/storetheindex (v0.3.5 -> v0.4.17)
  • deps: libp2p: update to the latest golibp2p tag (filecoin-project/lotus#8704)
  • chore: update and fix libp2p (filecoin-project/lotus#8996)

Others

Contributors

Contributor Commits Lines ± Files Changed
Masih H. Derkani 153 +15515/-16832 660
Łukasz Magiera 92 +10429/-8024 1580
Andrew Gillis 43 +4149/-1765 208
Jennifer Wang 10 +1441/-1138 34
Geoff Stuart 18 +1348/-859 113
dirkmc 11 +1827/-210 70
Aayush 21 +1134/-894 69
Steven Allen 9 +743/-889 66
Marco Munizaga 15 +990/-252 36
gammazero 47 +681/-411 104
Will 4 +514/-246 29
web3-bot 15 +409/-348 20
Steven Fraser 1 +671/-0 36
Cory Schwartz 27 +520/-89 36
Hannah Howard 3 +318/-105 8
Piotr Galar 2 +337/-59 7
swift-mx 14 +264/-131 17
vyzo 7 +357/-15 16
Petar Maymounkov 6 +221/-23 14
LexLuthr 7 +182/-21 14
Aayush Rajasekaran 5 +97/-70 33
Raúl Kripalani 5 +87/-45 7
unknown 1 +114/-0 8
sti-bot 44 +54/-60 44
Aarsh Shah 2 +61/-50 8
Lucas Molas 1 +74/-27 3
zenground0 8 +80/-18 14
Dirk McCormick 3 +52/-33 8
frank 3 +73/-7 3
Will Scott 4 +45/-11 5
kaola526 5 +44/-11 5
dependabot[bot] 3 +16/-10 8
zl 1 +15/-4 4
Phi 5 +12/-6 6
Marcin Rataj 1 +11/-7 1
github-actions[bot] 7 +8/-8 7
Anton Evangelatov 2 +13/-0 4
Nicolas Gimenez 1 +12/-0 1
Marten Seemann 2 +5/-7 5
Chris Harden 1 +10/-0 2
jennijuju 1 +4/-4 7
Travis Person 2 +2/-6 2
Rod Vagg 1 +3/-3 2
Rob Quist 1 +3/-3 1
Jiaying Wang 1 +2/-3 2
zengroundumbass 1 +3/-1 1
lifei 1 +1/-1 1
Mike 1 +2/-0 1
Hubert 1 +1/-1 1
Daniel N 1 +1/-1 1
BMZ 1 +1/-1 1

1.16.1 / 2022-07-07

This is an OPTIONAL PATCH releases for storage providers who have failed to publish SubmitWindowedPoSt due to out of gas error. The error log looks like /wdpost_run.go:xxx estimating gas {"error": "estimating gas used: message execution failed: exit SysErrOutOfGas(7)....

New Features

  • feat: declare fault recovery: Config for maximum partition count per message (#8988 / #8986)
    • configure MaxPartitionsPerRecoveryMessage in miner configuration setting.
  • feat: wdpost: Config for maximum partition count per message (#8982 / #8986)
    • configure MaxPartitionsPerPoStMessage in miner configuration setting.

1.16.0 / 2022-06-24

This is a MANDATORY release of Lotus that introduces Filecoin network v16, codenamed the Skyr upgrade.

The network is scheduled to upgrade to nv16 at epoch 1960320, on July 6th at 2022-07-06T14:00:00Z. All node operators, including storage providers, must upgrade to this release (or a later release) before that time. Storage providers must update their daemons, miners, market and worker(s).

Your lotus node will switch from the Legacy VM (that depended on go-based specs-actors) to Filecoin Virtual Machine FVM (that uses Rust-based builtin-actors) atomically upon the upgrade.

The Skyr upgrade introduces the following FIPs, enhancements and bug fixes, delivered in built-actors v8 and ref-fvm v1.0.0:,

🆕 Things you may wanna know

Actor Code CIDs

As stated in FIP-0031- structure of the code cid, system actors' code CIDs will be real content-addressing

For lotus users, we are making the change minimal for you. This means the CODE output when you run lotus state get-actor will now be the actual CID that represents the executable code for the actor, followed by wrapped synthetic id like the ones you've got before, i.e fil/8/system.

Moreover, this also means that in the future, whenever the actor code changes, the CID changes accordingly, and a network upgrade is needed for the network participants to have consensus over what executable code we should use for each system actor.

Built-in actor bundles

As the network introduces FVM, it's also switching from spec-actor (written in GoLang) to built-in actor (written in rust), in which the latter comes with importable bundles. This means, like filecoin proof parameters, node operators now also need to fetch the actor bundles according to the network versions for the nodes to remain operational.

Bundles for all networks(mainnet, calibnet, and etc) are included in the lotus source tree (build/actors/) and embedded on build. Lotus verifies that the bundle CIDs are the right ones upon build & upgrade against the values in build/builtin_actors_gen.go, according to the network you are building. You may also check the bundle manifest CID matches the bundle gen-ed values by running lotus state actor-cids --network-version 16.

The manifest CID & full list of actor code CIDs for nv16 using v8.0.0 is:

    "_manifest":         "bafy2bzacebogjbpiemi7npzxchgcjjki3tfxon4ims55obfyfleqntteljsea"
	"account":          "bafk2bzacedudbf7fc5va57t3tmo63snmt3en4iaidv4vo3qlyacbxaa6hlx6y"
	"cron":             "bafk2bzacecqb3eolfurehny6yp7tgmapib4ocazo5ilkopjce2c7wc2bcec62"
	"init":             "bafk2bzaceaipvjhoxmtofsnv3aj6gj5ida4afdrxa4ewku2hfipdlxpaektlw"
	"multisig":         "bafk2bzacebhldfjuy4o5v7amrhp5p2gzv2qo5275jut4adnbyp56fxkwy5fag"
	"paymentchannel":   "bafk2bzacebalad3f72wyk7qyilvfjijcwubdspytnyzlrhvn73254gqis44rq"
	"reward":           "bafk2bzacecwzzxlgjiavnc3545cqqil3cmq4hgpvfp2crguxy2pl5ybusfsbe"
	"storagemarket":    "bafk2bzacediohrxkp2fbsl4yj4jlupjdkgsiwqb4zuezvinhdo2j5hrxco62q"
	"storageminer":     "bafk2bzacecgnynvd3tene3bvqoknuspit56canij5bpra6wl4mrq2mxxwriyu"
	"storagepower":     "bafk2bzacebjvqva6ppvysn5xpmiqcdfelwbbcxmghx5ww6hr37cgred6dyrpm"
	"system":           "bafk2bzacedwq5uppsw7vp55zpj7jdieizirmldceehu6wvombw3ixq2tcq57w"
	"verifiedregistry": "bafk2bzaceb3zbkjz3auizmoln2unmxep7dyfcmsre64vnqfhdyh7rkqfoxlw4"

All bundles are also available at https://github.com/filecoin-project/builtin-actors/releases, thus you can also manually download the bundles and place them in the right path.

Note: use customized bundle will risk you to lose sync with the network!

To get Code CIDs:

  • api:StateActorCodeCIDs
  • cli: lotus state actor-cids
  • cli: lotus-shed cid inspect-bundle

Execution Trace

For developers that are dependent on lotus execution trace, you will need to enable LOTUS_VM_ENABLE_TRACING envvar to get the exact execution trace response as before. Without the envvar enabled, Subcall details, DurationandGasCharges` fields will be missing from the new FVM trace.

Deal Proposal Migration

All deal proposals with non-utf8 string as the label in the metadata store will perform a light migration to new format as defined in v1.1.1.

Others

  • Resource manager is now only enabled by default on full daemon node. You can enable it manually for other nodes by setting env var LOTUS_RCMGR to 1.
  • Fix: drand: calculation of round from Filecoin epochs (filecoin-project/lotus#8606)

Dependency Update

  • chore: deps: update to go-libp2p v0.19.4 (filecoin-project/lotus#8801)
  • github.com/filecoin-project/go-fil-markets (v1.20.1 -> v1.20.1-v16-2):
  • github.com/filecoin-project/go-legs (v0.3.7 -> v0.3.10):
  • github.com/filecoin-project/go-state-types (v0.1.8 -> v0.1.10):
  • github.com/filecoin-project/specs-storage (v0.2.4 -> v0.4.1):

Contributors

Contributor Commits Lines ± Files Changed
@geoff-vball 37 +10565/-8454 150
@arajasek 33 +7964/-6340 473
@arajasek 37 +6220/-6976 457
@vyzo 135 +7287/-5380 546
@ZenGround0 19 +5958/-2920 226
@stebalien 18 +1566/-1101 116
Alex 5 +323/-2304 32
@zenground0 9 +583/-358 56
@jennijuju 5 +853/-27 19
@jennijuju 24 +392/-201 60
Marco Munizaga 3 +236/-83 5
@raulk 9 +93/-15 15
@travisperson 3 +37/-37 12
@Kubuxu 1 +41/-5 1
@koalacxr 1 +29/-13 3
@gammazero 2 +18/-10 4
Peter Rabbitson 1 +5/-3 1
Steve Loeppky 1 +6/-0 1
@masih 1 +3/-3 2
@magik6k 1 +4/-0 1
@jennijuju 1 +2/-2 1
tian zhou 1 +1/-1 1

1.15.3 / 2022-05-31

This is an optional release of lotus that include new APIs, some improvements and bug fixes.

New Features

Improvements

Bug Fixes

Dependency Updates

Others

Contributors

Contributor Commits Lines ± Files Changed
@stebalien 4 +607/-95 19
@magik6k 9 +550/-37 43
@geoff-vball 5 +279/-219 27
@simlecode 1 +306/-39 20
@arajasek 1 +256/-34 10
@zenground0 11 +214/-66 31
@arajasek 2 +149/-99 8
@vyzo 3 +125/-81 4
@Masih 1 +134/-15 7
@travisperson 3 +24/-32 6
@Rjan 6 +16/-16 9
@jennijuju 3 +9/-8 15
Rob Quist 3 +12/-4 3
@Icarus9913 1 +3/-3 3
@swift-mx 1 +3/-0 1
@Phi-rjan 1 +1/-1 1
@lifei 1 +1/-0 1

1.15.2 / 2022-05-06

This is a highly recommended feature lotus release v1.15.2. This feature release introduces many new features and for SPs, including PoSt workers, sealing scheduler, snap deal queue and so on.

Note: You need to be using go v1.18.1&up from this release onwards.

Highlights

❣️❣️❣️ PoSt Workers ❣️❣️❣️

‼️️Attention - the long-awaited yet highly requested PoSt workers, they are here! And they come in as a combo: you may setup PoSt workers for both winningPoSt or/and windowPoSt worker. You can also setup any number of PoSt workers as long as you have the hardware resources! For more details and learn how to set it up, see the docs here. You can also find early result of the PoSt workers performance by the community here.

In addition, we also added some handy toolings:

  • feat: miner: API/CLI to compute window-post (filecoin-project/lotus#8389)
    • run lotus-miner proving compute window-post to manually trigger a full windowPoSt computation for a specific deadline for full sanity checks.
  • feat: miner cli: Separate proving workers command (filecoin-project/lotus#8379)
    • run lotus-miner proving workers to list all the PoSt workers that's attached
  • feat: miner cli: proving check --faulty, faults in storage list sectors (filecoin-project/lotus#8349)
    • run lotus-miner proving check --faulty to identify the sectors that might be bad.

🔥🔥🔥 Sealing Scheduler Enhancement 🔥🔥🔥

Have you ever got a couple workers but only a few of them are super packed, while the rest are idling waiting for jobs? Now the task can be distributed more evenly with:

🌟🌟🌟 Snap Deal Enhancements 🌟🌟🌟

The Filecoin Network introduced Snap Deal with the network v15 OhSnap upgrade, and lotus shipped v1.14.0 with basic snappy support for SP to use this feature. Since then, we have received good ux feedbacks and bug reports from the community, and we are introducing a couple enhancement for SPs to better leverage this feature.

New Features

Improvements

Bug Fixes

Dependency Updates

Others

Contributors

Contributor Commits Lines ± Files Changed
@magik6k 95 +5147/-2922 401
@mz-sirius 3 +1789/-546 48
@nonsense 11 +777/-567 121
@arajasek 11 +336/-231 28
Darko Brdareski 1 +463/-13 95
@coryschwartz 11 +147/-217 13
spark8899 2 +300/-0 2
@zenground0 2 +6/-193 7
Eng Zer Jun 1 +31/-158 11
Kevin Li 2 +174/-0 14
@arajasek 5 +85/-86 18
@jennijuju 1 +0/-119 3
@jennijuju 1 +0/-98 6
@raulk 1 +60/-1 1
@frrist 1 +56/-0 2
@vyzo 3 +18/-16 5
@Masih 3 +29/-4 3
@jennijuju 4 +18/-11 11
@hannahhoward 1 +13/-10 2
@dirkmc 1 +21/-1 1
koalacxr 1 +10/-11 4
Aarsh Shah 1 +19/-1 1
@Rjan 6 +10/-8 7
@zl 1 +7/-1 1
KAYUII 1 +3/-2 1
@simlecode 1 +4/-0 1
@dirkmc 1 +1/-3 1
Jerry 1 +3/-0 1
@steblian 1 +1/-1 1
Geoff Stuart 1 +1/-0 1
Florian Ruen 1 +0/-1 1

1.15.1 / 2022-04-07

This is a HIGHLY recommended feature release v1.15.1, especially for node operators and storage providers who want to be a part of the content addressing network of Filecoin and IPFS. This feature release introduces Index Provider, GraphSync v2, and many other latest functionalities, improvements and bug fixes. More importantly, node operator can now enable the FVM(experimental) to sync mainnet!!

Highlights

🔥🔥🔥 FVM (Experimental) 🔥🔥🔥

🌟🌟🌟 Index Provider (Production Ready!) 🌟🌟🌟

More and more useful data is being stored on Filecoin via deals made by clients to Storage Providers. The goal is that this content is discoverable when people need them. To achieve that goal, one of the projects the Bedrock team is working on is building an Indexer Ecosystem, a project that's focus on content addressing on Filecoin, then potentially have interoperability with IPFS in the future and eventually serve the retrieval market. The Indexer Ecosystem high level architecture overview diagram can be found here and a detailed write up about can be found here.

That being said, with this release, lotus Storage Providers can easily become an Index Provider and serve the Indexer Content Addressing System. Index Providers generate advertisements from the deals made by a storage provider and announces the data to the indexer nodes for further processing:

  • To learn more about what is an Index Provider and how to be an Index Provider, read it here in lotus docuementation.
  • An one-off migration is needed in order for a Storage Provider to become an Index Provider and announce the proper formatted index. It's highly recommended for all Index Provider to do a force bulk initialization to enable index announcement on all existing deals.
    • Note that the Initialization places IO workload on your storage system. SP should set a proper concurrency based on your hardware or can stop/start initialization at their wish/convenience as proving deadlines approach and elapse, to avoid IOPS starvation or competition with window PoSt.
  • After the first one-time migration, being an Index Provider barely puts any extra usage on SP's market system.
    • You can find the testing result by SPX fellows here.

We recommend all Storage Providers that are serving deals in the Filecoin network to become a Index Provider, make the data you are storing discoverable for the retrieval market and retrieval clients!

❗️❗️❗️ Dag Migration For New CAR index format in DagStore ❗️❗️❗️

The index provider leverages the latest CARv2 indexing format MultihashIndexSorted, which stores the multihash code as well as the digest of all CIDs in a CAR file. Thus, all Storage Providers SHOULD perform an one-off DAG mirgation to regenerate DagStore CARv2 indices. You have to do it to become an index provider, failing to do so may also impact your future deal making. Follow the instruction here to perform the migration.

New Features

Improvements

Bug Fixes

Dependency Updates

Others

1.15.0 / 2022-03-09

This is an optional release with retrieval improvements(client side), SP ux with unsealing, snap deals and regular deal making and many other new features, improvements and bug fixes.

Highlights

  • feat:sealing: StartEpochSealingBuffer triggers packing on time(filecoin-project/lotus#7905)
    • use the StartEpochSealingBuffer configuration variable as a way to enforce that sectors are packed for sealing / updating no matter how many deals they have if the nearest deal start date is close enough to the present.
  • feat: #6017 market: retrieval ask CLI command (filecoin-project/lotus#7814)
  • feat(graphsync): allow setting of per-peer incoming requests for miners (filecoin-project/lotus#7578)
    • by setting SimultaneousTransfersForStoragePerClient in deal making configuration.
  • Make retrieval even faster (filecoin-project/lotus#7746)
  • feat: #7747 sealing: Adding conf variable for capping number of concurrent unsealing jobs (#7884) (filecoin-project/lotus#7884)
    • by setting MaxConcurrentUnseals in DAGStoreConfig

New Features

Improvements

Bug Fixes

Dependency Updates

  • github.com/filecoin-project/go-commp-utils (v0.1.2 -> v0.1.3):
  • github.com/filecoin-project/dagstore (v0.4.3 -> v0.4.4):
  • github.com/filecoin-project/go-fil-markets (v1.13.4 -> v1.19.2):
  • github.com/filecoin-project/go-statestore (v0.1.1 -> v0.2.0):
  • github.com/filecoin-project/go-storedcounter (v0.0.0-20200421200003-1c99c62e8a5b -> v0.1.0):
  • github.com/filecoin-project/specs-actors/v2 (v2.3.5 -> v2.3.6):

Others

Contributors

Contributor Commits Lines ± Files Changed
@arajasek 73 +7232/-2778 386
@zenground0 27 +5604/-1049 219
@vyzo 118 +4356/-1470 253
@zl 1 +3725/-309 8
@dirkmc 7 +1392/-1110 61
arajasek 37 +221/-1329 90
@magik6k 33 +1138/-336 101
@whyrusleeping 2 +483/-585 28
Darko Brdareski 14 +725/-276 154
@rvagg 2 +43/-947 10
@hannahhoward 5 +436/-335 31
@hannahhoward 12 +507/-133 37
@jennijuju 27 +333/-178 54
@TheMenko 8 +237/-179 17
c r 2 +227/-45 12
@dirkmck 12 +188/-40 27
@ribasushi 3 +128/-62 3
@raulk 6 +128/-49 9
@Whyrusleeping 1 +76/-70 8
@Stebalien 1 +55/-37 1
@jennijuju 11 +29/-16 11
@aarshkshah1992 1 +23/-19 5
@travisperson 1 +0/-18 2
@gstuart 3 +12/-1 3
@coryschwartz 4 +5/-6 4
@pefish 1 +4/-3 1
@Kubuxu 1 +5/-2 2
Colin Kennedy 1 +4/-2 1
Rob Quist 1 +2/-2 1
@shotcollin 1 +1/-1 1

1.14.4 / 2022-03-03

This is a highly recommended optional release for storage providers that are doing snap deals. This fix the bug that causes some snap deal sectors are stuck in FinalizeReplicaUpdate. In addition, SPs should be able to force update sectors status without getting blocked by normal shutdown of state machine.

v1.14.3 / 2022-02-28

This is an optional release, that includes a fix to properly register the --really-do-it flag for abort-upgrade.

1.14.2 / 2022-02-24

This is an optional release of lotus, that's had a couple more improvements w.r.t Snap experience for storage providers in preparation of theupcoming OhSnap upgrade.

Note that the network is STILL scheduled to upgrade to v15 on March 1st at 2022-03-01T15:00:00Z. All node operators, including storage providers, must upgrade to at least Lotus v1.14.0 before that time. Storage providers must update their daemons, miners, and worker(s).

Wanna know how to Snap your deal? Check this out!

Bug Fixes

  • fix lotus-bench for sealing jobs (#8173)
  • fix:sealing:really-do-it flag for abort upgrade (#8181)
  • fix:proving:post check sector handles snap deals replica faults (#8177)
  • fix: sealing: missing file type (#8180)

Others

  • Retract force-pushed v1.14.0 to work around stale gomod caches (#8159): We originally tagged v1.14.0 off the wrong commit and fixed that by a force push, in which is a really bad practise since it messes up the go mod. Therefore, we want to retract it and users may use v1.14.1&^.

Contributors

Contributor Commits Lines ± Files Changed
@zenground0 2 +73/-58 12
@eben.xie 1 +7/-0 1
@jennijuju 1 +4/-0 1
@jennijuju 1 +2/-1 1
@ribasushi 1 +2/-0 1

1.14.1 / 2022-02-18

This is an optional release of lotus, that fixes the incorrect comment of network v15 OhSnap upgrade date. Note the actual upgrade epoch in v1.14.0 was correct.

1.14.0 / 2022-02-17

This is a MANDATORY release of Lotus that introduces Filecoin network v15, codenamed the OhSnap upgrade.

The network is scheduled to upgrade to v15 on March 1st at 2022-03-01T15:00:00Z. All node operators, including storage providers, must upgrade to this release (or a later release) before that time. Storage providers must update their daemons, miners, and worker(s).

The OhSnap upgrade introduces the following FIPs, delivered in actors v7:

It is recommended that storage providers download the new params before updating their node, miner, and workers. To do so:

  • Download Lotus v1.14.0 or later
  • run make lotus-shed
  • run ./lotus-shed fetch-params with the appropriate proving-params flag
  • Upgrade the Lotus daemon and miner when the previous step is complete

All node operators, including storage providers, should be aware that a pre-migration will begin at 2022-03-01T13:30:00Z (90 minutes before the real upgrade). The pre-migration will take between 20 and 50 minutes, depending on hardware specs. During this time, expect slower block validation times, increased CPU and memory usage, and longer delays for API queries.

New Features and Changes

  • Integrate actor v7-rc1:
    • Integrate v7 actors (#7617)
    • feat: state: Fast migration for v15 (#7933)
    • fix: blockstore: Add missing locks to autobatch::Get() #7939)
    • correctness fixes for the autobatch blockstore (#7940)
  • Implement and support FIP-0019 Snap Deals
    • chore: deps: Integrate proof v11.0.0 (#7923)
    • Snap Deals Lotus Integration: FSM Posting and integration test (#7810)
    • Feat/sector storage unseal (#7730)
    • Feat/snap deals storage (#7615)
    • fix: sealing: Add more deal expiration checks during PRU pipeline (#7871)
    • chore: deps: Update go-paramfetch (#7917)
    • feat: #7880 gas: add gas charge for VerifyReplicaUpdate (#7897)
    • enhancement: sectors: disable existing cc upgrade path 2 days before the upgrade epoch (#7900)

Improvements

  • updating to new datastore/blockstore code with contexts (#7646)
  • reorder transfer checks so as to ensure sending 2B FIL to yourself fails if you don't have that amount (#7637)
  • VM: Circ supply should be constant per epoch (#7811)

Bug Fixes

  • Fix: state: circsuypply calc around null blocks (#7890)
  • Mempool msg selection should respect block message limits (#7321) SplitStore: supress compaction near upgrades (#7734)

Others

  • chore: create pull_request_template.md (#7726)

Contributors

Contributor Commits Lines ± Files Changed
Aayush Rajasekaran 41 +5538/-1205 189
zenground0 11 +3316/-524 124
Jennifer Wang 29 +714/-599 68
ZenGround0 3 +263/-25 11
c r 2 +198/-30 6
vyzo 4 +189/-7 7
Aayush 11 +146/-48 49
web3-bot 10 +99/-17 10
Steven Allen 1 +55/-37 1
Jiaying Wang 5 +30/-8 5
Jakub Sztandera 2 +8/-3 3
Łukasz Magiera 1 +3/-3 2
Travis Person 1 +2/-2 2
Rod Vagg 1 +2/-2 2

v1.13.2 / 2022-01-09

Lotus v1.13.2 is a highly recommended feature release with remarkable retrieval improvements, new features like worker management, schedule enhancements and so on.

Highlights

  • 🚀🚀🚀Improve retrieval deal experience
    • Testing result with MinerX.3 shows the retrieval deal success rate has increased dramatically with faster transfer speed, you can join or follow along furthur performance testings here. We recommend application developers to integrate with the new retrieval APIs to provide a better client experience.
    • 🌟🌟🌟 Reduce retrieval Time-To-First-Byte over 100x (#7693)
      • This change makes most free, small retrievals sub-second
    • 🌟🌟🌟 Partial retrieval ux improvements (#7610)
      • New retrieval commands for clients:
        • lotus client ls: retrieve and list desired object links
        • lotus client cat: retrieve and print the data from the network
      • 🌟🌟 The monolith ClientRetrieve method was broken into:
        • ClientRetrieve which retrieves data into the local repo (or into an IPFS node if ipfs integration is enabled)
        • ClientRetrieveWait which will wait for the retrieval to complete
        • ClientExport which will export data from the local node
        • Note: this change only applies to v1 API. v0 API remains unchanged.
      • 🌟 Support for full ipld selectors was added (for example making it possible to only retrieve list of directories in a deal, without fetching any file data)
        • To learn more, see here
  • 🚀🚀 Sealing scheduler enhancements (#7703, #7269), #7714
  • 🚀🚀 Sector storage groups (#7453)
    • Storage groups allow for better control of data flow between workers, for example, it makes it possible to define that data from PC1 on a given worker has to have it's PC2 step executed on the same worker
    • To set it up, follow the instructions under the Sector Storage Group section here

New Features

  • Add RLE dump code (#7691)
  • Shed: Add a util to list miner faults (#7605)
  • lotus-shed msg: Decode submessages/msig proposals (#7639)
  • CLI: Add a lotus multisig cancel command (#7645)
  • shed: simple wallet balancer util (#7414)
    • balancing token balance between multiple accounts

Improvements

  • Add verbose mode to lotus-miner pieces list-cids (#7699)
  • retrieval: Only output matching nodes, MatchPath dagspec (#7706)
  • Cleanup partial retrieval codepaths ( zero functional changes ) (#7688)
  • storage: Use 1M buffers for Tar transfers (#7681)
  • Chore/dm level tests plus merkle proof cars (#7673)
  • Shed: Add a util to create miners more easily (#7595)
  • add timeout flag to wait-api command (#7592)
  • add log for restart windows post scheduler (#7613)
  • remove jaeger envvars (#7631)
  • remove api and jaeger env from docker file (#7624)
  • Wdpost worker: Reduce challenge confidence to 1 epoch (#7572)
  • add additional methods to lotus gateway (#7644)
  • Add caches to lotus-stats and splitcode (#7329)
  • remote store: Remove debug printf (#7664)
  • docsgen-cli: Handle commands with no description correctly (#7659)

Bug Fixes

  • fix docker logic error (#7709)
  • add missing NodeType tag (#7559)
  • checkCommit should return SectorCommitFailed (#7555)
  • ffiwrapper: Validate PC2 by calling C1 with random seeds (#7710)

Dependency Updates

  • Update go-graphsync v0.10.6 (#7708)
  • update go-libp2p-pubsub to v0.5.6 (#7581)
  • Update go-state-types (#7591)
  • disable mplex stream muxer (#7689)
  • Bump ws from 5.2.2 to 5.2.3 in /lotuspond/front (#7660)
  • Bump color-string from 1.5.3 to 1.6.0 in /lotuspond/front (#7658)
  • Bump postcss from 7.0.17 to 7.0.39 in /lotuspond/front (#7657)
  • Bump path-parse from 1.0.6 to 1.0.7 in /lotuspond/front (#7656)
  • Bump tmpl from 1.0.4 to 1.0.5 in /lotuspond/front (#7655)
  • Bump url-parse from 1.4.7 to 1.5.3 in /lotuspond/front (#7654)
  • github.com/filecoin-project/go-state-types (v0.1.1-0.20210915140513-d354ccf10379 -> v0.1.1):

Others

  • Update archive script (#7690)

Contributors

Contributor Commits Lines ± Files Changed
@magik6k 89 +5200/-1818 232
Travis Person 5 +1473/-953 38
@arajasek 6 +550/-38 19
@clinta 4 +393/-123 26
@ribasushi 3 +334/-68 7
@jennijuju 13 +197/-120 67
@Kubuxu 10 +153/-30 10
@coryschwartz 6 +18/-26 6
Marten Seemann 2 +6/-34 5
@vyzo 1 +3/-3 2
@hannahhoward 1 +3/-3 2
@zenground0 2 +2/-2 2
@yaohcn 2 +2/-2 2
@jennijuju 1 +1/-1 1
@hunjixin 1 +1/-0 1

v1.13.1 / 2021-11-26

This is an optional Lotus v1.13.1 release.

New Features

Improvements

Bug Fixes

Dependency Updates

  • Update go-graphsync v0.10.1 (filecoin-project/lotus#7457)
  • update to proof v10.1.0 (filecoin-project/lotus#7564)
  • github.com/filecoin-project/specs-actors/v6 (v6.0.0 -> v6.0.1):
  • github.com/filecoin-project/go-jsonrpc (v0.1.4-0.20210217175800-45ea43ac2bec -> v0.1.5):
  • github.com/filecoin-project/go-fil-markets (v1.13.1 -> v1.13.3):
  • github.com/filecoin-project/go-data-transfer (v1.11.1 -> v1.11.4):
  • github.com/filecoin-project/go-crypto (v0.0.0-20191218222705-effae4ea9f03 -> v0.0.1):
  • github.com/filecoin-project/go-commp-utils (v0.1.1-0.20210427191551-70bf140d31c7 -> v0.1.2):
  • github.com/filecoin-project/go-cbor-util (v0.0.0-20191219014500-08c40a1e63a2 -> v0.0.1):
  • github.com/filecoin-project/go-address (v0.0.5 -> v0.0.6):
  • unpin the yamux dependency (filecoin-project/lotus#7532
  • peerstore@v0.2.9 was withdrawn, let's not depend on it directly (filecoin-project/lotus#7481)
  • chore(deps): use tagged github.com/ipld/go-ipld-selector-text-lite (filecoin-project/lotus#7464)
  • Stop indirectly depending on deprecated github.com/prometheus/common (filecoin-project/lotus#7473)

Others

Contributors

Contributor Commits Lines ± Files Changed
@magik6k 27 +1285/-531 76
@ribasushi 7 +265/-1635 21
@raulk 2 +2/-737 13
@nonsens 4 +391/-21 19
@arajasek 6 +216/-23 14
@jennijuju 8 +102/-37 29
Steven Allen 2 +77/-29 6
@jennijuju 4 +19/-18 11
@dirkmc 2 +9/-9 4
@@coryschwartz 1 +16/-2 2
@frrist 1 +12/-0 2
@Kubuxu 5 +5/-5 5
@hunjixin 2 +6/-3 2
@vyzo 1 +3/-3 2
@@rvagg 1 +3/-3 2
@hannahhoward 1 +3/-2 2
Marten Seemann 1 +3/-0 1
@ZenGround0 1 +1/-1 1

v1.13.0 / 2021-10-18

Lotus v1.13.0 is a highly recommended feature release for all lotus users(i.e: storage providers, data brokers, application developers and so on) that supports the upcoming Network v14 Chocolate upgrade. This feature release includes the latest functionalities and improvements, like data transfer rate-limiting for both storage and retrieval deals, proof v10 with CUDA support, etc. You can find more details in the Changelog below.

Highlights

  • Enable separate storage and retrieval transfer limits (filecoin-project/lotus#7405)
    • SimultaneousTransfer is now replaced by SimultaneousTransfersForStorage and SimultaneousTransfersForRetrieval, where users may set the amount of ongoing data transfer for storage and retrieval deals in parallel separately. The default value for both is set to 20.
    • If you are using the lotus client, these two configuration variables are under the Client section in ./lotus/config.toml.
    • If you are a service provider, these two configuration variables should be set under the Dealmaking section in /.lotusminer/config.toml.
  • Update proofs to v10.0.0 (filecoin-project/lotus#7420)
    • This version supports CUDA. To enable CUDA instead of openCL, build lotus with FFI_USE_CUDA=1 FFI_BUILD_FROM_SOURCE=1 ....
    • You can find additional Nvidia driver installation instructions written by MinerX fellows here and perf improvements result on PC2/C2/WindowPoSt computation on different profiles here, most people observe a 30-50% decrease in computation time.

New Features

Improvements

Bug Fixes

Dependency Updates

  • github.com/filecoin-project/go-data-transfer (v1.10.1 -> v1.11.1):
  • github.com/filecoin-project/go-fil-markets (v1.12.0 -> v1.13.1):
  • github.com/filecoin-project/go-paramfetch (v0.0.2-0.20210614165157-25a6c7769498 -> v0.0.2):
  • update go-libp2p to v0.15.0 (filecoin-project/lotus#7362)
  • update to go-graphsync v0.10.1 (filecoin-project/lotus#7359)

Others

Contributors

Contributor Commits Lines ± Files Changed
@dirkmc 8 +845/-375 55
@magik6k 10 +1056/-60 26
@aarshkshah1992 6 +813/-259 16
@arajasek 10 +552/-251 43
@ribasushi 6 +505/-78 22
@jennijuju 7 +212/-323 34
@nonsense 10 +335/-139 19
@dirkmc 8 +149/-55 16
@hannahhoward 4 +56/-32 17
@rvagg 4 +61/-13 9
@jennijuju 2 +0/-57 2
@hannahhoward 1 +33/-18 7
@Kubuxu 8 +27/-16 9
@coryschwartz 1 +16/-2 2
@travisperson 1 +14/-0 1
@frrist 1 +12/-0 2
@ognots 1 +0/-10 2
@lanzafame 1 +3/-3 1
@jennijuju 1 +2/-2 1
@swift-mx 1 +1/-1 1

v1.12.0 / 2021-10-12

This is a mandatory release of Lotus that introduces Filecoin Network v14, codenamed the Chocolate upgrade. The Filecoin mainnet will upgrade at epoch 1231620, on 2021-10-26T13:30:00Z.

The Chocolate upgrade introduces the following FIPs, delivered in v6 actors

  • FIP-0020: Add return value to WithdrawBalance
  • FIP-0021: Correct quality calculation on expiration
  • FIP-0022: Bad deals don't fail PublishStorageDeals
  • FIP-0023: Break ties between tipsets of equal weight
  • FIP-0024: BatchBalancer & BatchDiscount Post-HyperDrive Adjustment
  • FIP-0026: Extend sector faulty period from 2 weeks to 6 weeks

Note that this release is built on top of lotus v1.11.3. Enterprising users like storage providers, data brokers and others are recommended to use lotus v1.13.0 for latest new features, improvements and bug fixes.

New Features and Changes

  • Implement and support FIP-0024 BatchBalancer & BatchDiscount Post-HyperDrive Adjustment:
    • Precommit batch balancer support/config (filecoin-project/lotus#7410)
      • Set BatchPreCommitAboveBaseFee to decide whether sending out a PreCommits in individual messages or in a batch.
      • The default value of BatchPreCommitAboveBaseFee and AggregateAboveBaseFee are now updated to 0.32nanoFIL.
  • The amount of FIL withdrawn from WithdrawBalance from miner or market via lotus CLI is now printed out upon message landing on the chain.

Improvements

Bug Fixes

Dependency Updates

  • Add v6 actors
    • Protocol changes
      • Multisig Approve only hashes when hash in params
      • FIP 0020 WithdrawBalance methods return withdrawn value
      • FIP 0021 Fix bug in power calculation when extending verified deals sectors
      • FIP 0022 PublishStorageDeals drops errors in batch
      • FIP 0024 BatchBalancer update and burn added to PreCommitBatch
      • FIP 0026 Add FaultMaxAge extension
      • Reduce calls to power and reward actors by passing values from power cron
      • Defensive programming hardening power cron against programmer error
    • Implementation changes
      • Move to xerrors
      • Improved logging: burn events are not logged with reasons and burned value.
  • github.com/filecoin-project/go-state-types (v0.1.1-0.20210810190654-139e0e79e69e -> v0.1.1-0.20210915140513-d354ccf10379):

Others

Contributors

Contributor Commits Lines ± Files Changed
@ZenGround0 12 +4202/-2752 187
@arajasek 25 +4567/-854 190
@laudiacay 4 +1276/-435 37
@laudiacay 12 +1350/-209 43
@magik6k 1 +171/-13 8
@Stebalien 2 +115/-12 6
@jennijuju 7 +73/-34 26
@travisperson 2 +19/-19 7
@coryschwartz 1 +16/-2 2
@Kubuxu 5 +5/-5 5
@ribasushi 1 +5/-3 1

v1.11.3 / 2021-09-29

lotus v1.11.3 is a feature release that's highly recommended to ALL lotus users to upgrade, including node operators, storage providers and clients. It includes many improvements and bug fixes that result in perf improvements in different area, like deal making, sealing and so on.

Highlights

  • 🌟🌟Introduce `MaxStagingDealsBytes - reject new deals if our staging deals area is full (filecoin-project/lotus#7276)
    • Set MaxStagingDealsBytes under the [Dealmaking] section of the markets' subsystem's config.toml to reject new incoming deals when the deal-staging directory of market subsystem's repo gets too large.
  • 🌟🌟miner: Command to list/remove expired sectors locally (filecoin-project/lotus#7140)
    • run ./lotus-miner sectors expired -h for more details.
  • 🚀update to ffi to update-bellperson-proofs-v9-0-2 (filecoin-project/lotus#7369)
    • MinerX fellows(early testers of lotus releases) have reported faster WindowPoSt computation!
  • 🌟dealpublisher: Fully validate deals before publishing (filecoin-project/lotus#7234)
    • This excludes the expired deals before sending out a PSD message which reduces the chances of PSD message failure due to invalid deals.
  • 🌟Simple alert system; FD limit alerts (filecoin-project/lotus#7108)

New Features

Improvements

Bug Fixes

Dependency Updates

  • github.com/filecoin-project/go-fil-markets (v1.8.1 -> v1.12.0):
  • github.com/filecoin-project/go-data-transfer (v1.7.8 -> v1.10.1):
  • update to ffi to update-bellperson-proofs-v9-0-2 (filecoin-project/lotus#7369)
  • fix(deps): use go-graphsync v0.9.3 with hotfix
  • Update to unified go-graphsync v0.9.0 (filecoin-project/lotus#7197)

Others

Contributors

Contributor Commits Lines ± Files Changed
@magik6k 39 +3311/-1825 179
@Stebalien 23 +1935/-1417 84
@dirkmc 12 +921/-732 111
@dirkmc 12 +663/-790 30
@hannahhoward 3 +482/-275 46
@travisperson 1 +317/-65 5
@jennijuju 11 +223/-126 24
@hannahhoward 7 +257/-55 16
@nonsense 9 +258/-37 19
@raulk 4 +127/-36 13
@raulk 1 +43/-60 15
@arajasek 4 +74/-8 10
@Frank 2 +68/-8 3
@placer14 2 +52/-1 4
@ldoublewood 2 +15/-13 3
@lanzafame 1 +16/-2 1
@aarshkshah1992 2 +11/-6 2
@ZenGround0 2 +7/-6 2
@ognots 1 +0/-10 2
@KAYUII 2 +4/-4 2
@lanzafame 1 +6/-0 1
@jacobheun 1 +3/-3 1
@frank 1 +4/-0 1

v1.11.2 / 2021-09-06

lotus v1.11.2 is a feature release that's highly recommended ALL lotus users to upgrade, including node operators, storage providers and clients.

Highlights

  • 🌟🌟🌟 Introduce Dagstore and CARv2 for deal-making (#6671) (filecoin-project/lotus#6671)
    • lotus miner markets' Dagstore is a component of the markets subsystem in lotus-miner. It is a sharded store to hold large IPLD graphs efficiently, packaged as location-transparent attachable CAR files and it replaces the former Badger staging blockstore. It is designed to provide high efficiency and throughput, and minimize resource utilization during deal-making operations.
      The dagstore also leverages the indexing features of CARv2 to enable plan CAR files to act as read and write blockstores, which are served as the direct medium for data exchanges in markets for both storage and retrieval deal making without requiring intermediate buffers.
    • In the future, lotus will leverage and interact with Dagstore a lot for new features and improvements for deal making, therefore, it's highly recommended to lotus users to go through Lotus Miner: About the markets dagstore thoroughly to learn more about Dagstore's conceptual overview, terminology, directory structure, configuration and so on.
    • Note:
      • When you first start your lotus-miner or market subsystem with this release, a one-time/first-time dagstore migration will be triggered which replaces the former Badger staging blockstore with dagstore. We highly recommend storage providers to read this section to learn more about what the process does, what to expect and how monitor it.
      • It is highly recommended to wait all ongoing data transfer to finish or cancel inbound storage deals that are still transferring, using the lotus-miner data-transfers cancel command before upgrade your market nodes. Reason being that the new dagstore changes attributes in the internal deal state objects, and the paths to the staging CARs where the deal data was being placed will be lost.
      • ‼️Having your dags initialized will become important in the near feature for you to provide a better storage and retrieval service. We'd suggest you to start [forced bulk initialization] soon if possible as this process places relatively high IP workload on your storage system and is better to be carried out gradually and over a longer timeframe. Read how to do properly perform a force bulk initialization here.
      • ⏮ Rollback Alert(from v1.11.2-rcX to any version lower): If a storages deal is initiated with M1/v1.11.2(-rcX) release, it needs to get to the StorageDealAwaitingPrecommit state before you can do a version rollback or the markets process may panic.
    • 💙 Special thanks to MinerX fellows for testing and providing valuable feedbacks for Dagstore in the past month!
  • 🌟🌟 rpcenc: Support reader redirect (filecoin-project/lotus#6952)
    • This allows market processes to send piece bytes directly to workers involved on AddPiece.
  • Extending sectors: more practical and flexible tools (filecoin-project/lotus#6097)
    • lotus-miner sectors check-expire to inspect expiring sectors.
    • lotus-miner sectors renew for renewing expiring sectors, see the command help menu for customizable option like extension, new-expiration and so on.
  • ‼️ MpoolReplaceCmd ( lotus mpool replace`) now takes FIL for fee-limit (filecoin-project/lotus#6927)
  • Drop townhall/chainwatch (filecoin-project/lotus#6912)
    • ChainWatch is no longer supported by lotus.
  • Configurable CC Sector Expiration (filecoin-project/lotus#6803)
    • Set CommittedCapacitySectorLifetime in lotus-miner/config.toml to specify the default expiration for a new CC sector, value must be between 180-540 days inclusive.

New Features

Improvements

Bug Fixes

Dependency Updates

Others

Contributors

Contributor Commits Lines ± Files Changed
@magik6k 23 +5040/-8389 114
@aarshkshah1992 11 +4859/-1078 101
@raulk 5 +4170/-1662 104
@vyzo 30 +1092/-702 49
@nonsense 6 +630/-472 19
@ZenGround0 31 +556/-274 74
@He Weidong 16 +680/-128 16
@raulk 16 +444/-277 49
@Stebalien 11 +403/-259 48
@jennijuju 17 +276/-281 42
@dirkmc 5 +204/-138 20
@placer14 7 +178/-77 17
@BlocksOnAChain 1 +138/-0 1
@Frrist 1 +63/-56 2
@arajasek 7 +74/-42 13
@frrist 2 +67/-6 6
@hannahhoward 2 +13/-11 3
@coryschwartz 1 +16/-6 3
@whyrusleeping 1 +7/-7 1
@hunjixin 1 +8/-6 1
@aarshkshah1992 1 +6/-6 2
@dirkmc 2 +8/-0 2
@mx 2 +6/-1 2
@travisperson 1 +3/-2 1
@jennijuju 2 +2/-2 2
@ribasushi 1 +1/-2 2

1.11.1 / 2021-08-16

Note: for discussion about this release, please comment here

This is a highly recommended but optional Lotus v1.11.1 release that introduces many deal making and datastore improvements and new features along with other bug fixes.

Highlights

  • ⭐️⭐️⭐️lotus-miner market subsystem is introduced in this release! It is highly recommended for storage providers to run markets processes on a separate machine! Doing so, only this machine needs to exposes public ports for deal making. This also means that the other miner operations can now be completely isolated by from the deal making processes and storage providers can stop and restarts the markets process without affecting an ongoing Winning/Window PoSt!
    • More details on the concepts, architecture and how to split the market process can be found here.
    • Base on your system setup(running on separate machines, same machine and so on), please see the suggested practice by community members here.
      • Note: if you are running lotus-worker on a different machine, you will need to set MARKETS_API_INFO for certain CLI to work properly. This will be improved by #7072.
    • Huge thanks to MinerX fellows for helping testing the implementation, reporting the issues so they were fixed by now and providing feedbacks to user docs in the past three weeks!
  • Config for collateral from miner available balance (filecoin-project/lotus#6629)
    • Better control your sector collateral payment by setting CollateralFromMinerBalance, AvailableBalanceBuffer and DisableCollateralFallback.
      • CollateralFromMinerBalance: whether to use available miner balance for sector collateral instead of sending it with each message, default is false.
      • AvailableBalanceBuffer: minimum available balance to keep in the miner actor before sending it with messages, default is 0FIL.
      • DisableCollateralFallback: whether to send collateral with messages even if there is no available balance in the miner actor, default is false.
  • Config for deal publishing control addresses (filecoin-project/lotus#6697)
    • Set DealPublishControl to set the wallet used for sending PublishStorageDeals messages, instructions here.
  • Config UX improvements (filecoin-project/lotus#6848)
    • You can now preview the the default and updated node config by running lotus/lotus-miner config default/updated

New Features

Bug Fixes

Improvements

Dependency Updates

  • Update to proof v8.0.3 (filecoin-project/lotus#6890)
  • update to go-fil-market v1.6.0 (filecoin-project/lotus#6885)
  • Bump go-multihash, adjust test for supported version (filecoin-project/lotus#6674)
  • github.com/filecoin-project/go-data-transfer (v1.6.0 -> v1.7.2):
  • github.com/filecoin-project/go-fil-markets (v1.5.0 -> v1.6.2):
  • github.com/filecoin-project/go-padreader (v0.0.0-20200903213702-ed5fae088b20 -> v0.0.0-20210723183308-812a16dc01b1)
  • github.com/filecoin-project/go-state-types (v0.1.1-0.20210506134452-99b279731c48 -> v0.1.1-0.20210810190654-139e0e79e69e)
  • github.com/filecoin-project/go-statemachine (v0.0.0-20200925024713-05bd7c71fbfe -> v1.0.1)
  • update go-libp2p-pubsub to v0.5.0 (filecoin-project/lotus#6764)

Others

Contributors

Contributor Commits Lines ± Files Changed
@vyzo 313 +8928/-6010 415
@nonsense 103 +6041/-4041 304
@magik6k 37 +3851/-1611 146
@ZenGround0 24 +1693/-1394 95
@placer14 1 +2310/-578 8
@dirkmc 7 +1154/-726 29
@raulk 44 +969/-616 141
@jennijuju 15 +682/-354 47
@ribasushi 18 +469/-273 64
@coryschwartz 5 +576/-135 14
@hunjixin 7 +404/-82 19
@dirkmc 17 +348/-47 17
@tchardin 2 +262/-34 5
@aarshkshah1992 9 +233/-63 44
@Kubuxu 4 +254/-16 4
@hannahhoward 6 +163/-75 8
@whyrusleeping 4 +157/-16 9
@Whyrusleeping 2 +87/-66 10
@arajasek 10 +81/-53 13
@zgfzgf 2 +104/-4 2
@aarshkshah1992 6 +85/-19 10
@llifezou 4 +59/-20 4
@Stebalien 7 +47/-17 9
@johnli-helloworld 3 +46/-15 5
@frrist 1 +28/-23 2
@hannahhoward 4 +46/-5 11
@Jennifer 4 +31/-2 4
@wangchao 1 +1/-27 1
@jennijuju 2 +7/-21 2
@chadwick2143 1 +15/-1 1
@Jerry 2 +9/-4 2
Steve Loeppky 2 +12/-0 2
David Dias 1 +9/-0 1
dependabot[bot] 1 +3/-3 1
zhoutian527 1 +2/-2 1
xloem 1 +4/-0 1
2 +2/-2 3
Liviu Damian 2 +2/-2 2
@jimpick 2 +2/-2 2
Frank 1 +3/-0 1
turuslan 1 +1/-1 1
Kirk Baird 1 +0/-0 1

1.11.0 / 2021-07-22

This is a highly recommended release of Lotus that have many bug fixes, improvements and new features.

Highlights

More New Features

Bug Fixes

Improvements

Dependency Updates

Others

Contributors

Contributor Commits Lines ± Files Changed
@raulk 118 +11972/-10860 472
@magik6k 65 +10824/-4158 353
@aarshkshah1992 59 +8057/-3355 224
@arajasek 41 +8786/-1691 331
@Stebalien 106 +7653/-2718 273
dirkmc 11 +2580/-1371 77
@dirkmc 39 +1865/-1194 79
19 +1973/-485 81
@vyzo 4 +1748/-330 50
@aarshkshah1992 5 +1462/-213 27
@coryschwartz 35 +568/-206 59
@chadwick2143 3 +739/-1 4
@ribasushi 21 +487/-164 36
@hannahhoward 5 +544/-5 19
@jennijuju 9 +241/-174 19
@frrist 1 +137/-88 7
@travisperson 3 +175/-6 7
@wadeAlexC 1 +48/-129 1
@whyrusleeping 8 +161/-13 11
lotus 1 +114/-46 1
@nonsense 8 +107/-53 20
@rjan90 4 +115/-33 4
@ZenGround0 3 +114/-1 4
@Aloxaf 1 +43/-61 7
@yaohcn 4 +89/-9 5
@mitchellsoo 1 +51/-0 1
@placer14 3 +28/-18 4
@jennijuju 6 +9/-14 6
@Frank 2 +11/-10 2
@wangchao 3 +5/-4 4
@Steve Loeppky 1 +7/-1 1
@Lion 1 +4/-2 1
@Mimir 1 +2/-2 1
@raulk 1 +1/-1 1
@Jack Yao 1 +1/-1 1
@IPFSUnion 1 +1/-1 1

1.10.1 / 2021-07-05

This is an optional but highly recommended release of Lotus for lotus miners that has many bug fixes and improvements based on the feedback we got from the community since HyperDrive.

New Features

  • commit batch: AggregateAboveBaseFee config #6650
    • AggregateAboveBaseFee is added to miner sealing configuration for setting the network base fee to start aggregating proofs. When the network base fee is lower than this value, the prove commits will be submitted individually via ProveCommitSector. According to the Batch Incentive Alignment introduced in FIP-0013, we recommend miners to set this value to 0.15 nanoFIL(which is the default value) to avoid unexpected aggregation fee in burn and enjoy the most benefits of aggregation!

Bug Fixes

  • storage: Fix FinalizeSector with sectors in storage paths #6652
  • Fix tiny error in check-client-datacap #6664
  • Fix: precommit_batch method used the wrong cfg.PreCommitBatchWait #6658
  • to optimize the batchwait #6636
  • fix getTicket: sector precommitted but expired case #6635
  • handleSubmitCommitAggregate() exception handling #6595
  • remove precommit check in handleCommitFailed #6634
  • ensure agg fee is adequate
  • fix: miner balance is not enough, so that ProveCommitAggregate msg exec failed #6623
  • commit batch: Initialize the FailedSectors map #6647

Contributors

Contributor Commits Lines ± Files Changed
@magik6k 7 +151/-56 21
@llifezou 4 +59/-20 4
@johnli-helloworld 2 +45/-14 4
@wangchao 1 +1/-27 1
Jerry 2 +9/-4 2
@zhoutian527 1 +2/-2 1
@ribasushi 1 +1/-1 1

1.10.1 / 2021-07-05

This is an optional but highly recommended release of Lotus for lotus miners that has many bug fixes and improvements based on the feedback we got from the community since HyperDrive.

New Features

  • commit batch: AggregateAboveBaseFee config #6650
    • AggregateAboveBaseFee is added to miner sealing configuration for setting the network base fee to start aggregating proofs. When the network base fee is lower than this value, the prove commits will be submitted individually via ProveCommitSector. According to the Batch Incentive Alignment introduced in FIP-0013, we recommend miners to set this value to 0.15 nanoFIL(which is the default value) to avoid unexpected aggregation fee in burn and enjoy the most benefits of aggregation!

Bug Fixes

  • storage: Fix FinalizeSector with sectors in storage paths #6652
  • Fix tiny error in check-client-datacap #6664
  • Fix: precommit_batch method used the wrong cfg.PreCommitBatchWait #6658
  • to optimize the batchwait #6636
  • fix getTicket: sector precommitted but expired case #6635
  • handleSubmitCommitAggregate() exception handling #6595
  • remove precommit check in handleCommitFailed #6634
  • ensure agg fee is adequate
  • fix: miner balance is not enough, so that ProveCommitAggregate msg exec failed #6623
  • commit batch: Initialize the FailedSectors map #6647

Contributors

Contributor Commits Lines ± Files Changed
@magik6k 7 +151/-56 21
@llifezou 4 +59/-20 4
@johnli-helloworld 2 +45/-14 4
@wangchao 1 +1/-27 1
Jerry 2 +9/-4 2
@zhoutian527 1 +2/-2 1
@ribasushi 1 +1/-1 1

1.10.0 / 2021-06-23

This is a mandatory release of Lotus that introduces Filecoin network v13, codenamed the HyperDrive upgrade. The Filecoin mainnet will upgrade, which is epoch 892800, on 2021-06-30T22:00:00Z. The network upgrade introduces the following FIPs:

  • FIP-0008: Add miner batched sector pre-commit method
  • FIP-0011: Remove reward auction from reporting consensus faults
  • FIP-0012: DataCap Top up for FIL+ Client Addresses
  • FIP-0013: Add ProveCommitSectorAggregated method to reduce on-chain congestion
  • FIP-0015: Revert FIP-0009(Exempt Window PoSts from BaseFee burn)

Note that this release is built on top of Lotus v1.9.0. Enterprising users can use the master branch of Lotus to get the latest functionality, including all changes in this release candidate.

Proof batching and aggregation

FIPs 0008 and 0013 combine to allow for a significant increase in the rate of onboarding storage on the Filecoin network. This aims to lead to more useful data being stored on the network, reduced network congestion, and lower network base fee.

Check out the documentation [here]((https://lotus.filecoin.io/storage-providers/advanced-configurations/sealing/#precommitsectorsbatch) for details on the new Lotus miner sealing config options, here for fee config options, and explanations of the new features.

Note:

  • We recommend to keep PreCommitSectorsBatch as 1.
  • We recommend miners to set PreCommitBatchWait lower than 30 hours.
  • We recommend miners to set a longer CommitBatchSlack and PreCommitBatchSlack to prevent message failures due to expirations.

Projected state tree growth

In order to validate the Hyperdrive changes, we wrote a simulation to seal as many sectors as quickly as possible, assuming the same number and mix of 32GiB and 64GiB miners as the current network.

Given these assumptions:

  • We'd expect a network storage growth rate of around 530PiB per day. 😳 🎉 🥳 😅
  • We'd expect network bandwidth dedicated to SubmitWindowedPoSt to grow by about 0.02% per day.
  • We'd expect the state-tree (and therefore snapshot) size to grow by 1.16GiB per day.
    • Nearly all of the state-tree growth is expected to come from new sector metadata.
  • We'd expect the daily lotus datastore growth rate to increase by about 10-15% (from current ~21GiB/day).
    • Most "growth" of the lotus datastore is due to "churn", historical data that's no longer referenced by the latest state-tree.

Future improvements

Various Lotus improvements are planned moving forward to mitigate the effects of the growing state tree size. The primary improvement is the Lotus splitstore, which will soon be enabled by default. The feature allows for online garbage collection for nodes that do not seek to maintain full chain and state history, thus eliminating the need for users to delete their datastores and sync from snapshots.

Other improvements including better compressed snapshots, faster pre-migrations, and improved chain exports are in the roadmap.

WindowPost base fee burn

Included in the HyperDrive upgrade is FIP-0015 which eliminates the special-case gas treatment of SubmitWindowedPoSt messages that was introduced in FIP-0009. Although SubmitWindowedPoSt messages will be relatively cheap, thanks to the introduction of optimistic acceptance of these proofs in FIP-0010, storage providers should pay attention to their MaxWindowPoStGasFee config option: too low and PoSts may not land on chain; too high and they may cost an exorbitant amount!

Changelog

New Features

Bug Fixes

Improvements

Dependency Updates

  • Proofs v8.0.2 (filecoin-project/lotus#6524)
  • Update to fixed Bellperson (filecoin-project/lotus#6480)
  • Update to go-praamfetch with fslocks (filecoin-project/lotus#6473)
  • Update ffi with fixed multicore sdr support (filecoin-project/lotus#6471)
  • github.com/filecoin-project/go-paramfetch (v0.0.2-0.20200701152213-3e0f0afdc261 -> v0.0.2-0.20210614165157-25a6c7769498)
  • github.com/filecoin-project/specs-actors/v5 (v5.0.0-20210512015452-4fe3889fff57 -> v5.0.0)
  • github.com/filecoin-project/go-hamt-ipld/v3 (v3.0.1 -> v3.1.0)
  • github.com/ipfs/go-log/v2 (v2.1.2-0.20200626104915-0016c0b4b3e4 -> v2.1.3)
  • github.com/filecoin-project/go-amt-ipld/v3 (v3.0.0 -> v3.1.0)

Network Version v13 HyperDrive Upgrade

Contributors

💙Thank you to all the contributors!

Contributor Commits Lines ± Files Changed
@magik6k 81 +9606/-1536 361
@arajasek 41 +6543/-679 189
@ZenGround0 11 +4074/-727 110
@anorth 10 +2035/-1177 55
@iand 1 +779/-12 5
@frrist 2 +722/-6 6
@Stebalien 6 +368/-24 15
@jennijuju 11 +204/-111 19
@vyzo 6 +155/-66 13
@coryschwartz 10 +171/-27 14
@Kubuxu 4 +177/-13 7
@ribasushi 4 +65/-42 5
@travisperson 2 +11/-11 4
@kirk-baird 1 +1/-5 1
@wangchao 2 +3/-2 2

1.9.0 / 2021-05-17

This is an optional Lotus release that introduces various improvements to the sealing, mining, and deal-making processes.

Highlights

Changes

Fixes

1.8.0 / 2021-04-05

This is a mandatory release of Lotus that upgrades the network to version 12, which introduces various performance improvements to the cron processing of the power actor. The network will upgrade at height 712320, which is 2021-04-29T06:00:00Z.

Changes

1.6.0 / 2021-04-05

This is a mandatory release of Lotus that upgrades the network to version 11, which implements FIP-0014. The network will upgrade at height 665280, which is 2021-04-12T22:00:00Z.

v1 sector extension CLI

This release also expands the lotus-miner sectors extend CLI, with a new option that automatically extends all extensible v1 sectors. The option can be run using lotus-miner sectors extend --v1-sectors.

  • The tolerance flag can be passed to indicate what durations aren't "worth" extending. It defaults to one week, which means that sectors whose current lifetime's are within one week of the maximum possible lifetime will not be extended.

  • The expiration-cutoff flag can be passed to skip sectors whose expiration is past a certain point from the current head. It defaults to infinity (no cutoff), but if, say, 28800 was specified, then only sectors expiring in the next 10 days would be extended (2880 epochs in 1 day).

Changes

1.5.3 / 2021-03-24

This is a patch release of Lotus that introduces small fixes to the Storage FSM.

Changes

1.5.2 / 2021-03-11

This is an hotfix release of Lotus that fixes a critical bug introduced in v1.5.1 in the miner windowPoSt logic. This upgrade is only affecting miner nodes.

Changes

1.5.1 / 2021-03-10

This is an optional release of Lotus that introduces an important fix to the WindowPoSt computation process. The change is to wait for some confidence before drawing beacon randomness for the proof. Without this, invalid proofs might be generated as the result of a null tipset.

Splitstore

This release also introduces the splitstore, a new optional blockstore that segregates the monolithic blockstore into cold and hot regions. The hot region contains objects from the last 4-5 finalities plus all reachable objects from two finalities away. All other objects are moved to the cold region using a compaction process that executes every finality, once 5 finalities have elapsed.

The splitstore allows us to separate the two regions quite effectively, using two separate badger blockstores. The separation means that the live working set is much smaller, which results in potentially significant performance improvements. In addition, it means that the coldstore can be moved to a separate (bigger, slower, cheaper) disk without loss of performance.

The design also allows us to use different implementations for the two blockstores; for example, an append-only blockstore could be used for coldstore and a faster memory mapped blockstore could be used for the hotstore (eg LMDB). We plan to experiment with these options in the future.

Once the splitstore has been enabled, the existing monolithic blockstore becomes the coldstore. On the first head change notification, the splitstore will warm up the hotstore by copying all reachable objects from the current tipset into the hotstore. All new writes go into the hotstore, with the splitstore tracking the write epoch. Once 5 finalities have elapsed, and every finality thereafter, the splitstore compacts by moving cold objects into the coldstore. There is also experimental support for garbage collection, whereby nunreachable objects are simply discarded.

To enable the splitstore, add the following to config.toml:

[Chainstore]
  EnableSplitstore = true

Highlights

Other highlights include:

  • Improved deal data handling - now multiple deals can be adding to sectors in parallel
  • Rewriten sector pledging - it now actually cares about max sealing sector limits
  • Better handling for sectors stuck in the RecoverDealIDs state
  • lotus-miner sectors extend command
  • Optional configurable storage path size limit
  • Config to disable owner/worker fallback from control addresses (useful when owner is a key on a hardware wallet)
  • A write log for node metadata, which can be restored as a backup when the metadata leveldb becomes corrupted (e.g. when you run out of disk space / system crashes in some bad way)

Changes

1.5.0 / 2021-02-23

This is a mandatory release of Lotus that introduces the fifth upgrade to the Filecoin network. The network upgrade occurs at height 550321, before which time all nodes must have updated to this release (or later). At this height, v3 specs-actors will take effect, which in turn implements the following two FIPs:

  • FIP-0007 h/amt-v3 which improves the performance of the Filecoin HAMT and AMT.
  • FIP-0010 off-chain Window PoSt Verification which reduces the gas consumption of SubmitWindowedPoSt messages significantly by optimistically accepting Window PoSt proofs without verification, and allowing them to be disputed later by off-chain verifiers.

Note that the integration of v3 actors was already completed in 1.4.2, this upgrade simply sets the epoch for the upgrade to occur.

Disputer

FIP-0010 introduces the ability to dispute bad Window PoSts. Node operators are encouraged to run the new Lotus disputer alongside their Lotus daemons. For more information, see the announcement here.

Changes

  • #5341 Add a LOTUS_DISABLE_V3_ACTOR_MIGRATION envvar
    • Setting this envvar to 1 disables the v3 actor migration, should only be used in the event of a failed migration

1.4.2 / 2021-02-17

This is a large, and highly recommended, optional release with new features and improvements for lotus miner and deal-making UX. The release also integrates v3 specs-actors, which implements two FIPs:

  • FIP-0007 h/amt-v3 which improves the performance of the Filecoin HAMT and AMT.
  • FIP-0010 off-chain Window PoSt Verification which reduces the gas consumption of SubmitWindowedPoSt messages significantly by optimistically accepting Window PoSt proofs without verification, and allowing them to be disputed later by off-chain verifiers.

Note that this release does NOT set an upgrade epoch for v3 actors to take effect. That will be done in the upcoming 1.5.0 release.

New Features

  • #5341 Added sector termination API and CLI
    • Run lotus-miner sectors terminate
  • #5342 Added CLI for using a multisig wallet as miner's owner address
    • See how to set it up here
  • #5363, #5418, #5476, #5459 Integrated spec-actor v3
    • #5472 Generate actor v3 methods for pond
  • #5379 Added WindowPoSt disputer
  • #5309 Batch multiple deals in one PublishStorageMessages
    • #5411 Handle batch PublishStorageDeals message in sealing recovery
    • #5505 Exclude expired deals from batching in PublishStorageDeals messages
    • Added PublishMsgPeriod and MaxDealsPerPublishMsg to miner Dealmaking configuration. See how they work here.
    • #5538, #5549 Added a command to list pending deals and force publish messages.
      • Run lotus-miner market pending-publish
    • #5428 Moved waiting for PublishStorageDeals messages' receipt from markets to lotus
  • #5510 Added nerpanet build option
    • To build nerpanet, run make nerpanet
  • #5433 Added AlwaysKeepUnsealedCopy option to the miner configuration
  • #5520 Added MsigGetPending to get pending transactions for multisig wallets
  • #5219 Added interactive mode for lotus-wallet
  • 5529 Added support for minder nodes in lotus-shed rpc util

Bug Fixes

  • #5210 Miner should not dial client on restart
  • #5403 When estimating GasLimit only apply prior messages up to the nonce
  • #5410 Fix the calibnet build option
  • #5492 Fixed has for ipfsbstore for non-existing blocks
  • #5361 Fixed retrieval hangs when using IpfsOnlineMode=true
  • #5493 Fixed retrieval failure when price-per-byte is zero
  • #5506 Fixed contexts in the storage adpater
  • #5515 Properly wire up StateReadState on gateway API
  • #5582 Fixed error logging format strings
  • #5614 Fixed websocket reconnecting handling

Improvements

  • #5389 Show verified indicator for ./lotus-miner storage-deals list
  • #5229 Show power for verified deals in ./lotus-miner setocr list
  • #5407 Added explicit check of the miner address protocol
  • #5399 watchdog: increase heapprof capture threshold to 90%
  • #5398 storageadapter: Look at precommits on-chain since deal publish msg
  • #5470 Added --no-timing option for ./lotus state compute-state --html
  • #5417 Storage Manager: Always unseal full sectors
  • #5393 Switched to filecoin-ffi bls api for bls signatures
  • #5380 Refactor deals API tests
  • #5397 Fixed a flake in the sync manager edge case test
  • #5406 Added a test to ensure a correct window post cannot be disputed
  • #5294 Added jobs to build Lotus docker image and push it to AWS ECR
  • #5387 Added network info(mainnet|calibnet) in version
  • #5497 Export metric for lotus-gateaway
  • #4950 Removed bench policy
  • #5047 Improved the UX for ./lotus-shed bitfield enc
  • #5282 Snake a context through the chian blockstore creation
  • #5350 Avoid using mp.cfg directrly to prevent race condition
  • #5449 Documented the block-header better
  • #5404 Added retrying proofs if an incorrect one is generated
  • #4545 Made state tipset usage consistent in the API
  • #5540 Removed unnecessary database reads in validation check
  • #5554 Fixed build lotus-soup CI job
  • #5552 Updated CircleCI to halt gracefully
  • #5555 Cleanup and add docstrings of node builder
  • #5564 Stopped depending on gocheck with gomod
  • #5574 Updated CLI UI
  • #5570 Added code CID to StateReadState return object
  • #5565 Added storageadapter.PublishMsgConfig to miner in testkit for lotus-soup testplan
  • #5571 Added lotus-seed gensis car to generate lotus block for devnets
  • #5613 Check format in client commP util
  • #5507 Refactored coalescing logic into its own function and take both cancellation sets into account
  • #5592 Verify FFI version before building

Dependency Updates

Network Version v10 Upgrade

  • #5473 Merged staging branch for v1.5.0
  • #5603 Set nerpanet's upgrade epochs up to v3 actors
  • #5471, #5456 Set calibration net actor v3 migration epochs for testing
  • #5434 Implemented pre-migration framework
  • #5476 Tune migration

1.4.1 / 2021-01-20

This is an optional Lotus release that introduces various improvements to the sealing, mining, and deal-making processes. In particular, #5341 introduces the ability for Lotus miners to terminate sectors.

Changes

Core Lotus

Sealing

Dealmaking

UX

Testing

Build & Networks

Management

1.4.0 / 2020-12-19

This is a MANDATORY hotfix release of Lotus that resolves a chain halt at height 336,459 caused by nondeterminism in specs-actors. The fix is to update actors to 2.3.3 in order to incorporate this fix https://github.com/filecoin-project/specs-actors/pull/1334.

1.3.0 / 2020-12-16

This is a mandatory release of Lotus that introduces the third post-liftoff upgrade to the Filecoin network. The network upgrade occurs at height 343200, before which time all nodes must have updated to this release (or later). The change that breaks consensus is an implementation of FIP-0009(https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0009.md).

Changes

1.2.3 / 2020-12-15

This is an optional Lotus release that introduces many performance improvements, bugfixes, and UX improvements.

Changes

1.2.2 / 2020-12-03

This is an optional Lotus release that introduces various improvements to the mining logic and deal-making workflow, as well as several new UX features.

Changes

1.2.1 / 2020-11-20

This is a very small release of Lotus that fixes an issue users are experiencing when importing snapshots. There is no need to upgrade unless you experience an issue with creating a new datastore directory in the Lotus repo.

Changes

1.2.0 / 2020-11-18

This is a mandatory release of Lotus that introduces the second post-liftoff upgrade to the Filecoin network. The network upgrade occurs at height 265200, before which time all nodes must have updated to this release (or later). This release also bumps the required version of Go to 1.15.

The changes that break consensus are:

Other Changes

1.1.3 / 2020-11-13

This is an optional release of Lotus that upgrades Lotus dependencies, and includes many performance enhancements, bugfixes, and UX improvements.

Highlights

Changes

Core Lotus

Mining

Markets and Data Transfer

UX

Testing and Tooling

Dependencies

1.1.2 / 2020-10-24

This is a patch release of Lotus that builds on the fixes involving worker keys that was introduced in v1.1.1. Miners and node operators should update to this release as soon as possible in order to ensure their blocks are propagated and validated.

Changes

1.1.1 / 2020-10-24

This is a patch release of Lotus that addresses some issues caused by when miners change their worker keys. Miners and node operators should update to this release as soon as possible, especially any miner who has changed their worker key recently.

Changes

1.1.0 / 2020-10-20

This is a mandatory release that introduces the first post-liftoff upgrade to the Filecoin network. The changes that break consensus are an upgrade to specs-actors v2.2.0 at epoch 170000.

Changes

1.0.0 / 2020-10-19

It's 1.0.0! This is an optional release of Lotus that introduces some UX improvements to the 0.10 series.

This very small release is largely cosmetic, and intended to flag the code that the Filecoin mainnet was launched with.

API changes

  • StateMsgGasCost has been removed. The equivalent information can be gained by calling StateReplay.
  • A GasCost field has been added to the InvocResult type, meaning detailed gas costs will be returned when calling StateReplay, StateCompute, and StateCall.
  • The behaviour of StateReplay in response to an empty tipset key has been changed. Instead of simply using the heaviest tipset (which is almost guaranteed to be an unsuccessful replay), we search now search the chain for the tipset that included the message, and replay the message in that tipset (we fail if no such tipset is found).

Changes

0.10.2 / 2020-10-14

This is an optional release of Lotus that updates markets to 0.9.1, which fixes an issue affecting deals that were mid-transfer when the node was upgraded to 0.9.0. This release also includes some tweaks to default gas values and minor performance improvements.

Changes

0.10.1 / 2020-10-14

This is an optional release of Lotus that updates markets to 0.9.0, which adds the ability to restart data transfers. This release also introduces Ledger support, and various UX improvements.

Changes

0.10.0 / 2020-10-12

This is a consensus-breaking hotfix that addresses an issue in specs-actors v2.0.3 that made it impossible to pledge new 32GiB sectors. The change in Lotus is to update to actors v2.1.0, behind the new network version 5.

Changes

0.9.1 / 2020-10-10

This release fixes an issue which may cause the actors v2 migration to compute the state incorrectly when more than one migration is running in parallel.

Changes

0.9.0 / 2020-10-07

This consensus-breaking release of Lotus upgrades the actors version to v2.0.0. This requires migrating actor state from v0 to v2. The changes that break consensus are:

Changes

Dependencies

Core Lotus

Mining

UX

Tooling and validation

Contributors

The following contributors had 5 or more commits go into this release. We are grateful for every contribution!

Contributor Commits Lines ±
Stebalien 84 +3425/-2287
magik6k 41 +2121/-506
arajasek 39 +2467/-424
Kubuxu 25 +2344/-775
raulk 21 +287/-196
whyrusleeping 13 +727/-71
hsanjuan 13 +5886/-7956
dirkmc 11 +2634/-576
travisperson 8 +923/-202
ribasushi 6 +188/-128
zgfzgf 5 +21/-17

0.8.1 / 2020-09-30

This optional release of Lotus introduces a new version of markets which switches to CBOR-map encodings, and allows datastore migrations. The release also introduces several improvements to the mining process, a few performance optimizations, and a battery of UX additions and enhancements.

Changes

Dependencies

Core Lotus

Performance

Mining

UX

Tooling

Bootstrap

0.8.0 / 2020-09-26

This consensus-breaking release of Lotus introduces an upgrade to the network. The changes that break consensus are:

This release also updates go-fil-markets to fix an incompatibility issue between v0.7.2 and earlier versions.

Changes

Dependencies

Core Lotus

UX

Tooling and validation

Contributors

The following contributors had 5 or more commits go into this release. We are grateful for every contribution!

Contributor Commits Lines ±
arajasek 66 +3140/-1261
Stebalien 64 +3797/-3434
magik6k 48 +1892/-976
raulk 40 +2412/-1549
vyzo 22 +287/-196
alanshaw 15 +761/-146
whyrusleeping 15 +736/-52
hannahhoward 14 +1237/-837
anton 6 +32/-8
travisperson 5 +502/-6
Frank 5 +78/-39
Jennifer 5 +148/-41

0.7.2 / 2020-09-23

This optional release of Lotus introduces a major refactor around how a Lotus node interacts with code from the specs-actors repo. We now use interfaces to read the state of actors, which is required to be able to reason about different versions of actors code at the same time.

Additionally, this release introduces various improvements to the sync process, as well as changes to better the overall UX experience.

Changes

Core Lotus

User Experience

Testing and validation

0.7.1 / 2020-09-17

This optional release of Lotus introduces some critical fixes to the window PoSt process. It also upgrades some core dependencies, and introduces many improvements to the mining process, deal-making cycle, and overall User Experience.

Changes

Some notable improvements:

Dependencies

0.7.0 / 2020-09-10

This consensus-breaking release of Lotus is designed to test a network upgrade on the space race testnet. The changes that break consensus are:

  • Upgrading the Drand network used from the test Drand network to the League of Entropy main drand network. This is the same Drand network that will be used in the Filecoin mainnet.
  • Upgrading to specs-actors v0.9.8, which adds a new method to the Multisig actor.

Changes

Core Lotus

Storage Miner

Message pool

Maintenance

Contributors

The following contributors had commits go into this release. We are grateful for every contribution!

Contributor Commits Lines ±
arajasek 28 +1144/-239
Kubuxu 19 +452/-261
whyrusleeping 13 +456/-87
vyzo 11 +318/-20
raulk 10 +1289/-350
magik6k 6 +188/-55
dirkmc 3 +31/-8
alanshaw 3 +176/-37
Stebalien 2 +9/-12
lanzafame 1 +1/-1
frrist 1 +1/-1
mishmosh 1 +1/-1
nonsense 1 +1/-0

0.6.2 / 2020-09-09

This release introduces some critical fixes to message selection and gas estimation logic. It also adds the ability for nodes to mark a certain tipset as checkpointed, as well as various minor improvements and bugfixes.

Changes

Messagepool

Core Lotus

Storage

Maintenance

0.6.1 / 2020-09-08

This optional release introduces a minor improvement to the sync process, ensuring nodes don't fall behind and then resync.

Changes

0.6.0 / 2020-09-07

This consensus-breaking release of Lotus is designed to test a network upgrade on the space race testnet. The changes that break consensus are:

This release also introduces many improvements to Lotus! Among them are a new version of go-fil-markets that supports non-blocking retrieval, various spam reduction measures in the messagepool and p2p logic, and UX improvements to payment channels, dealmaking, and state inspection.

Changes

Core Lotus and dependencies

Messagepool

Payment channels

Peer-to-peer

Miscellaneous

UX

Testing & tooling

0.5.10 / 2020-09-03

This patch includes a crucial fix to the message pool selection logic, strongly disfavouring messages that might cause a miner penalty.

Changes

0.5.9 / 2020-09-03

This patch includes a hotfix to the GasEstimateFeeCap method, capping the estimated fee to a reasonable level by default.

Changes

0.5.8 / 2020-09-02

This patch includes some bugfixes to the sector sealing process, and updates go-fil-markets. It also improves the performance of blocksync, adds a method to export chain state trees, and improves chainwatch.

Changes

0.5.7 / 2020-08-31

This patch release includes some bugfixes and enhancements to the sector lifecycle and message pool logic.

Changes

0.5.6 / 2020-08-29

Hotfix release that fixes a panic in the sealing scheduler (https://github.com/filecoin-project/lotus/pull/3389).

0.5.5

This patch release introduces a large number of improvements to the sealing process. It also updates go-fil-markets to version 0.5.8, and go-libp2p-pubsub to v0.3.5.

Downstream upgrades

Sector sealing

  • The following improvements were introduced in https://github.com/filecoin-project/lotus/pull/3350.

    • Allow lotus-miner sectors remove to remove a sector in any state.
    • Create a separate state in the storage FSM dedicated to submitting the Commit message.
    • Recovery for when the Deal IDs of deals in a sector get changed in a reorg.
    • Auto-retry sending Precommit and Commit messages if they run out of gas
    • Auto-retry sector remove tasks when they fail
    • Compact worker windows, and allow their tasks to be executed in any order

  • Don't simply skip PoSt for bad sectors (https://github.com/filecoin-project/lotus/pull/3323)

Message Pool

Chainwatch

0.5.4

A patch release, containing a few nice bugfixes and improvements:

  • Fix parsing of peer ID in lotus-miner actor set-peer-id (@whyrusleeping)
  • Update dependencies, fixing several bugs (@Stebalien)
  • Fix remaining linter warnings (@Stebalien)
  • Use safe string truncation (@Ingar)
  • Allow tweaking of blocksync message window size (@whyrusleeping)
  • Add some additional gas stats to metrics (@Kubuxu)
  • Fix an edge case bug in message selection, add many tests (@vyzo)

0.5.3

Yet another hotfix release. A lesson for readers, having people who have been awake for 12+ hours review your hotfix PR is not a good idea. Find someone who has enough slept recently enough to give you good code review, otherwise you'll end up quickly bumping versions again.

  • Fixed a bug in the mempool that was introduced in v0.5.2

0.5.2 / 2020-08-24

This is a hotfix release.

  • Fix message selection to not include messages that are invalid for block inclusion.
  • Improve SelectMessage handling of the case where the message pools tipset differs from our mining base.

0.5.1 / 2020-08-24

The Space Race release! This release contains the genesis car file and bootstrap peers for the space race network.

Additionally, we included two small fixes to genesis creation:

  • Randomize ticket value in genesis generation
  • Correctly set t099 (burnt funds actor) to have valid account actor state

0.5.0 / 2020-08-20

This version of Lotus will be used for the incentivized testnet Space Race competition, and can be considered mainnet-ready code. It includes some protocol changes, upgrades of core dependencies, and various bugfixes and UX/performance improvements.

Highlights

Among the highlights included in this release are:

  • Gas changes: We implemented EIP-1559 and introduced real gas values.
  • Deal-making: We now support "Committed Capacity" sectors, "fast-retrieval" deals, and the packing of multiple deals into a single sector.
  • Renamed features: We renamed some of the binaries, environment variables, and default paths associated with a Lotus node.

Gas changes

We made some significant changes to the mechanics of gas in this release.

Network fee

We implemented something similar to Ethereum's EIP-1559. The Message structure had three changes:

  • The GasPrice field has been removed
  • A new GasFeeCap field has been added, which controls the maximum cost the sender incurs for the message
  • A new GasPremium field has been added, which controls the reward a miner earns for including the message

A sender will never be charged more than GasFeeCap * GasLimit. A miner will typically earn GasPremium * GasLimit as a reward.

The Blockheader structure has one new field, called ParentBaseFee. Informally speaking,the ParentBaseFee is increased when blocks are densely packed with messages, and decreased otherwise.

The ParentBaseFee is used when calculating how much a sender burns when executing a message. Burning simply refers to sending attoFIL to a dedicated, unreachable account. A message causes ParentBaseFee * GasUsed attoFIL to be burnt.

Real gas values

This release also includes our first "real" gas costs for primitive operations. The costs were designed to account for both the time that message execution takes, as well as the space a message adds to the state tree.

Deal-making changes

There are three key changes to the deal-making process.

Committed Capacity sectors

Miners can now pledge "Committed Capacity" (CC) sectors, which are explicitly stated as containing junk data, and must not include any deals. Miners can do this to increase their storage power, and win block rewards from this pledged storage.

They can mark these sectors as "upgradable" with lotus-miner sectors mark-for-upgrade. If the miner receives and accepts one or more storage deals, the sector that includes those deals will replace the CC sector. This is intended to maximize the amount of useful storage on the Filecoin network.

Fast-retrieval deals

Clients can now include a fast-retrieval flag when proposing deals with storage miners. If set to true, the miner will include an extra copy of the deal data. This data can be quickly served in a retrieval deal, since it will not need to be unsealed.

Multiple deals per sector

Miners can now pack multiple deals into a single sector, so long as all the deals fit into the sector capacity. This should increase the packing efficiency of miners.

Renamed features

To improve the user experience, we updated several names to mainatin standard prefixing, and to better reflect the meaning of the features being referenced.

In particular, the Lotus miner binary is now called lotus-miner, the default path for miner data is now ~/.lotusminer, and the environment variable that sets the path for miner data is now $LOTUS_MINER_PATH. A full list of renamed features can be found here.

Changelog

Downstream upgrades

Core protocol

Deal-making lifecycle

Enhancements

UX

Contributors

The following contributors had 10 or more commits go into this release. We are grateful for every contribution!

Contributor Commits Lines ±
magik6k 361 +13197/-6136
Kubuxu 227 +5670/-2587
arajasek 120 +2916/-1264
whyrusleeping 112 +3979/-1089
vyzo 99 +3343/-1305
dirkmc 68 +8732/-3621
laser 45 +1489/-501
hannahhoward 43 +2654/-990
frrist 37 +6630/-4338
schomatis 28 +3016/-1368
placer14 27 +824/-350
raulk 25 +28718/-29849
mrsmkl 22 +560/-368
travisperson 18 +1354/-314
nonsense 16 +2956/-2842
ingar 13 +331/-123
daviddias 11 +311/-11
Stebalien 11 +1204/-980
RobQuistNL 10 +69/-74

0.1.0 / 2019-12-11

We are very excited to release lotus 0.1.0. This is our testnet release. To install lotus and join the testnet, please visit lotu.sh. Please file bug reports as issues.

A huge thank you to all contributors for this testnet release!