release: v1.26.3 (#11908) (#11915) (#11922)

* deps: update dependencies to address migration memory bloat

to address memory concerns during a heavy migration

Ref: https://github.com/filecoin-project/go-state-types/pull/260
Ref: https://github.com/whyrusleeping/cbor-gen/pull/96
Ref: https://github.com/filecoin-project/go-amt-ipld/pull/90

* release: prep v1.26.3 patch

Prep v1.26.3 patch release:
- Update changelog, version and make gen + make docsgen-cli

* deps: update cbor-gen to tagged version

deps: update cbor-gen to tagged version

* deps: update go-state-types to tagged version

deps: update go-state-types to tagged version v0.13.2

* chore: deps: update go-state-types to v0.13.3

Fixes a panic when we have fewer than 1k proposals.

---------

Co-authored-by: Phi-rjan <orjan.roren@gmail.com>
Co-authored-by: Rod Vagg <rod@vagg.org>
Co-authored-by: Steven Allen <steven@stebalien.com>

CHANGELOG.md

@@ -6,6 +6,12 @@
 
 ## Improvements
 
+# v1.26.3 / 2024-04-22
+
+**This is a patch release that addresses high memory load concerns for the Lotus daemon in the coming network migration for network version 22, scheduled on epoch `3855360 - 2024-04-24 - 14:00:00Z`.**
+
+If your Lotus daemon is running on a machine with less memory and swap than 160GB, you should upgrade to this patch release to ensure you do not encounter any Out-Of-Memory issues during the pre-migration.
+
 # v1.26.2 / 2024-04-08
 
 **This is a mandatory patch release for the Filecoin network version 22 mainnet upgrade, for all node operators.**

documentation/en/cli-lotus-provider.md

@@ -0,0 +1,410 @@
+# lotus-provider
+```
+NAME:
+   lotus-provider - Filecoin decentralized storage network provider
+
+USAGE:
+   lotus-provider [global options] command [command options] [arguments...]
+
+VERSION:
+   1.26.3
+
+COMMANDS:
+   run      Start a lotus provider process
+   stop     Stop a running lotus provider
+   config   Manage node config by layers. The layer 'base' will always be applied. 
+   test     Utility functions for testing
+   version  Print version
+   help, h  Shows a list of commands or help for one command
+   DEVELOPER:
+     auth          Manage RPC permissions
+     log           Manage logging
+     wait-api      Wait for lotus api to come online
+     fetch-params  Fetch proving parameters
+
+GLOBAL OPTIONS:
+   --color              use color in display output (default: depends on output being a TTY)
+   --db-host value      Command separated list of hostnames for yugabyte cluster (default: "yugabyte") [$LOTUS_DB_HOST]
+   --db-name value      (default: "yugabyte") [$LOTUS_DB_NAME, $LOTUS_HARMONYDB_HOSTS]
+   --db-user value      (default: "yugabyte") [$LOTUS_DB_USER, $LOTUS_HARMONYDB_USERNAME]
+   --db-password value  (default: "yugabyte") [$LOTUS_DB_PASSWORD, $LOTUS_HARMONYDB_PASSWORD]
+   --layers value       (default: "base") [$LOTUS_LAYERS, $LOTUS_CONFIG_LAYERS]
+   --repo-path value    (default: "~/.lotusprovider") [$LOTUS_REPO_PATH]
+   --vv                 enables very verbose mode, useful for debugging the CLI (default: false)
+   --help, -h           show help
+   --version, -v        print the version
+```
+
+## lotus-provider run
+```
+NAME:
+   lotus-provider run - Start a lotus provider process
+
+USAGE:
+   lotus-provider run [command options] [arguments...]
+
+OPTIONS:
+   --listen value                     host address and port the worker api will listen on (default: "0.0.0.0:12300") [$LOTUS_WORKER_LISTEN]
+   --nosync                           don't check full-node sync status (default: false)
+   --manage-fdlimit                   manage open file limit (default: true)
+   --layers value [ --layers value ]  list of layers to be interpreted (atop defaults). Default: base (default: "base")
+   --storage-json value               path to json file containing storage config (default: "~/.lotus-provider/storage.json")
+   --journal value                    path to journal files (default: "~/.lotus-provider/")
+   --help, -h                         show help
+```
+
+## lotus-provider stop
+```
+NAME:
+   lotus-provider stop - Stop a running lotus provider
+
+USAGE:
+   lotus-provider stop [command options] [arguments...]
+
+OPTIONS:
+   --help, -h  show help
+```
+
+## lotus-provider config
+```
+NAME:
+   lotus-provider config - Manage node config by layers. The layer 'base' will always be applied. 
+
+USAGE:
+   lotus-provider config command [command options] [arguments...]
+
+COMMANDS:
+   default, defaults                Print default node config
+   set, add, update, create         Set a config layer or the base by providing a filename or stdin.
+   get, cat, show                   Get a config layer by name. You may want to pipe the output to a file, or use 'less'
+   list, ls                         List config layers you can get.
+   interpret, view, stacked, stack  Interpret stacked config layers by this version of lotus-provider, with system-generated comments.
+   remove, rm, del, delete          Remove a named config layer.
+   from-miner                       Express a database config (for lotus-provider) from an existing miner.
+   help, h                          Shows a list of commands or help for one command
+
+OPTIONS:
+   --help, -h  show help
+```
+
+### lotus-provider config default
+```
+NAME:
+   lotus-provider config default - Print default node config
+
+USAGE:
+   lotus-provider config default [command options] [arguments...]
+
+OPTIONS:
+   --no-comment  don't comment default values (default: false)
+   --help, -h    show help
+```
+
+### lotus-provider config set
+```
+NAME:
+   lotus-provider config set - Set a config layer or the base by providing a filename or stdin.
+
+USAGE:
+   lotus-provider config set [command options] a layer's file name
+
+OPTIONS:
+   --title value  title of the config layer (req'd for stdin)
+   --help, -h     show help
+```
+
+### lotus-provider config get
+```
+NAME:
+   lotus-provider config get - Get a config layer by name. You may want to pipe the output to a file, or use 'less'
+
+USAGE:
+   lotus-provider config get [command options] layer name
+
+OPTIONS:
+   --help, -h  show help
+```
+
+### lotus-provider config list
+```
+NAME:
+   lotus-provider config list - List config layers you can get.
+
+USAGE:
+   lotus-provider config list [command options] [arguments...]
+
+OPTIONS:
+   --help, -h  show help
+```
+
+### lotus-provider config interpret
+```
+NAME:
+   lotus-provider config interpret - Interpret stacked config layers by this version of lotus-provider, with system-generated comments.
+
+USAGE:
+   lotus-provider config interpret [command options] a list of layers to be interpreted as the final config
+
+OPTIONS:
+   --layers value [ --layers value ]  comma or space separated list of layers to be interpreted (default: "base")
+   --help, -h                         show help
+```
+
+### lotus-provider config remove
+```
+NAME:
+   lotus-provider config remove - Remove a named config layer.
+
+USAGE:
+   lotus-provider config remove [command options] [arguments...]
+
+OPTIONS:
+   --help, -h  show help
+```
+
+### lotus-provider config from-miner
+```
+NAME:
+   lotus-provider config from-miner - Express a database config (for lotus-provider) from an existing miner.
+
+USAGE:
+   lotus-provider config from-miner [command options] [arguments...]
+
+DESCRIPTION:
+   Express a database config (for lotus-provider) from an existing miner.
+
+OPTIONS:
+   --miner-repo value, --storagerepo value  Specify miner repo path. flag(storagerepo) and env(LOTUS_STORAGE_PATH) are DEPRECATION, will REMOVE SOON (default: "~/.lotusminer") [$LOTUS_MINER_PATH, $LOTUS_STORAGE_PATH]
+   --to-layer value, -t value               The layer name for this data push. 'base' is recommended for single-miner setup.
+   --overwrite, -o                          Use this with --to-layer to replace an existing layer (default: false)
+   --help, -h                               show help
+```
+
+## lotus-provider test
+```
+NAME:
+   lotus-provider test - Utility functions for testing
+
+USAGE:
+   lotus-provider test command [command options] [arguments...]
+
+COMMANDS:
+   window-post, wd, windowpost, wdpost  Compute a proof-of-spacetime for a sector (requires the sector to be pre-sealed). These will not send to the chain.
+   help, h                              Shows a list of commands or help for one command
+
+OPTIONS:
+   --help, -h  show help
+```
+
+### lotus-provider test window-post
+```
+NAME:
+   lotus-provider test window-post - Compute a proof-of-spacetime for a sector (requires the sector to be pre-sealed). These will not send to the chain.
+
+USAGE:
+   lotus-provider test window-post command [command options] [arguments...]
+
+COMMANDS:
+   here, cli                                       Compute WindowPoSt for performance and configuration testing.
+   task, scheduled, schedule, async, asynchronous  Test the windowpost scheduler by running it on the next available lotus-provider. 
+   help, h                                         Shows a list of commands or help for one command
+
+OPTIONS:
+   --help, -h  show help
+```
+
+#### lotus-provider test window-post here
+```
+NAME:
+   lotus-provider test window-post here - Compute WindowPoSt for performance and configuration testing.
+
+USAGE:
+   lotus-provider test window-post here [command options] [deadline index]
+
+DESCRIPTION:
+   Note: This command is intended to be used to verify PoSt compute performance.
+   It will not send any messages to the chain. Since it can compute any deadline, output may be incorrectly timed for the chain.
+
+OPTIONS:
+   --deadline value                   deadline to compute WindowPoSt for  (default: 0)
+   --layers value [ --layers value ]  list of layers to be interpreted (atop defaults). Default: base (default: "base")
+   --storage-json value               path to json file containing storage config (default: "~/.lotus-provider/storage.json")
+   --partition value                  partition to compute WindowPoSt for (default: 0)
+   --help, -h                         show help
+```
+
+#### lotus-provider test window-post task
+```
+NAME:
+   lotus-provider test window-post task - Test the windowpost scheduler by running it on the next available lotus-provider. 
+
+USAGE:
+   lotus-provider test window-post task [command options] [arguments...]
+
+OPTIONS:
+   --deadline value                   deadline to compute WindowPoSt for  (default: 0)
+   --layers value [ --layers value ]  list of layers to be interpreted (atop defaults). Default: base (default: "base")
+   --help, -h                         show help
+```
+
+## lotus-provider version
+```
+NAME:
+   lotus-provider version - Print version
+
+USAGE:
+   lotus-provider version [command options] [arguments...]
+
+OPTIONS:
+   --help, -h  show help
+```
+
+## lotus-provider auth
+```
+NAME:
+   lotus-provider auth - Manage RPC permissions
+
+USAGE:
+   lotus-provider auth command [command options] [arguments...]
+
+COMMANDS:
+   create-token  Create token
+   api-info      Get token with API info required to connect to this node
+   help, h       Shows a list of commands or help for one command
+
+OPTIONS:
+   --help, -h  show help
+```
+
+### lotus-provider auth create-token
+```
+NAME:
+   lotus-provider auth create-token - Create token
+
+USAGE:
+   lotus-provider auth create-token [command options] [arguments...]
+
+OPTIONS:
+   --perm value  permission to assign to the token, one of: read, write, sign, admin
+   --help, -h    show help
+```
+
+### lotus-provider auth api-info
+```
+NAME:
+   lotus-provider auth api-info - Get token with API info required to connect to this node
+
+USAGE:
+   lotus-provider auth api-info [command options] [arguments...]
+
+OPTIONS:
+   --perm value  permission to assign to the token, one of: read, write, sign, admin
+   --help, -h    show help
+```
+
+## lotus-provider log
+```
+NAME:
+   lotus-provider log - Manage logging
+
+USAGE:
+   lotus-provider log command [command options] [arguments...]
+
+COMMANDS:
+   list       List log systems
+   set-level  Set log level
+   alerts     Get alert states
+   help, h    Shows a list of commands or help for one command
+
+OPTIONS:
+   --help, -h  show help
+```
+
+### lotus-provider log list
+```
+NAME:
+   lotus-provider log list - List log systems
+
+USAGE:
+   lotus-provider log list [command options] [arguments...]
+
+OPTIONS:
+   --help, -h  show help
+```
+
+### lotus-provider log set-level
+```
+NAME:
+   lotus-provider log set-level - Set log level
+
+USAGE:
+   lotus-provider log set-level [command options] [level]
+
+DESCRIPTION:
+   Set the log level for logging systems:
+
+      The system flag can be specified multiple times.
+
+      eg) log set-level --system chain --system chainxchg debug
+
+      Available Levels:
+      debug
+      info
+      warn
+      error
+
+      Environment Variables:
+      GOLOG_LOG_LEVEL - Default log level for all log systems
+      GOLOG_LOG_FMT   - Change output log format (json, nocolor)
+      GOLOG_FILE      - Write logs to file
+      GOLOG_OUTPUT    - Specify whether to output to file, stderr, stdout or a combination, i.e. file+stderr
+
+
+OPTIONS:
+   --system value [ --system value ]  limit to log system
+   --help, -h                         show help
+```
+
+### lotus-provider log alerts
+```
+NAME:
+   lotus-provider log alerts - Get alert states
+
+USAGE:
+   lotus-provider log alerts [command options] [arguments...]
+
+OPTIONS:
+   --all       get all (active and inactive) alerts (default: false)
+   --help, -h  show help
+```
+
+## lotus-provider wait-api
+```
+NAME:
+   lotus-provider wait-api - Wait for lotus api to come online
+
+USAGE:
+   lotus-provider wait-api [command options] [arguments...]
+
+CATEGORY:
+   DEVELOPER
+
+OPTIONS:
+   --timeout value  duration to wait till fail (default: 30s)
+   --help, -h       show help
+```
+
+## lotus-provider fetch-params
+```
+NAME:
+   lotus-provider fetch-params - Fetch proving parameters
+
+USAGE:
+   lotus-provider fetch-params [command options] [sectorSize]
+
+CATEGORY:
+   DEVELOPER
+
+OPTIONS:
+   --help, -h  show help
+```

go.mod

@@ -35,7 +35,7 @@ require (
 	github.com/filecoin-project/dagstore v0.5.2
 	github.com/filecoin-project/filecoin-ffi v0.30.4-0.20220519234331-bfd1f5f9fe38
 	github.com/filecoin-project/go-address v1.1.0
-	github.com/filecoin-project/go-amt-ipld/v4 v4.2.0
+	github.com/filecoin-project/go-amt-ipld/v4 v4.3.0
 	github.com/filecoin-project/go-bitfield v0.2.4
 	github.com/filecoin-project/go-cbor-util v0.0.1
 	github.com/filecoin-project/go-commp-utils v0.1.3
@@ -49,7 +49,7 @@ require (
 	github.com/filecoin-project/go-jsonrpc v0.3.1
 	github.com/filecoin-project/go-padreader v0.0.1
 	github.com/filecoin-project/go-paramfetch v0.0.4
-	github.com/filecoin-project/go-state-types v0.13.1
+	github.com/filecoin-project/go-state-types v0.13.3
 	github.com/filecoin-project/go-statemachine v1.0.3
 	github.com/filecoin-project/go-statestore v0.2.0
 	github.com/filecoin-project/go-storedcounter v0.1.0
@@ -144,7 +144,7 @@ require (
 	github.com/triplewz/poseidon v0.0.0-20230828015038-79d8165c88ed
 	github.com/urfave/cli/v2 v2.25.5
 	github.com/whyrusleeping/bencher v0.0.0-20190829221104-bb6607aa8bba
-	github.com/whyrusleeping/cbor-gen v0.1.0
+	github.com/whyrusleeping/cbor-gen v0.1.1
 	github.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7
 	github.com/xeipuuv/gojsonschema v1.2.0
 	github.com/xorcare/golden v0.6.1-0.20191112154924-b87f686d7542

go.sum

@@ -305,8 +305,8 @@ github.com/filecoin-project/go-amt-ipld/v3 v3.0.0/go.mod h1:Qa95YNAbtoVCTSVtX38a
 github.com/filecoin-project/go-amt-ipld/v3 v3.1.0 h1:ZNJ9tEG5bE72vBWYiuh5bkxJVM3ViHNOmQ7qew9n6RE=
 github.com/filecoin-project/go-amt-ipld/v3 v3.1.0/go.mod h1:UjM2QhDFrrjD5s1CdnkJkat4ga+LqZBZgTMniypABRo=
 github.com/filecoin-project/go-amt-ipld/v4 v4.0.0/go.mod h1:gF053YQ4BIpzTNDoEwHZas7U3oAwncDVGvOHyY8oDpE=
-github.com/filecoin-project/go-amt-ipld/v4 v4.2.0 h1:DQTXQwMXxaetd+lhZGODjt5qC1WYT7tMAlYrWqI/fwI=
+github.com/filecoin-project/go-amt-ipld/v4 v4.3.0 h1:bY42N1gR0DqrLMCKUPzX1VhYVgXaETQm0Um4ohvyEP8=
-github.com/filecoin-project/go-amt-ipld/v4 v4.2.0/go.mod h1:0eDVF7pROvxrsxvLJx+SJZXqRaXXcEPUcgb/rG0zGU4=
+github.com/filecoin-project/go-amt-ipld/v4 v4.3.0/go.mod h1:39Ep/yBbF6xN94WevLG9qSbglBJepHa5zeEbAE1pYsc=
 github.com/filecoin-project/go-bitfield v0.2.0/go.mod h1:CNl9WG8hgR5mttCnUErjcQjGvuiZjRqK9rHVBsQF4oM=
 github.com/filecoin-project/go-bitfield v0.2.3/go.mod h1:CNl9WG8hgR5mttCnUErjcQjGvuiZjRqK9rHVBsQF4oM=
 github.com/filecoin-project/go-bitfield v0.2.4 h1:uZ7MeE+XfM5lqrHJZ93OnhQKc/rveW8p9au0C68JPgk=
@@ -355,8 +355,8 @@ github.com/filecoin-project/go-state-types v0.1.0/go.mod h1:ezYnPf0bNkTsDibL/psS
 github.com/filecoin-project/go-state-types v0.1.6/go.mod h1:UwGVoMsULoCK+bWjEdd/xLCvLAQFBC7EDT477SKml+Q=
 github.com/filecoin-project/go-state-types v0.1.10/go.mod h1:UwGVoMsULoCK+bWjEdd/xLCvLAQFBC7EDT477SKml+Q=
 github.com/filecoin-project/go-state-types v0.11.2-0.20230712101859-8f37624fa540/go.mod h1:SyNPwTsU7I22gL2r0OAPcImvLoTVfgRwdK/Y5rR1zz8=
-github.com/filecoin-project/go-state-types v0.13.1 h1:4CivvlcHAIoAtFFVVlZtokynaMQu5XLXGoTKhQkfG1I=
+github.com/filecoin-project/go-state-types v0.13.3 h1:9JPkC0E6HDtfHbaOIrFiCDzT/Z0jRTb9En4Y4Ci/b3w=
-github.com/filecoin-project/go-state-types v0.13.1/go.mod h1:cHpOPup9H1g2T29dKHAjC2sc7/Ef5ypjuW9A3I+e9yY=
+github.com/filecoin-project/go-state-types v0.13.3/go.mod h1:cHpOPup9H1g2T29dKHAjC2sc7/Ef5ypjuW9A3I+e9yY=
 github.com/filecoin-project/go-statemachine v0.0.0-20200925024713-05bd7c71fbfe/go.mod h1:FGwQgZAt2Gh5mjlwJUlVB62JeYdo+if0xWxSEfBD9ig=
 github.com/filecoin-project/go-statemachine v1.0.3 h1:N07o6alys+V1tNoSTi4WuuoeNC4erS/6jE74+NsgQuk=
 github.com/filecoin-project/go-statemachine v1.0.3/go.mod h1:jZdXXiHa61n4NmgWFG4w8tnqgvZVHYbJ3yW7+y8bF54=
@@ -1681,8 +1681,8 @@ github.com/whyrusleeping/cbor-gen v0.0.0-20200826160007-0b9f6c5fb163/go.mod h1:f
 github.com/whyrusleeping/cbor-gen v0.0.0-20210118024343-169e9d70c0c2/go.mod h1:fgkXqYy7bV2cFeIEOkVTZS/WjXARfBqSH6Q2qHL33hQ=
 github.com/whyrusleeping/cbor-gen v0.0.0-20210303213153-67a261a1d291/go.mod h1:fgkXqYy7bV2cFeIEOkVTZS/WjXARfBqSH6Q2qHL33hQ=
 github.com/whyrusleeping/cbor-gen v0.0.0-20220323183124-98fa8256a799/go.mod h1:fgkXqYy7bV2cFeIEOkVTZS/WjXARfBqSH6Q2qHL33hQ=
-github.com/whyrusleeping/cbor-gen v0.1.0 h1:Jneeq3V5enErVcuL0NKEbD1Gi+iOvEeFhXOV1S1Fc6g=
+github.com/whyrusleeping/cbor-gen v0.1.1 h1:eKfcJIoxivjMtwfCfmJAqSF56MHcWqyIScXwaC1VBgw=
-github.com/whyrusleeping/cbor-gen v0.1.0/go.mod h1:pM99HXyEbSQHcosHc0iW7YFmwnscr+t9Te4ibko05so=
+github.com/whyrusleeping/cbor-gen v0.1.1/go.mod h1:pM99HXyEbSQHcosHc0iW7YFmwnscr+t9Te4ibko05so=
 github.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f h1:jQa4QT2UP9WYv2nzyawpKMOCl+Z/jW7djv2/J50lj9E=
 github.com/whyrusleeping/chunker v0.0.0-20181014151217-fe64bd25879f/go.mod h1:p9UJB6dDgdPgMJZs7UjUOdulKyRr9fqkS+6JKAInPy8=
 github.com/whyrusleeping/go-keyspace v0.0.0-20160322163242-5b898ac5add1 h1:EKhdznlJHPMoKr0XTrX+IlJs1LH3lyx2nfr1dOlZ79k=