Commit Graph

2665 Commits

Author SHA1 Message Date
Łukasz Magiera
28099a3905 storage: Move storage/sealer/stores to storage/paths 2022-06-14 20:25:52 +02:00
Łukasz Magiera
82857e6d5d chore: rerun gen with moved packages 2022-06-14 20:13:31 +02:00
Łukasz Magiera
a9600b8a6f storage: Move extern/sector-storage to storage/sealer 2022-06-14 20:03:38 +02:00
Łukasz Magiera
98a48a47f8 storage: Move extern/storage-sealing to storage/pipeline 2022-06-14 19:41:59 +02:00
Łukasz Magiera
717bda63b1 storage: Move control address logic to storage/ctladdr 2022-06-14 19:32:29 +02:00
Łukasz Magiera
dff1bf2868 storage: Move window-post logic into a separate package 2022-06-14 19:27:09 +02:00
Łukasz Magiera
e65fae28de chore: fix imports 2022-06-14 17:00:51 +02:00
Steven Allen
ddc9425c07 feat: refactor: actor bundling system (#8838)
1. Include the builtin-actors in the lotus source tree.
2. Embed the bundle on build instead of downloading at runtime.
3. Avoid reading the bundle whenever possible by including bundle
   metadata (the bundle CID, the actor CIDs, etc.).
4. Remove everything related to dependency injection.
    1. We're no longer downloading the bundle, so doing anything ahead
       of time doesn't really help.
    2. We register the manifests on init because, unfortunately, they're
       global.
    3. We explicitly load the current actors bundle in the genesis
       state-tree method.
    4. For testing, we just change the in-use bundle with a bit of a
       hack. It's not great, but using dependency injection doesn't make
       any sense either because, again, the manifest information is
       global.
    5. Remove the bundle.toml file. Bundles may be overridden by
       specifying an override path in the parameters file, or an
       environment variable.

fixes #8701
2022-06-13 10:51:49 -07:00
Steven Allen
30981d0fdd
feat: refactor: actor bundling system (#8838)
1. Include the builtin-actors in the lotus source tree.
2. Embed the bundle on build instead of downloading at runtime.
3. Avoid reading the bundle whenever possible by including bundle
   metadata (the bundle CID, the actor CIDs, etc.).
4. Remove everything related to dependency injection.
    1. We're no longer downloading the bundle, so doing anything ahead
       of time doesn't really help.
    2. We register the manifests on init because, unfortunately, they're
       global.
    3. We explicitly load the current actors bundle in the genesis
       state-tree method.
    4. For testing, we just change the in-use bundle with a bit of a
       hack. It's not great, but using dependency injection doesn't make
       any sense either because, again, the manifest information is
       global.
    5. Remove the bundle.toml file. Bundles may be overridden by
       specifying an override path in the parameters file, or an
       environment variable.

fixes #8701
2022-06-13 10:15:00 -07:00
Łukasz Magiera
1cd94f598d
Merge pull request #8820 from filecoin-project/asr/deprecate-blocksync
refactor: remove old BlockSyncProtocolID
2022-06-13 18:36:21 +02:00
vyzo
d4d5ce3021 only enable rcmgr by default in full nodes
opt-in with envvar for other nodes, as there are still some issues
with markets related protocols
2022-06-13 12:16:20 -04:00
Geoff Stuart
b7010c9e60 Implement function to migrate actors with only code changes 2022-06-10 15:52:32 -04:00
Geoff Stuart
801c670edd Remove ChainPutMany 2022-06-10 14:09:05 -04:00
Geoff Stuart
5c0f2c8ae6 Add putObj and putMany to apiBlockstore 2022-06-09 15:13:42 -04:00
Aayush
789f212f6c refactor: remove old BlockSyncProtocolID 2022-06-08 13:38:56 -04:00
Aayush
b28c11a57d Merge branch 'feat/nv16' 2022-06-03 14:01:49 -04:00
Aayush
91e69644c2 Merge branch 'feat/nv16' into jen/mergev1153to16 2022-06-03 13:18:27 -04:00
Geoff Stuart
782011b572 Cleanup LoadBundle 2022-06-01 22:22:52 -04:00
Jennifer Wang
2e59d0129d Merge branch 'release/v1.15.3' into jen/mergev1153to16 2022-05-31 16:33:18 -04:00
vyzo
207ff66b3b
update comment.
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
2022-05-31 12:33:08 +03:00
vyzo
bc180c8d78
update comment.
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
2022-05-31 12:32:56 +03:00
vyzo
a1e6410178 remove outdated comment 2022-05-31 12:09:04 +03:00
vyzo
a313dcba74 only enable rcmgr by default in full nodes
opt-in with envvar for other nodes, as there are still some issues
with markets related protocols
2022-05-31 12:03:03 +03:00
Łukasz Magiera
56bde5adf9
Merge pull request #8692 from filecoin-project/feat/health-endpoints
feat: networking: add healthz and livez endpoints
2022-05-30 15:42:06 +02:00
Łukasz Magiera
48aa0131bf fix: rpc: readd rpc.discover aliases 2022-05-27 17:03:03 +02:00
Aayush
4fa04abb8c Add disable flag for Skyr upgrade 2022-05-26 18:20:49 -04:00
Geoff Stuart
30e7f89662 Check for manifest CID while loading bundle 2022-05-26 17:30:32 -04:00
Aayush Rajasekaran
06279b5f1a
Merge pull request #8606 from filecoin-project/asr/fix-drand-round
Fix: drand: calculation of round from Filecoin epochs
2022-05-25 15:36:53 -04:00
Aayush Rajasekaran
6924a3d5f2 Fix calculation of Drand round from Filecoin epochs 2022-05-25 12:43:52 -04:00
vyzo
bb0e3a6cb0 don't clobber test bundles during migration 2022-05-25 19:16:11 +03:00
Aayush Rajasekaran
c2f0bd3c62
Merge pull request #8668 from filecoin-project/asr/fix-post-test
fix wdpost_dispute tests
2022-05-24 14:22:01 -04:00
Aayush
2b847a98ae fix wdpost_dispute tests 2022-05-24 13:58:30 -04:00
Cory Schwartz
d2299dfbf8 backoff/reconnect loop 2022-05-24 10:24:37 -07:00
Steven Allen
496799f867 build: set NetworkBundle in params file
Otherwise, we end up overriding the ldflags.

fixes #8684
2022-05-24 09:42:22 -07:00
Łukasz Magiera
70f3b98574 Fix config doc 2022-05-24 01:33:56 +02:00
Łukasz Magiera
7612860d15 config: Storage.DisallowRemoteFinalize 2022-05-23 23:53:25 +02:00
Łukasz Magiera
16f434c790 config: Plumb Assigner config correctly 2022-05-23 22:02:39 +02:00
Łukasz Magiera
443488b096 lint, docsgen 2022-05-23 22:02:39 +02:00
Łukasz Magiera
5ba8bd3b99 sched: Configurable assigners 2022-05-23 22:02:39 +02:00
Jennifer Wang
5cfedacf9f make jen 2022-05-23 15:11:46 -04:00
Cory Schwartz
7d55ab0734 make atomic 2022-05-23 12:04:13 -07:00
Cory Schwartz
0a7fbd0fce Merge remote-tracking branch 'origin/feat/health-endpoints' into feat/health-endpoints 2022-05-23 11:14:34 -07:00
Cory Schwartz
b4852038a1
Update node/health.go
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2022-05-23 11:11:45 -07:00
Łukasz Magiera
389df95b00 config: Update batching comment in all the places 2022-05-23 18:49:01 +02:00
Cory Schwartz
45d15cb8ee handle error during liveness check 2022-05-23 09:29:11 -07:00
Cory Schwartz
444d0b1b8a add healthz and livez endpoints 2022-05-20 18:38:17 -07:00
vyzo
91daae2e36 add the network name to the bundle key in the datastore 2022-05-18 19:14:11 +03:00
Aayush
8cca9b1970 Use new go-state-types accessors 2022-05-17 15:21:27 -04:00
vyzo
93417fca7d
Merge pull request #8660 from filecoin-project/fix/issue-8659
[nv16] use network bundle names consistent with the network name
2022-05-17 22:13:02 +03:00
vyzo
31c8460116 fix typo 2022-05-17 21:47:01 +03:00
vyzo
ccb2e44e36 make path/url be a map of network name to car uri 2022-05-17 21:45:17 +03:00
vyzo
d949b6c8ef fix envvar handling 2022-05-17 21:14:49 +03:00
vyzo
daf452a180 flexible bundle loading 2022-05-17 20:38:32 +03:00
vyzo
314b417446 use network bundle names consistent with the network name 2022-05-16 22:02:37 +03:00
vyzo
eee73eed59 use switch for release/path logic 2022-05-16 21:45:12 +03:00
vyzo
5eccdaaab7
fix typo
Co-authored-by: raulk <raul@protocol.ai>
2022-05-16 21:38:43 +03:00
vyzo
221e7952ab
fix typo
Co-authored-by: raulk <raul@protocol.ai>
2022-05-16 21:38:09 +03:00
vyzo
46a85fc11e niceties for development bundle loading 2022-05-16 21:09:09 +03:00
vyzo
b4be759b2c don't store dev bundle release keys in the datastore
This is a nicety for development, so that we always load a development bundle to avoid having
to give them distinct names etc. Just call you release "dev" or "dev.xxx..." and put the bundle
in `.lotus/builtin-actors/v8/dev/builting-actors-${network}.{car,sha256sum}` and it will be
unconditionally loaded.
2022-05-16 15:11:10 +03:00
vyzo
68b280b695 add comments in the DI bundle loader 2022-05-12 22:03:18 +03:00
LexLuthr
7e492f23b1 add dagstore register-shard command 2022-05-12 19:46:53 +05:30
vyzo
7be42d9935 refactor to pass the network bundle name through ldflags from build 2022-05-12 16:42:59 +03:00
vyzo
7b6f109401 add retry logic for bundle fetcher 2022-05-12 10:00:20 +03:00
vyzo
26d07fd987 move bundle fetcher to node/bundle instead of chain/actors 2022-05-12 09:51:08 +03:00
vyzo
da8bda3248 fix typo 2022-05-12 09:47:27 +03:00
vyzo
98730b90a3 fix lint 2022-05-11 22:41:56 +03:00
vyzo
74f300290e dynamic loading of builtin actor bundles 2022-05-11 22:30:07 +03:00
Aayush Rajasekaran
c69b579201 Merge branch 'releases' into asr/merge-releases 2022-05-10 20:02:01 -04:00
Raúl Kripalani
6050d04c7c Merge tag 'v1.15.2' into feat/nv16-merge-v1.15.2 2022-05-10 14:17:29 +01:00
Jakub Sztandera
5fb69281d8
Estimate gas across the upgrade with per message multipliers
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2022-05-09 17:08:13 +02:00
Aayush Rajasekaran
8282484f65 Gas estimation: Overestimate by 2 around the M1 upgrade 2022-05-06 11:50:43 -04:00
Aayush Rajasekaran
26a780a198 Gas estimation: Refactor the special PayCh collect case 2022-05-06 11:42:26 -04:00
simlecode
962e37e893 add StateGetNetworkParams api 2022-05-05 14:16:30 +08:00
Łukasz Magiera
f9a4a400da
Merge pull request #8557 from filecoin-project/feat/worker-commp
sealing: DataCid on workers
2022-04-28 17:45:32 +02:00
Łukasz Magiera
d7c52f8adf sealing: ComputeDataCid miner API 2022-04-26 21:37:48 +02:00
vyzo
30ed272aa9 fetch testing-fwp bundle
use testing-fwp bundle when InsecurePostValidation is true

testing-fwp is testing-fake-proofs
2022-04-26 18:38:52 +03:00
Travis Person
ea4f599233 Make MarketListDeals and DealsList return points for MarketDeal 2022-04-22 23:26:29 +00:00
Masih H. Derkani
d044e30e12 Infer index provider topic from network name by default
Index provider integration uses a gossipsub topic to announce changes to
the advertised content. The topic name was fixed to the default topic
which is `/indexer/ingest/mainnet`.

In the case of lotus, the gossipsub validators enforce a list of topics
the instance is permitted to join by setting subscription filter option
when `PubSub` instance is constructed via DI.

Having the fixed topic name meant that any SP starting up a node on a
network other than `mainnet` would have to override the default config
to avoid the node crashing when index provider is enabled.

Instead of a fixed default, the changes here infer the allowed indexer
topic name from network name automatically if the topic configuration is
left empty.

Fixes #8510
2022-04-22 02:46:05 +02:00
Masih H. Derkani
f273a440f0
Infer index provider topic from network name by default
Index provider integration uses a gossipsub topic to announce changes to
the advertised content. The topic name was fixed to the default topic
which is `/indexer/ingest/mainnet`.

In the case of lotus, the gossipsub validators enforce a list of topics
the instance is permitted to join by setting subscription filter option
when `PubSub` instance is constructed via DI.

Having the fixed topic name meant that any SP starting up a node on a
network other than `mainnet` would have to override the default config
to avoid the node crashing when index provider is enabled.

Instead of a fixed default, the changes here infer the allowed indexer
topic name from network name automatically if the topic configuration is
left empty.

Fixes #8510
2022-04-21 14:00:53 +01:00
vyzo
adaa02fd26 load testing bundles for tests 2022-04-19 17:33:20 +03:00
Geoff Stuart
49d6c07280 Implemented StateLoockupRobustAddress 2022-04-14 17:41:18 -04:00
vyzo
6bd2d89e30 use memory blockstore for miner manifest loading 2022-04-14 19:34:26 +03:00
vyzo
507b835a9c need a blockstore for the manifest loader 2022-04-14 19:34:26 +03:00
vyzo
395c772222 make the miner load the manifest so that it can correctly map actors 2022-04-14 19:34:26 +03:00
vyzo
93f1fac76b simplify DI -- no need for the special key for builtin actor loading 2022-04-14 19:33:24 +03:00
Aayush
339b7db2b7 Integrate FIP 0027 2022-04-14 19:32:56 +03:00
vyzo
36cb76a322 genesis depends on actor bundle 2022-04-14 19:30:47 +03:00
vyzo
5e3112fa95 introduce a marker type for builtin actor loading to make DI work 2022-04-14 19:29:22 +03:00
vyzo
f26d19ed7d load actor bundle at startup and initialize the manifest CID from the loaded file 2022-04-14 19:29:22 +03:00
Icarus9913
98c861e3c9 fix the IndexProviderConfig comment 2022-04-13 17:44:06 +08:00
Łukasz Magiera
cc7055774d chore: config: default-disable kvlog 2022-04-12 23:17:13 +02:00
Jiaying Wang
ce3c6472fb
Merge pull request #8470 from filecoin-project/feat/enable-rcmgr-by-default
enable rcmgr by default
2022-04-12 16:12:03 +02:00
Łukasz Magiera
cf92776c64
Merge pull request #8468 from filecoin-project/jen/mergeback
chore: merge releases back to master
2022-04-12 15:41:36 +02:00
vyzo
3c8aa18f70 enabled rcmgr by default; disable with LOTUS_RCMGR=0 2022-04-12 11:32:37 +03:00
Aayush Rajasekaran
656a33863f Fixup merge 2022-04-11 20:45:13 -04:00
jennijuju
0dcad9cc18 Merge branch 'releases' into jen/mergeback 2022-04-12 01:25:02 +02:00
Łukasz Magiera
6bef1aeb82 feat: cli: lotus client list-asks --protocols 2022-04-11 19:49:52 +02:00
Masih H. Derkani
0a2429b467 Fix DI of PubSub in markets by binding dependencies it needs
Bind drand and bootstrap peers config so that `PubSub` instantiated by
`ConfigCommon` has all the dependencies it needs when `PubSub` instance
is needed in markets. In ths case, the instance is needed by the index
provider engine to announce new indexing advertisements.
2022-04-07 11:06:29 -04:00
Masih H. Derkani
e37c54f38e Reuse the market process PubSub instance in index provider engine
The markets process instantiates its own `PubSub` instance with all
validators, peer scoring, etc. set up. Use that instane to join the
indexing topic, otherwise the default topic instantiated by
index-provider internally (via go-legs) has no validators.
2022-04-07 11:06:23 -04:00
Masih H. Derkani
4043923e56 Set index provider options based on lotus config
Always set index provider options based on the lotus configuration.
Otherwise, they will have no effect.
2022-04-07 11:05:09 -04:00