lotus

Author	SHA1	Message	Date
Łukasz Magiera	cb2b4fa4aa	Merge remote-tracking branch 'origin/master' into feat/storage-redeclare	2022-08-05 21:26:15 +02:00
Łukasz Magiera	819314de46	Merge remote-tracking branch 'origin/master' into feat/storage-redeclare	2022-08-05 09:01:30 +02:00
LexLuthr	a59d73d64a	Merge branch 'master' into feat/SchedRemoveRequest	2022-08-04 02:24:14 +05:30
LexLuthr	cdc08e566f	fix everything pointed as per review	2022-08-04 02:00:24 +05:30
LexLuthr	c736dedfa6	move requestremove inside runSched	2022-08-04 02:00:24 +05:30
LexLuthr	cf78fa99ee	Rebase on master	2022-08-04 01:59:48 +05:30
Łukasz Magiera	da33d82e1b	feat: sealer: Custom worker name config	2022-08-03 12:58:49 +02:00
LexLuthr	f5a7a650e7	rename to ActorWithdrawBalance	2022-08-02 13:04:54 +05:30
LexLuthr	e0e0b0b62f	don't wait for msg to land on chain	2022-08-01 23:41:24 +05:30
Łukasz Magiera	3309e10b0d	itests: Test storage detach/redeclare	2022-08-01 15:58:06 +02:00
Łukasz Magiera	8cff52aef6	Storage detach/attach in lotus-miner, cli commands	2022-08-01 15:58:06 +02:00
LexLuthr	e9705c4e26	remove comment pointed out in review	2022-07-30 17:16:50 +05:30
LexLuthr	498334e860	fix spelling mistakes	2022-07-29 20:18:10 +05:30
LexLuthr	4f03486011	miner withdrawbalance API	2022-07-29 20:09:49 +05:30
Aayush Rajasekaran	57e5aa667f	Merge pull request #8991 from filecoin-project/fix/estimate-with-zero-base-fee fix: gas: estimate gas with a zero base-fee	2022-07-13 11:57:22 -04:00
Jennifer Wang	8486ae0d51	Merge branch 'releases' into jen/backport	2022-07-11 14:27:26 -04:00
Łukasz Magiera	d9311b200c	Merge pull request #8978 from filecoin-project/fix/marketnode-storageauth fix: Delegate storage auth on market nodes	2022-07-11 12:23:47 +02:00
Steven Allen	d192b821a9	fix: gas: estimate gas with a zero base-fee Otherwise, an account will need funds to estimate the max possible gas a message could take (which is usually the block gas limit). This does mean gas estimation no longer checks if the sending account has enough funds to cover the message cost, but MpoolPush will now do this.	2022-07-08 09:47:45 -07:00
Marten Seemann	a825682bf9	chore: stop using the deprecated github.com/libp2p/go-libp2p-yamux (#8998 )	2022-07-08 09:44:24 -07:00
Steven Allen	6bed08fe54	chore: update libp2p 1. This pulls in a multistream fix that should fix that fixes a few tests (and probably a few actual bugs). 2. This deduplicates dependencies that have been merged into the monorepo. fixes #8953	2022-07-07 21:38:13 -07:00
Marten Seemann	d52f01a19f	fix incorrect usage of peer.IDFromString (should be peer.Decode)	2022-07-07 21:47:33 +00:00
Aayush	d68a60e8d9	feat: recovery: Config for maximum partition count per message	2022-07-07 14:48:37 -04:00
Łukasz Magiera	673f9238be	feat: wdpost: Config for maximum partition count per message	2022-07-07 14:10:31 -04:00
Aayush	770c32c3cb	feat: recovery: Config for maximum partition count per message	2022-07-07 12:46:33 -04:00
Łukasz Magiera	b499ef0c3a	Merge pull request #8982 from filecoin-project/feat/post-maxpart-config feat: wdpost: Config for maximum partition count per message	2022-07-07 16:48:38 +02:00
Łukasz Magiera	7741175b7d	feat: wdpost: Config for maximum partition count per message	2022-07-07 12:38:21 +02:00
Łukasz Magiera	413183e4fa	fix: Delegate storage auth on market nodes	2022-07-06 22:22:41 +02:00
Dirk McCormick	2d2b06464c	feat: update go-fil-markets to v1.23.0	2022-07-06 11:43:06 -04:00
Aayush Rajasekaran	b35a325816	Fixup typos	2022-07-04 11:27:56 -04:00
Łukasz Magiera	d5100f883d	cfgdoc-gen: Handle empty lines in comments	2022-07-01 22:32:10 +02:00
Łukasz Magiera	59f3161fd6	wdpost: Config for disabling sector prechecks	2022-07-01 22:20:05 +02:00
Łukasz Magiera	84881f64ad	sealer: Config for disabling builtin PoSt	2022-07-01 21:24:54 +02:00
Geoff Stuart	3bb7e9d113	Merge pull request #8941 from filecoin-project/gstuart/actors-cids-older-versions Feat: api: Api call to get actor cids works for versions < 16	2022-06-29 15:31:50 -04:00
Geoff Stuart	4bebce0201	Remove StateActorManifestCID	2022-06-29 15:16:13 -04:00
Geoff Stuart	4268b426f9	review fixes	2022-06-29 14:58:58 -04:00
Geoff Stuart	ed65b4c3bf	Api call to get actor cids works for versions < 16	2022-06-29 14:58:56 -04:00
Masih H. Derkani	e8c44babcf	Remove dependency to archived quic and regenerate CLI docs The quic transport implementation is now moved to `go-libp2p` mono repo. Replace the dependency to the archived repo with the new one. Regenerate CLI docs.	2022-06-29 14:55:35 -04:00
Masih H. Derkani	39a710e1d6	Address lint issues as a result of libp2p deprecations and tidy go mod Address staticcheck issues caused by deprecated APIs and `go mod tidy`.	2022-06-29 14:55:35 -04:00
Jennifer Wang	c3f3eb0812	Merge branch 'releases' into jen/masterbp	2022-06-27 15:13:12 -04:00
Geoff Stuart	83d7db88cf	review fixes	2022-06-23 14:32:14 -04:00
Geoff Stuart	e684248f48	Added api call to get actors cids	2022-06-23 14:07:23 -04:00
Łukasz Magiera	db676acdf8	storage: Cleanup interfaces a bit	2022-06-17 13:52:19 +02:00
Łukasz Magiera	58416d0881	storage: Don't depend on specs-storage, move to storiface	2022-06-17 13:36:33 +02:00
Łukasz Magiera	9c4d10ec73	api: handle no-precommit in StateSectorPreCommitInfo gracefully	2022-06-16 15:20:58 +02:00
Łukasz Magiera	2dc1df878d	api: Move StateComputeDataCID to full-node	2022-06-16 13:29:35 +02:00
Łukasz Magiera	05cdeb80c3	chore: remove redundant import prefixes	2022-06-15 12:06:22 +02:00
Łukasz Magiera	28099a3905	storage: Move storage/sealer/stores to storage/paths	2022-06-14 20:25:52 +02:00
Łukasz Magiera	82857e6d5d	chore: rerun gen with moved packages	2022-06-14 20:13:31 +02:00
Łukasz Magiera	a9600b8a6f	storage: Move extern/sector-storage to storage/sealer	2022-06-14 20:03:38 +02:00
Łukasz Magiera	98a48a47f8	storage: Move extern/storage-sealing to storage/pipeline	2022-06-14 19:41:59 +02:00
Łukasz Magiera	717bda63b1	storage: Move control address logic to storage/ctladdr	2022-06-14 19:32:29 +02:00
Łukasz Magiera	dff1bf2868	storage: Move window-post logic into a separate package	2022-06-14 19:27:09 +02:00
Łukasz Magiera	e65fae28de	chore: fix imports	2022-06-14 17:00:51 +02:00
Steven Allen	ddc9425c07	feat: refactor: actor bundling system (#8838 ) 1. Include the builtin-actors in the lotus source tree. 2. Embed the bundle on build instead of downloading at runtime. 3. Avoid reading the bundle whenever possible by including bundle metadata (the bundle CID, the actor CIDs, etc.). 4. Remove everything related to dependency injection. 1. We're no longer downloading the bundle, so doing anything ahead of time doesn't really help. 2. We register the manifests on init because, unfortunately, they're global. 3. We explicitly load the current actors bundle in the genesis state-tree method. 4. For testing, we just change the in-use bundle with a bit of a hack. It's not great, but using dependency injection doesn't make any sense either because, again, the manifest information is global. 5. Remove the bundle.toml file. Bundles may be overridden by specifying an override path in the parameters file, or an environment variable. fixes #8701	2022-06-13 10:51:49 -07:00
Steven Allen	30981d0fdd	feat: refactor: actor bundling system (#8838 ) 1. Include the builtin-actors in the lotus source tree. 2. Embed the bundle on build instead of downloading at runtime. 3. Avoid reading the bundle whenever possible by including bundle metadata (the bundle CID, the actor CIDs, etc.). 4. Remove everything related to dependency injection. 1. We're no longer downloading the bundle, so doing anything ahead of time doesn't really help. 2. We register the manifests on init because, unfortunately, they're global. 3. We explicitly load the current actors bundle in the genesis state-tree method. 4. For testing, we just change the in-use bundle with a bit of a hack. It's not great, but using dependency injection doesn't make any sense either because, again, the manifest information is global. 5. Remove the bundle.toml file. Bundles may be overridden by specifying an override path in the parameters file, or an environment variable. fixes #8701	2022-06-13 10:15:00 -07:00
Łukasz Magiera	1cd94f598d	Merge pull request #8820 from filecoin-project/asr/deprecate-blocksync refactor: remove old BlockSyncProtocolID	2022-06-13 18:36:21 +02:00
vyzo	d4d5ce3021	only enable rcmgr by default in full nodes opt-in with envvar for other nodes, as there are still some issues with markets related protocols	2022-06-13 12:16:20 -04:00
Geoff Stuart	b7010c9e60	Implement function to migrate actors with only code changes	2022-06-10 15:52:32 -04:00
Geoff Stuart	801c670edd	Remove ChainPutMany	2022-06-10 14:09:05 -04:00
Geoff Stuart	5c0f2c8ae6	Add putObj and putMany to apiBlockstore	2022-06-09 15:13:42 -04:00
Aayush	789f212f6c	refactor: remove old BlockSyncProtocolID	2022-06-08 13:38:56 -04:00
Aayush	b28c11a57d	Merge branch 'feat/nv16'	2022-06-03 14:01:49 -04:00
Aayush	91e69644c2	Merge branch 'feat/nv16' into jen/mergev1153to16	2022-06-03 13:18:27 -04:00
Geoff Stuart	782011b572	Cleanup LoadBundle	2022-06-01 22:22:52 -04:00
Jennifer Wang	2e59d0129d	Merge branch 'release/v1.15.3' into jen/mergev1153to16	2022-05-31 16:33:18 -04:00
vyzo	207ff66b3b	update comment. Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>	2022-05-31 12:33:08 +03:00
vyzo	bc180c8d78	update comment. Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>	2022-05-31 12:32:56 +03:00
vyzo	a1e6410178	remove outdated comment	2022-05-31 12:09:04 +03:00
vyzo	a313dcba74	only enable rcmgr by default in full nodes opt-in with envvar for other nodes, as there are still some issues with markets related protocols	2022-05-31 12:03:03 +03:00
Łukasz Magiera	56bde5adf9	Merge pull request #8692 from filecoin-project/feat/health-endpoints feat: networking: add healthz and livez endpoints	2022-05-30 15:42:06 +02:00
Łukasz Magiera	48aa0131bf	fix: rpc: readd rpc.discover aliases	2022-05-27 17:03:03 +02:00
Aayush	4fa04abb8c	Add disable flag for Skyr upgrade	2022-05-26 18:20:49 -04:00
Geoff Stuart	30e7f89662	Check for manifest CID while loading bundle	2022-05-26 17:30:32 -04:00
Aayush Rajasekaran	06279b5f1a	Merge pull request #8606 from filecoin-project/asr/fix-drand-round Fix: drand: calculation of round from Filecoin epochs	2022-05-25 15:36:53 -04:00
Aayush Rajasekaran	6924a3d5f2	Fix calculation of Drand round from Filecoin epochs	2022-05-25 12:43:52 -04:00
vyzo	bb0e3a6cb0	don't clobber test bundles during migration	2022-05-25 19:16:11 +03:00
Aayush Rajasekaran	c2f0bd3c62	Merge pull request #8668 from filecoin-project/asr/fix-post-test fix wdpost_dispute tests	2022-05-24 14:22:01 -04:00
Aayush	2b847a98ae	fix wdpost_dispute tests	2022-05-24 13:58:30 -04:00
Cory Schwartz	d2299dfbf8	backoff/reconnect loop	2022-05-24 10:24:37 -07:00
Steven Allen	496799f867	build: set NetworkBundle in params file Otherwise, we end up overriding the ldflags. fixes #8684	2022-05-24 09:42:22 -07:00
Łukasz Magiera	70f3b98574	Fix config doc	2022-05-24 01:33:56 +02:00
Łukasz Magiera	7612860d15	config: Storage.DisallowRemoteFinalize	2022-05-23 23:53:25 +02:00
Łukasz Magiera	16f434c790	config: Plumb Assigner config correctly	2022-05-23 22:02:39 +02:00
Łukasz Magiera	443488b096	lint, docsgen	2022-05-23 22:02:39 +02:00
Łukasz Magiera	5ba8bd3b99	sched: Configurable assigners	2022-05-23 22:02:39 +02:00
Jennifer Wang	5cfedacf9f	make jen	2022-05-23 15:11:46 -04:00
Cory Schwartz	7d55ab0734	make atomic	2022-05-23 12:04:13 -07:00
Cory Schwartz	0a7fbd0fce	Merge remote-tracking branch 'origin/feat/health-endpoints' into feat/health-endpoints	2022-05-23 11:14:34 -07:00
Cory Schwartz	b4852038a1	Update node/health.go Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>	2022-05-23 11:11:45 -07:00
Łukasz Magiera	389df95b00	config: Update batching comment in all the places	2022-05-23 18:49:01 +02:00
Cory Schwartz	45d15cb8ee	handle error during liveness check	2022-05-23 09:29:11 -07:00
Cory Schwartz	444d0b1b8a	add healthz and livez endpoints	2022-05-20 18:38:17 -07:00
vyzo	91daae2e36	add the network name to the bundle key in the datastore	2022-05-18 19:14:11 +03:00
Aayush	8cca9b1970	Use new go-state-types accessors	2022-05-17 15:21:27 -04:00
vyzo	93417fca7d	Merge pull request #8660 from filecoin-project/fix/issue-8659 [nv16] use network bundle names consistent with the network name	2022-05-17 22:13:02 +03:00
vyzo	31c8460116	fix typo	2022-05-17 21:47:01 +03:00
vyzo	ccb2e44e36	make path/url be a map of network name to car uri	2022-05-17 21:45:17 +03:00
vyzo	d949b6c8ef	fix envvar handling	2022-05-17 21:14:49 +03:00
vyzo	daf452a180	flexible bundle loading	2022-05-17 20:38:32 +03:00
vyzo	314b417446	use network bundle names consistent with the network name	2022-05-16 22:02:37 +03:00
vyzo	eee73eed59	use switch for release/path logic	2022-05-16 21:45:12 +03:00
vyzo	5eccdaaab7	fix typo Co-authored-by: raulk <raul@protocol.ai>	2022-05-16 21:38:43 +03:00
vyzo	221e7952ab	fix typo Co-authored-by: raulk <raul@protocol.ai>	2022-05-16 21:38:09 +03:00
vyzo	46a85fc11e	niceties for development bundle loading	2022-05-16 21:09:09 +03:00
vyzo	b4be759b2c	don't store dev bundle release keys in the datastore This is a nicety for development, so that we always load a development bundle to avoid having to give them distinct names etc. Just call you release "dev" or "dev.xxx..." and put the bundle in `.lotus/builtin-actors/v8/dev/builting-actors-${network}.{car,sha256sum}` and it will be unconditionally loaded.	2022-05-16 15:11:10 +03:00
vyzo	68b280b695	add comments in the DI bundle loader	2022-05-12 22:03:18 +03:00
LexLuthr	7e492f23b1	add dagstore register-shard command	2022-05-12 19:46:53 +05:30
vyzo	7be42d9935	refactor to pass the network bundle name through ldflags from build	2022-05-12 16:42:59 +03:00
vyzo	7b6f109401	add retry logic for bundle fetcher	2022-05-12 10:00:20 +03:00
vyzo	26d07fd987	move bundle fetcher to node/bundle instead of chain/actors	2022-05-12 09:51:08 +03:00
vyzo	da8bda3248	fix typo	2022-05-12 09:47:27 +03:00
vyzo	98730b90a3	fix lint	2022-05-11 22:41:56 +03:00
vyzo	74f300290e	dynamic loading of builtin actor bundles	2022-05-11 22:30:07 +03:00
Aayush Rajasekaran	c69b579201	Merge branch 'releases' into asr/merge-releases	2022-05-10 20:02:01 -04:00
Raúl Kripalani	6050d04c7c	Merge tag 'v1.15.2' into feat/nv16-merge-v1.15.2	2022-05-10 14:17:29 +01:00
Jakub Sztandera	5fb69281d8	Estimate gas across the upgrade with per message multipliers Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2022-05-09 17:08:13 +02:00
Aayush Rajasekaran	8282484f65	Gas estimation: Overestimate by 2 around the M1 upgrade	2022-05-06 11:50:43 -04:00
Aayush Rajasekaran	26a780a198	Gas estimation: Refactor the special PayCh collect case	2022-05-06 11:42:26 -04:00
simlecode	962e37e893	add StateGetNetworkParams api	2022-05-05 14:16:30 +08:00
Łukasz Magiera	f9a4a400da	Merge pull request #8557 from filecoin-project/feat/worker-commp sealing: DataCid on workers	2022-04-28 17:45:32 +02:00
Łukasz Magiera	d7c52f8adf	sealing: ComputeDataCid miner API	2022-04-26 21:37:48 +02:00
vyzo	30ed272aa9	fetch testing-fwp bundle use testing-fwp bundle when InsecurePostValidation is true testing-fwp is testing-fake-proofs	2022-04-26 18:38:52 +03:00
Travis Person	ea4f599233	Make MarketListDeals and DealsList return points for MarketDeal	2022-04-22 23:26:29 +00:00
Masih H. Derkani	d044e30e12	Infer index provider topic from network name by default Index provider integration uses a gossipsub topic to announce changes to the advertised content. The topic name was fixed to the default topic which is `/indexer/ingest/mainnet`. In the case of lotus, the gossipsub validators enforce a list of topics the instance is permitted to join by setting subscription filter option when `PubSub` instance is constructed via DI. Having the fixed topic name meant that any SP starting up a node on a network other than `mainnet` would have to override the default config to avoid the node crashing when index provider is enabled. Instead of a fixed default, the changes here infer the allowed indexer topic name from network name automatically if the topic configuration is left empty. Fixes #8510	2022-04-22 02:46:05 +02:00
Masih H. Derkani	f273a440f0	Infer index provider topic from network name by default Index provider integration uses a gossipsub topic to announce changes to the advertised content. The topic name was fixed to the default topic which is `/indexer/ingest/mainnet`. In the case of lotus, the gossipsub validators enforce a list of topics the instance is permitted to join by setting subscription filter option when `PubSub` instance is constructed via DI. Having the fixed topic name meant that any SP starting up a node on a network other than `mainnet` would have to override the default config to avoid the node crashing when index provider is enabled. Instead of a fixed default, the changes here infer the allowed indexer topic name from network name automatically if the topic configuration is left empty. Fixes #8510	2022-04-21 14:00:53 +01:00
vyzo	adaa02fd26	load testing bundles for tests	2022-04-19 17:33:20 +03:00
Geoff Stuart	49d6c07280	Implemented StateLoockupRobustAddress	2022-04-14 17:41:18 -04:00
vyzo	6bd2d89e30	use memory blockstore for miner manifest loading	2022-04-14 19:34:26 +03:00
vyzo	507b835a9c	need a blockstore for the manifest loader	2022-04-14 19:34:26 +03:00
vyzo	395c772222	make the miner load the manifest so that it can correctly map actors	2022-04-14 19:34:26 +03:00
vyzo	93f1fac76b	simplify DI -- no need for the special key for builtin actor loading	2022-04-14 19:33:24 +03:00
Aayush	339b7db2b7	Integrate FIP 0027	2022-04-14 19:32:56 +03:00
vyzo	36cb76a322	genesis depends on actor bundle	2022-04-14 19:30:47 +03:00
vyzo	5e3112fa95	introduce a marker type for builtin actor loading to make DI work	2022-04-14 19:29:22 +03:00
vyzo	f26d19ed7d	load actor bundle at startup and initialize the manifest CID from the loaded file	2022-04-14 19:29:22 +03:00
Icarus9913	98c861e3c9	fix the IndexProviderConfig comment	2022-04-13 17:44:06 +08:00
Łukasz Magiera	cc7055774d	chore: config: default-disable kvlog	2022-04-12 23:17:13 +02:00
Jiaying Wang	ce3c6472fb	Merge pull request #8470 from filecoin-project/feat/enable-rcmgr-by-default enable rcmgr by default	2022-04-12 16:12:03 +02:00
Łukasz Magiera	cf92776c64	Merge pull request #8468 from filecoin-project/jen/mergeback chore: merge releases back to master	2022-04-12 15:41:36 +02:00
vyzo	3c8aa18f70	enabled rcmgr by default; disable with LOTUS_RCMGR=0	2022-04-12 11:32:37 +03:00
Aayush Rajasekaran	656a33863f	Fixup merge	2022-04-11 20:45:13 -04:00
jennijuju	0dcad9cc18	Merge branch 'releases' into jen/mergeback	2022-04-12 01:25:02 +02:00
Łukasz Magiera	6bef1aeb82	feat: cli: lotus client list-asks --protocols	2022-04-11 19:49:52 +02:00
Masih H. Derkani	0a2429b467	Fix DI of `PubSub` in markets by binding dependencies it needs Bind drand and bootstrap peers config so that `PubSub` instantiated by `ConfigCommon` has all the dependencies it needs when `PubSub` instance is needed in markets. In ths case, the instance is needed by the index provider engine to announce new indexing advertisements.	2022-04-07 11:06:29 -04:00
Masih H. Derkani	e37c54f38e	Reuse the market process `PubSub` instance in index provider engine The markets process instantiates its own `PubSub` instance with all validators, peer scoring, etc. set up. Use that instane to join the indexing topic, otherwise the default topic instantiated by index-provider internally (via go-legs) has no validators.	2022-04-07 11:06:23 -04:00
Masih H. Derkani	4043923e56	Set index provider options based on lotus config Always set index provider options based on the lotus configuration. Otherwise, they will have no effect.	2022-04-07 11:05:09 -04:00
Masih H. Derkani	f369d99138	Fix DI of `PubSub` in markets by binding dependencies it needs Bind drand and bootstrap peers config so that `PubSub` instantiated by `ConfigCommon` has all the dependencies it needs when `PubSub` instance is needed in markets. In ths case, the instance is needed by the index provider engine to announce new indexing advertisements.	2022-04-06 20:04:39 +01:00
Masih H. Derkani	7e7e88e330	Reuse the market process `PubSub` instance in index provider engine The markets process instantiates its own `PubSub` instance with all validators, peer scoring, etc. set up. Use that instane to join the indexing topic, otherwise the default topic instantiated by index-provider internally (via go-legs) has no validators.	2022-04-06 19:39:12 +01:00
Masih H. Derkani	b88307f875	Set index provider options based on lotus config Always set index provider options based on the lotus configuration. Otherwise, they will have no effect.	2022-04-06 16:08:10 +01:00
Łukasz Magiera	2b3fad1261	fix: node: Fix market node startup	2022-04-01 17:32:55 -04:00
Łukasz Magiera	d95f24b9d2	config: Sane default parallel sector read defaults	2022-03-29 14:52:37 -04:00
Łukasz Magiera	ebd34f1884	feat: storage: Parallel proving checks	2022-03-29 11:45:08 -04:00
Łukasz Magiera	54cb55a7ae	feat: miner: API/CLI to compute window-post	2022-03-28 17:06:51 -04:00
Łukasz Magiera	c951e348b4	feat: config: Move MakeNewSectorForDeals config into the Sealing sectoin	2022-03-26 23:45:48 -04:00
Łukasz Magiera	b78892934b	feat: config: Move MakeNewSectorForDeals config into the Sealing sectoin	2022-03-26 15:50:21 -04:00
Łukasz Magiera	82343a06e7	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-03-25 16:49:46 -04:00
Łukasz Magiera	7c4d3a457c	Merge pull request #8343 from filecoin-project/feat/more-snap-config feat: sealing: More SnapDeals config knobs	2022-03-25 15:07:00 -04:00
Łukasz Magiera	945b93590d	ping: Match go-ipfs in cmd output	2022-03-21 15:11:19 +01:00
Łukasz Magiera	701d0a111e	Merge branch 'feat/net-ping' of github.com:ychiaoli18/lotus into feat/net-ping	2022-03-21 13:06:11 +01:00
Łukasz Magiera	92e4caa151	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-03-21 12:09:47 +01:00
Łukasz Magiera	433e9c351d	config: Fix doc on PreferNewSectorsForDeals; gofmt	2022-03-21 12:07:00 +01:00
Łukasz Magiera	97a73f1c29	sealing: More SnapDeals config knobs	2022-03-21 12:04:23 +01:00
Łukasz Magiera	fc34d9ba1c	Merge pull request #8086 from filecoin-project/nonsense/refactor-nodetype refactor: convert RepoType from int to interface	2022-03-21 11:57:39 +01:00
Anton Evangelatov	0ed8647b6f	extract unixfs filestore into lib	2022-03-21 10:48:52 +01:00
Łukasz Magiera	5365ccfdb1	cli: Show separate worker types in miner info	2022-03-18 21:31:15 +01:00
Łukasz Magiera	7b81369c8c	wdpost: Don't attempt to snark with skipped sectors	2022-03-18 19:45:25 +01:00
Anton Evangelatov	0d6493ec3f	resolve merge conflicts with master	2022-03-18 12:28:22 +01:00
Łukasz Magiera	6ddbe41376	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-03-18 10:54:44 +01:00
Aayush Rajasekaran	2ed2ee1da7	Merge branch 'release/v1.15.1' into feat/fvm	2022-03-17 11:31:56 -04:00
Łukasz Magiera	037e80c70c	gofmt	2022-03-17 12:10:24 +01:00
Łukasz Magiera	1c055fe83b	Merge branch 'master' into test/t.TempDir	2022-03-17 12:06:52 +01:00
Łukasz Magiera	d3cec2f0d1	sealing: Add MakeCCSectorsAvailable config	2022-03-16 22:29:40 +01:00
Łukasz Magiera	a3bdd29615	Merge pull request #8330 from filecoin-project/feat/snap-queue feat: sealing: Sector upgrade queue	2022-03-16 22:24:45 +01:00
Aayush Rajasekaran	90d78f924f	Merge pull request #8293 from filecoin-project/feat/fvm Filecoin Virtual Machine integration	2022-03-16 15:51:56 -04:00
Łukasz Magiera	4ded059a1c	sealing: Add MakeCCSectorsAvailable config	2022-03-16 19:29:47 +01:00
Darko Brdareski	79453663b3	feat: Add additional test annotations (#8272 ) * Annotate api,proxy_util,blockstore_badger, policy tests * Annotate splitstore: bsbadger / markset * Annotate splitstore feature * Annotate union/timed blockstore tests * Annotate openrpc, diff_adt tests * Annotate error,drand,events tests * Annotate predicates_test * Fix annotations * Annotate tscache, gen tests * Annotate fundmanager test * Annotate repub and selection tests * Annotate statetree_test * Annotate forks_test * Annotate searchwait_test.go * Fix duplicated @@ symbols * Annotate chain stmgr/store tests * Annotate more (types) tests * More tests annotated * Annotate conformance chaos actor tests * Annotate more integration tests * Annotate journal system tests * Annotate more tests. * Annotate gas,head buffer behaviors * Fix markset annotations * doc: test annotations for the markets dagstore wrapper * Annotate miner_api test in dagstore * Annotate more test files * Remove bad annotations from fsrepo * Annotate wdpost system * Remove bad annotations * Renamce "conformance" to "chaos_actor" tests * doc: stm annotations for blockheader & election proof tests * Annotate remaining "A" tests * annotate: stm for error_test * memrepo_test.go * Annotate "b" file tests * message_test.go * doc: stm annotate for fsrepo_test * Annotate "c" file tests * Annotate "D" test files * message_test.go * doc: stm annotate for chain, node/config & client * docs: stm annotate node_test * Annotate u,v,wl tests * doc: stm annotations for various test files * Annotate "T" test files * doc: stm annotate for proxy_util_test & policy_test * doc: stm annotate for various tests * doc: final few stm annotations * Add mempool unit tests * Add two more memPool Add tests * Update submodules * Add check function tests * Add stm annotations, refactor test helper * Annotate api,proxy_util,blockstore_badger, policy tests * Annotate splitstore: bsbadger / markset solving merge conflicts * Annotate splitstore feature * Annotate union/timed blockstore tests * Annotate openrpc, diff_adt tests * Annotate error,drand,events tests * Annotate predicates_test * Fix annotations * Annotate tscache, gen tests * Annotate fundmanager test * Annotate statetree_test * Annotate forks_test * Annotate searchwait_test.go * Fix duplicated @@ symbols * Annotate chain stmgr/store tests * Annotate more (types) tests * More tests annotated * Annotate conformance chaos actor tests * Annotate more integration tests * Annotate journal system tests * Annotate more tests. * Annotate gas,head buffer behaviors solve merge conflict * Fix markset annotations * Annotate miner_api test in dagstore * Annotate more test files * doc: test annotations for the markets dagstore wrapper * Annotate wdpost system * Renamce "conformance" to "chaos_actor" tests * Annotate remaining "A" tests * doc: stm annotations for blockheader & election proof tests * annotate: stm for error_test * Annotate "b" file tests * memrepo_test.go * Annotate "c" file tests * message_test.go * Annotate "D" test files * doc: stm annotate for fsrepo_test * Annotate u,v,wl tests * message_test.go * doc: stm annotate for chain, node/config & client * docs: stm annotate node_test * Annotate "T" test files * doc: stm annotations for various test files * Add mempool unit tests solve merge conflict * doc: stm annotate for proxy_util_test & policy_test * doc: stm annotate for various tests * doc: final few stm annotations * Add two more memPool Add tests * Update submodules * Add check function tests solve conflict * Add stm annotations, refactor test helper solve merge conflict * Change CLI test kinds to "unit" * Fix double merged test * Fix ccupgrade_test merge * Fix lint issues * Add stm annotation to types_Test * Test vectors submodule * Add file annotation to burn_test Co-authored-by: Nikola Divic <divicnikola@gmail.com> Co-authored-by: TheMenko <themenkoprojects@gmail.com>	2022-03-16 18:37:34 +01:00
Aayush	16128a0212	Merge branch 'master' into feat/fvm	2022-03-16 11:57:46 -04:00
Aayush	9ea623e125	Rename the Lotus VM to LegacyVM	2022-03-15 19:40:17 -04:00
Jiaying Wang	ada499ef14	Merge pull request #8314 from filecoin-project/jen/indexsetting feat: config: enable indexer providing by default	2022-03-15 19:33:49 -04:00
Jiaying Wang	c98ef0aaf8	Merge pull request #8317 from filecoin-project/jen/backport chore: backport: release to master	2022-03-15 19:12:20 -04:00
vyzo	9fcafff847	adjust conns to 2x maxconns for duplicate connections	2022-03-15 10:02:27 +02:00
vyzo	53c525f0ed	improve resource manager integration - add opt-in env var to control instantation, until we are comfortable with testing to enble by default. - adjust default limits if the connection manager high mark is higher than the default inbound conn limit.	2022-03-15 09:43:18 +02:00
Jennifer Wang	8a2a2d065d	Merge branch 'releases' into jen/backport	2022-03-15 02:07:35 -04:00
Jennifer Wang	6240729ee4	enable indexer providing by default	2022-03-14 23:24:52 -04:00
Eng Zer Jun	c229842f57	test: use `T.TempDir` to create temporary test directory The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete. Reference: https://pkg.go.dev/testing#T.TempDir Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	2022-03-15 00:21:31 +08:00
Łukasz Magiera	135aef78d7	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-03-11 17:04:58 +01:00
Łukasz Magiera	81f0cfdd1e	Merge pull request #8276 from filecoin-project/feat/faster-clientcalccommp feat: api: Make ClientCalcCommP multithreaded	2022-03-10 23:22:48 +01:00
Łukasz Magiera	0d9e34d651	client impl: Fix typo in ClientCalcCommP CopyBuffer error	2022-03-10 22:41:45 +01:00
Łukasz Magiera	4421bf2fa1	feat: config: Persistent subsystem log level config	2022-03-10 14:44:56 +01:00
Łukasz Magiera	9c4f8254ee	post workers: Fix build post-merge	2022-03-10 14:20:07 +01:00
Łukasz Magiera	a715fb3063	feat: Make ClientCalcCommP multithreaded	2022-03-09 23:25:29 +01:00
hannahhoward	49742f8fdc	feat(deps): update to graphsync v0.13.0 with 2.0 protocol	2022-03-09 18:06:35 +00:00
Łukasz Magiera	046a9f8af0	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-03-09 16:27:03 +01:00
Anton Evangelatov	7cdf6dc680	remove .Type() now that we have singletons for repos	2022-03-07 13:00:26 +01:00
Anton Evangelatov	b9fe61f9da	Merge branch 'master' into nonsense/refactor-nodetype	2022-03-07 11:51:28 +01:00
Anton Evangelatov	5734a39e8e	move DealmakingConfiger and SealingConfiger away from config/types.go	2022-03-03 16:51:47 +01:00
Anton Evangelatov	62e352ef88	move APIInfoEnvVars, RepoFlags, APIFlags under RepoType	2022-03-03 16:45:11 +01:00
Anton Evangelatov	acbfc879aa	config: extract DealmakingConfig and Sealing config from Lotus for Boost (#8172 ) * refactor sealingCfg and dealmakingCfg to be able to abstract Config * fix error messages	2022-03-03 15:37:23 +01:00
Anton Evangelatov	066d0edf51	add SupportsStagingDeals trait	2022-03-03 15:37:23 +01:00
Anton Evangelatov	949ec65118	drop defConfForType	2022-03-03 15:37:23 +01:00
Anton Evangelatov	f1f878a000	unexport repo types; use a global var for every repo type	2022-03-03 15:37:23 +01:00
Anton Evangelatov	e3edab66e3	refactor RepoType	2022-03-03 15:37:23 +01:00
Łukasz Magiera	7dcfeee0ea	use old go-path	2022-03-03 14:17:25 +00:00
Masih H. Derkani	1bb65189af	Upgrade to go-path `v0.2.1` and remove `replace` directive The dependency was used in the chain API. Integrated the new fetcher factory.	2022-03-03 13:29:32 +00:00
Aayush	85886287b4	Revert "use the libp2p connmgr" This reverts commit `826cdb2186`.	2022-03-02 15:39:23 -05:00
Aayush	b5912d1543	Revert "instantiate resource manager in DI" This reverts commit `8d3f98fe38`.	2022-03-02 15:39:22 -05:00
Aayush	e9e28f75e2	Revert "add resource manager NetStat api" This reverts commit `d05d5bcb70`.	2022-03-02 15:39:22 -05:00
Aayush	b8747e3afc	Revert "add resource manager Net(Set)Limit api" This reverts commit `554848fe6b`.	2022-03-02 15:39:21 -05:00
Aayush	48e28898a6	Revert "fix lint" This reverts commit `325a4c0e5b`.	2022-03-02 15:29:27 -05:00
Aayush	17ac583508	Revert "close the rcmgr on shutdown" This reverts commit `73ec10a49e`.	2022-03-02 15:29:26 -05:00
Masih H. Derkani	aacc246ba3	Merge branch 'master' into feat/cid-to-piece-idx	2022-03-02 14:06:30 +00:00
Masih H. Derkani	444a2bf355	Integrate the latest index-provider with config to disable announcements Integrate the latest `index-provider` and reflect the changes to engine configuration. Note that this commit disables announcements of indices on the network by default as requested for initial merge to master. Introduce dedicated index provider configuration parameters with documentation and defaults that match the defaults in index-provider. Re-generate code as needed.	2022-03-02 13:45:09 +00:00
Łukasz Magiera	e9a6f5f287	Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse	2022-03-02 12:56:56 +00:00
Masih H. Derkani	769b0f30ef	Address review comments - Add comment to clarify the reason for loop in testkit - Trim common prefix in state printed in CLI commands for better readability - Upgrade to a tagged release of `go-fil-markets` that includes indexing work; see: https://github.com/filecoin-project/go-fil-markets/pull/673 - Fix typo in CLI usage. - Add comments to note that it is safe to use fx `OnStart` context when starting the provider engine. - Fix string concatenation in error message formatting.	2022-03-02 11:04:50 +00:00
zenground0	77a954c7c3	Merge branch 'master' into feat/cid-to-piece-idx	2022-02-28 12:57:11 -07:00
jennijuju	c6361b391c	Merge branch 'releases' into jen/v15	2022-02-25 21:01:31 -05:00
zenground0	e5de237990	Fix fault tracker to handle snap deals	2022-02-24 17:53:21 -05:00
ZenGround0	d711775cdf	Merge pull request #8177 from filecoin-project/fix/fault-tracker-snap-deals fix:proving:post check sector handles snap deals replica faults	2022-02-24 11:30:10 -07:00
Aayush	fd8cb44e67	fix: client: calculate commps for pieces bigger than 32GB	2022-02-24 10:27:37 -05:00
zenground0	abe04c33c1	Fix fault tracker to handle snap deals	2022-02-23 09:56:47 -07:00
Aarsh Shah	e3f5b7c59f	update dagstore top level index db	2022-02-18 14:00:01 +04:00
Kevin Li	ba72eff3e6	feat: cli/net: implement 'net ping' command	2022-02-17 17:22:52 +08:00
Masih H. Derkani	a09f386537	Remove redundant `IndexProvider` config now that host is shared Now that host is shared having separate config for listen and announce addresses along with graphsync limit makes no sense since all of that is shared with the markets host and datatransfer manager.	2022-02-16 13:46:44 +00:00
Masih H. Derkani	2ebc111b70	Use the same host and datatransfer as markets for index provider Remove the bespoke instantiation of libp2p host and datatransfer manager for index provider and reuse the existing instances used by markets. The rationale for reuse is the following: 1. Separation of host introduces a discovery problem, where without gossipsub the index provider endpoint will not be discoverable. Using the same host as markets would mean the chain can be used to discover addresses, putting less empassis on criticality of gossipsub considering its set-up cost and lack of message delivery guarantees. 2. Only a single instance of graphsync/datatransfer can be instantiated per libp2p host; therefore, if the host is shared, so should datatransfer manager. 3. it is not clear if the assumptions under which separation was decided still hold.	2022-02-16 13:46:44 +00:00
Łukasz Magiera	681b90768a	Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse	2022-02-16 10:06:36 +01:00
vyzo	9c00af1b86	don't track peer ids in rcmgr metrics	2022-02-15 16:12:17 +02:00
Łukasz Magiera	48dea93c7e	Merge pull request #8089 from filecoin-project/feat/rcmgr-metrics Resource Manager Metrics	2022-02-15 14:40:44 +01:00
Aayush	75b8e4c981	fix: sealer: allow enable/disabling ReplicaUpdate tasks	2022-02-14 15:25:52 -05:00
Aayush	56df886b55	fix: sealer: allow enable/disabling ReplicaUpdate tasks	2022-02-14 15:24:13 -05:00
Łukasz Magiera	60b9acc09d	gen	2022-02-14 21:16:41 +01:00
Aayush	eebe784191	fix: sealer: allow enable/disabling ReplicaUpdate tasks	2022-02-14 15:08:28 -05:00
Łukasz Magiera	36a1934845	paychmgr: Fix tests after api changes	2022-02-14 20:16:30 +01:00
Łukasz Magiera	df5bd147fa	paychmgr: Review comments Co-authored-by: Hannah Howard <hannah@hannahhoward.net> Co-authored-by: dirkmc <dirkmdev@gmail.com>	2022-02-14 20:07:13 +01:00
Łukasz Magiera	f61eb23f8f	api: separate method for paych funding	2022-02-14 19:56:02 +01:00
Łukasz Magiera	10af768c60	Merge commit 'origin/release/v1.15.0~2' into feat/paych-avail-reuse	2022-02-14 19:27:12 +01:00
vyzo	a9ec408844	collect resource manager metrics	2022-02-14 19:46:05 +02:00
Masih H. Derkani	7e7844f169	Fix log message formatting when instantiating index provider	2022-02-11 12:15:21 +00:00
Will	296eab3045	Merge pull request #8045 from gammazero/feat/cid-to-piece-idx Add indexer pubsub message authentication and rate limiting	2022-02-10 13:49:58 -08:00
gammazero	eedcf910eb	change scoring based on review	2022-02-10 10:53:07 -08:00
gammazero	b72cd90c19	Add indexer message scoring	2022-02-10 09:21:21 -08:00
gammazero	3f3d61b043	Service creation takes interface, not implementation	2022-02-10 08:41:18 -08:00
Jennifer Wang	de1f4792cd	Merge remote-tracking branch 'origin/master' into jen/v15	2022-02-09 19:27:34 -05:00
Łukasz Magiera	44fd0e3349	Merge pull request #8008 from filecoin-project/feat/splitstore-sortless-compaction splitstore sortless compaction	2022-02-09 17:17:57 +00:00
Jennifer Wang	6ed97f0d04	Merge branch 'release/v1.14.0' into jen/v15	2022-02-09 03:17:38 -05:00
zenground0	e32704e7f6	Update node doc gen	2022-02-08 12:56:32 -05:00
zenground0	1e524b7210	Review Response	2022-02-08 12:39:35 -05:00
zenground0	5a09b6496d	Actually read config value	2022-02-08 12:39:35 -05:00
zenground0	2aa5911fb9	config gen	2022-02-08 12:39:35 -05:00
zenground0	4d8b912acc	New config for turning off auto deal creation	2022-02-08 12:39:35 -05:00
Masih H. Derkani	cc61650f86	Upgrade to latest `index-provider` and set miner ID as extra gossip data Upgrade to the latest `index-provider` which upgrades the go-legs protocol to allow the inclusion of extra gossip data that may be used for gossip validation purposes. In the case of lotus gossip message validators the miner ID is used to verify the sender's peer ID on chain. Relates to: - https://github.com/filecoin-project/lotus/pull/8045	2022-02-08 14:32:03 +00:00
gammazero	b2805823ce	Pass to validator the interfaces needed to get miner info	2022-02-08 04:55:59 -08:00
gammazero	1dc6a2fea6	Add indexer pubsub message authentication and rate limiting	2022-02-08 02:53:25 -08:00
Aarsh Shah	399a1afd94	Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx	2022-02-08 12:37:57 +04:00
vyzo	1221c0ba3f	make gen	2022-02-06 12:28:21 +02:00
vyzo	03352ea863	make badger the default splitstore markset type	2022-02-06 11:21:16 +02:00
Aarsh Shah	e155eb9fa3	Merge pull request #8016 from filecoin-project/nonsense/add-fullnode-api add indexProvider host to markets	2022-02-04 12:56:36 +04:00
Aarsh Shah	4691b2b809	changes to the indexer message relay PR	2022-02-04 12:15:01 +04:00
Aarsh Shah	3ecf478ff0	Merge pull request #8026 from gammazero/feat/cid-to-piece-idx Lotus chain nodes relay indexer pubsub messages	2022-02-04 12:05:46 +04:00
gammazero	c084130d3e	Lotus chain nodes relay indexer pubsub messages Content providers announce the availability of indexer data using gossip pubsub. The content providers are not connected directly to indexers, so the pubsub messages are relayed to indexers via chain nodes. This PR makes chain nodes relay gossip pubsub messages, on the /indexer/ingest/<netname> topic.	2022-02-03 14:56:21 -08:00
Will Scott	7247f8e41f	add net protect api methods fix #8015	2022-02-03 17:24:49 +01:00
Anton Evangelatov	dde2ab59ae	fixup	2022-02-03 15:53:46 +01:00
Anton Evangelatov	595ad44ee7	refactor indexprovider libp2p host connection to fullnode with meshcreator	2022-02-03 15:44:18 +01:00
Masih H. Derkani	575e6add43	Rename `IndexerProvider` to `IndexProvider` and add more docs Rename the config section corresponding to indexing to `IndexProvider` for better readability. Update existing docs for better clarity and add docs for config parameters embedded from `index-provider` `Ingest` config library.	2022-02-03 13:10:47 +00:00
Masih H. Derkani	235377585d	Clarify the purpose of `MaxSimultaneousTransfers` config Clarify the documentation of `MaxSimultaneousTransfers` to state that the "transfer" refers to multihash syncing only and not the data itself.	2022-02-03 11:51:01 +00:00
Anton Evangelatov	faa3a79a5f	use NetAddrListener iface	2022-02-02 16:01:19 +01:00
Anton Evangelatov	2790cf35af	change fullnodeApi to listen addrs	2022-02-02 13:06:53 +01:00
Anton Evangelatov	f9b63c5f8c	add idxProvHost to StorageProvider	2022-02-02 13:03:41 +01:00
Anton Evangelatov	db5e4da268	add fullnodeApi to StorageProvider	2022-02-02 13:03:41 +01:00
Łukasz Magiera	98f4d37f1a	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-01-31 19:50:52 +00:00
Darko Brdareski	e51ce5c508	Merge remote-tracking branch 'upstream/master' into bloxico/system-test-matrix	2022-01-27 10:57:56 +01:00
Darko Brdareski	8c41e17c93	Fix typo in client_test annotations	2022-01-27 10:56:33 +01:00
Masih H. Derkani	5d72ebe0cc	Upgrade `index-provider` and update topic name Update the topic name to be consistent with default configured in storetheindex. Upgrade to the latest `index-provider`.	2022-01-27 09:48:10 +00:00
Aayush	1b18236f91	feat: sealer: allow users to abort in-flight snap upgrades	2022-01-25 13:01:51 -05:00
Łukasz Magiera	4235a97cf4	retrieval: OffChainRetrieval config	2022-01-20 18:19:27 +01:00
Łukasz Magiera	8b19b84140	paych: option to force off-chain get	2022-01-20 18:19:26 +01:00
Łukasz Magiera	8e46b9ea5d	paych: API to pre-fund channels	2022-01-20 18:15:46 +01:00
Aarsh Shah	b8bf2af3e3	upgrade on master	2022-01-20 16:34:13 +04:00
Aarsh Shah	9ee940098d	Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx	2022-01-20 16:21:48 +04:00
Łukasz Magiera	e476cf7968	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-01-20 13:15:48 +01:00
vyzo	73ec10a49e	close the rcmgr on shutdown	2022-01-20 11:47:39 +02:00
vyzo	325a4c0e5b	fix lint	2022-01-20 11:46:39 +02:00
vyzo	554848fe6b	add resource manager Net(Set)Limit api	2022-01-20 11:36:11 +02:00
vyzo	d05d5bcb70	add resource manager NetStat api	2022-01-20 11:36:11 +02:00
vyzo	8d3f98fe38	instantiate resource manager in DI	2022-01-20 11:36:11 +02:00
vyzo	826cdb2186	use the libp2p connmgr	2022-01-20 11:36:11 +02:00
Łukasz Magiera	b38141601c	Untangle ffi from api	2022-01-18 11:57:04 +01:00
Łukasz Magiera	03fa955e9e	Cleanup imports	2022-01-18 11:37:24 +01:00
Łukasz Magiera	f2496d87c9	post workers: Snapdeals updates	2022-01-18 11:25:04 +01:00
Łukasz Magiera	0c062f9c08	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-01-18 10:49:42 +01:00
hannahhoward	92d56d82f4	feat(deps): update markets stack update go-fil-markets, go-data-transfer 1.13.0, go-graphsync 0.12.0	2022-01-14 17:21:04 -08:00
vyzo	7e43bfc109	hook the upgrade schedule to splitstore start	2022-01-14 17:14:32 -05:00
zenground0	d6aa17e21f	Snap Deals Integration - FSM handles the actual cc upgrade process including error states - PoSting (winning and window) works over upgraded and upgrading sectors - Integration test and changes to itest framework to reduce flakes - Update CLI to handle new upgrade - Update dependencies	2022-01-14 17:14:32 -05:00
Łukasz Magiera	4a874eff70	post workers: Cleanup, tests	2022-01-14 14:17:52 +01:00
c r	da6752eccb	feat: #7747 sealing: Adding conf variable for capping number of concurrent unsealing jobs (#7884 ) * adding the new variables- now time for logic * putting parameters into right placeS * adding unsealing throttle * fixing linter issues * removing one last thing...	2022-01-13 12:26:13 -06:00
Aarsh Shah	8459d75597	update index provider config	2022-01-13 16:15:43 +04:00
Aarsh Shah	b780a64ed9	send markets multiaddrs to the engine	2022-01-13 15:31:15 +04:00
Aarsh Shah	8c8652e7bb	add a new cli	2022-01-12 18:06:48 +04:00
Aarsh Shah	dec79c70a1	fix compilation	2022-01-12 15:03:46 +04:00
Aarsh Shah	21a8f71859	compiles	2022-01-12 14:59:09 +04:00
Aarsh Shah	edfc2c215c	Merge remote-tracking branch 'origin/main' into feat/cid-to-piece-idx	2022-01-12 12:48:54 +04:00
zenground0	33f2d24f54	Snap Deals Integration - FSM handles the actual cc upgrade process including error states - PoSting (winning and window) works over upgraded and upgrading sectors - Integration test and changes to itest framework to reduce flakes - Update CLI to handle new upgrade - Update dependencies	2022-01-10 15:39:38 +05:30
hannahhoward	1078dff3a9	feat(storageminer): also list data transfers with no id	2021-12-22 13:41:56 -08:00
hannahhoward	8a923fb420	fix(storageminer): fix conditional	2021-12-22 13:41:56 -08:00
hannahhoward	177282d9f9	fix(storageminer): lint/mod-tidy/extra checks	2021-12-22 13:41:55 -08:00
hannahhoward	cddf63efe9	feat(storageminer): add api for transfer diagnostics Add API + CLI for inspecting in depth diagnostics on graphsync transfers with a given peer	2021-12-22 13:41:29 -08:00
Aayush Rajasekaran	1bc987772c	Merge pull request #7818 from filecoin-project/asr/vm-nvfactor Refactor: VM: Remove the NetworkVersionGetter	2021-12-21 12:19:53 -05:00
Darko Brdareski	a54ac2ef79	Remove FS, MEM, GAS leftover test anotations.	2021-12-20 17:00:01 +01:00
Darko Brdareski	86cf17c1f4	Remove bad annotation from gas_test.go	2021-12-20 16:51:09 +01:00
Darko Brdareski	dda1a42a2a	Merge branch 'bloxico/system-test-matrix' of https://github.com/filecoin-project/lotus into merge_lotus	2021-12-20 15:48:16 +01:00
Aayush Rajasekaran	3f4eaa99d5	Refactor: State: Rename stmgr::GetNtwkVersion to GetNetworkVersion	2021-12-17 18:43:39 -05:00
Dirk McCormick	b1734f84b3	feat: retrieval ask CLI command	2021-12-17 15:55:12 +01:00
Łukasz Magiera	a4728d3c72	Merge pull request #7578 from filecoin-project/rvagg/SimultaneousTransfersForStoragePerClient feat(graphsync): allow setting of per-peer incoming requests for miners	2021-12-17 14:27:15 +01:00
vyzo	dd327f0b22	plumb more contexts	2021-12-17 11:42:09 +02:00
Rod Vagg	9e7d9affbe	feat(graphsync): allow setting of per-peer incoming requests for miners	2021-12-17 15:04:16 +11:00
Darko Brdareski	3e32aa896c	Annotate client storage deals feature	2021-12-16 14:36:02 +01:00
vyzo	22ce395130	fix some unit tests	2021-12-14 17:05:10 +02:00
vyzo	716b4a3b09	hook a great context in the sky to satisfy DI; sigh...	2021-12-14 16:57:09 +02:00
vyzo	eb48dc9b68	fix issues with new peerstore constructor signature in DI	2021-12-14 16:21:55 +02:00
vyzo	f157ac9b68	update for new ConnManager constructor	2021-12-14 15:30:29 +02:00
Dirk McCormick	a51b08e386	feat: add IndexerProvider AnnounceAddress config	2021-12-13 18:24:05 +01:00
Masih H. Derkani	6b7b9793ab	Set default index provider ingest config values Set default values for index-provider ingest config. Leave a TODO to use provider config APIs once defaults exposed. Run `go mod tidy`	2021-12-13 14:44:56 +00:00
Darko Brdareski	0addca1070	Fix bad annotations	2021-12-13 13:41:04 +01:00
vyzo	84710cf27e	plumb more contexts in lotus	2021-12-13 14:26:59 +02:00
Aayush Rajasekaran	dfb65ed89f	Plumb contexts through	2021-12-11 17:04:00 -05:00
vyzo	5d6398f20e	hook the upgrade schedule to splitstore start	2021-12-03 12:11:54 +02:00
Łukasz Magiera	f39942283b	Merge pull request #7706 from filecoin-project/feat/ret-matchding-cars retrieval: Only output matching nodes, MatchPath dagspec	2021-11-29 22:14:25 +01:00
Łukasz Magiera	d019853687	review: Cleanup some comments Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>	2021-11-29 21:29:00 +01:00
Łukasz Magiera	5b5e6b9e44	retrieval: DagSpec.MatchPath -> ExportMerkleProof	2021-11-29 21:14:00 +01:00
Łukasz Magiera	61791b90ea	retrieval: Only output matching nodes, MatchPath dagspec	2021-11-29 20:40:55 +01:00
Steven Allen	797147097c	Merge pull request #7689 from filecoin-project/disable-mplex disable mplex stream muxer	2021-11-29 09:46:24 -08:00
Łukasz Magiera	77d75b7739	Merge pull request #7688 from filecoin-project/chore/partret_cleanup_comment_and_flow Cleanup partial retrieval codepaths ( zero functional changes )	2021-11-29 16:39:55 +01:00
Łukasz Magiera	8d955d5f30	dagstore mount: Add random access support	2021-11-26 17:40:53 +01:00
Marten Seemann	cf0faf58dc	remove muxer config via LIBP2P_MUX_PREFS env	2021-11-26 11:01:54 +04:00
Marten Seemann	12dc3baff9	disable mplex stream muxer	2021-11-25 09:38:57 -08:00
Peter Rabbitson	d5b4d93575	Cleanup partial retrieval codepaths ( zero functional changes ) - Adjust the ExportRef comment to reflect change from #7673 - Simplify control flow in parseDagSpec() ( read diff under -w )	2021-11-25 10:35:05 +01:00
Łukasz Magiera	0c884734a3	Merge pull request #7673 from filecoin-project/chore/DM-level-tests_plus_merkle-proof-cars Chore/dm level tests plus merkle proof cars	2021-11-24 20:43:39 +01:00
Łukasz Magiera	bd4927b494	retrieval: Cleanup some comments	2021-11-24 20:13:49 +01:00
Peter Rabbitson	46937a1b9f	Add tests cementing the DM-level behavior on simple paths	2021-11-24 15:31:15 +01:00
Łukasz Magiera	78949d6c05	retrieval: Fix deadlock in ClientGetRetrievalUpdates	2021-11-23 18:32:56 +01:00
Łukasz Magiera	407c2ed114	retrieval: Drop the RootSelector hack	2021-11-23 17:42:43 +01:00
Łukasz Magiera	25e89d3a7a	retrieval: require unixfs exports to be aligned on block boundary	2021-11-22 13:29:09 +01:00
Łukasz Magiera	9ea229ed5a	retrieval: fix defult ask	2021-11-22 13:04:12 +01:00
Łukasz Magiera	2c583b03ff	retrieval: Support DM-paths in ls	2021-11-22 12:46:29 +01:00
Łukasz Magiera	74f645a098	retrieval: Fix parseDagSpec root check	2021-11-22 12:46:29 +01:00
Łukasz Magiera	597b72e286	retrieval: Fix lint, cli docsgen	2021-11-22 12:46:29 +01:00
Łukasz Magiera	9c119bfdad	retrieval: Make the ls command work	2021-11-22 12:46:17 +01:00
Łukasz Magiera	450d0687da	retrieval: REST export endpoint	2021-11-22 12:46:17 +01:00
Łukasz Magiera	b83a9b902a	gofmt	2021-11-22 12:46:17 +01:00
Łukasz Magiera	b26906963b	retrieval: Support multi-root export	2021-11-22 12:46:17 +01:00
Łukasz Magiera	b9bd061bdd	fix unixfs selector root node selection	2021-11-22 12:46:04 +01:00
Łukasz Magiera	b868769ec8	more retrieval api work	2021-11-22 12:46:02 +01:00
Łukasz Magiera	89138bab4d	Simplify retrieval APIs	2021-11-22 12:45:42 +01:00
Łukasz Magiera	60ea33b1c7	client: cleanup clientRetrieve	2021-11-22 12:45:42 +01:00
Łukasz Magiera	08e297a217	client: Support json selectors in retrieval	2021-11-22 12:45:42 +01:00
Aayush Rajasekaran	32fc03886d	CLI: Add a lotus multisig cancel command	2021-11-18 19:21:17 -05:00
Aarsh Shah	7655e660f3	integrate store-the-hash	2021-11-18 12:07:12 +04:00
Anton Evangelatov	5454aebf13	CLI tools for the DAGStore inverted index (#7361 ) * cli commands for dagstore * address comments from Dirk * rename inverted index to piece index	2021-11-17 15:39:45 +02:00
Dirk McCormick	501308239f	feat: CLI command to announce deal to indexers	2021-11-17 12:16:22 +01:00
Aarsh Shah	8d3017febb	make gen should work	2021-11-17 15:02:11 +04:00
Aarsh Shah	caf3cfa7d8	Merge branch 'feat/cid-to-piece-idx' into feat/indexer-provider	2021-11-17 11:43:49 +04:00
Aarsh Shah	bfee6699d2	update go mod and inject store the hash index in dagstore	2021-11-17 11:42:14 +04:00
Dirk McCormick	627d4fbf74	wip: storage miner index provider	2021-11-16 17:45:42 +01:00
Matija Petrunić	df17ea06f5	Merge branch 'master' into libp2p-pubsub-tracer	2021-11-10 12:47:09 +00:00
Matija Petrunic	59040d4419	Trace reject messages in pubsub	2021-11-10 13:45:40 +01:00
TheMenko	b45924a234	annotated repo_test	2021-11-01 10:49:30 +01:00
TheMenko	fb911a4542	annotated repo_test	2021-11-01 01:24:26 +01:00
TheMenko	9297e1b90d	annotated repo_test	2021-10-31 12:24:07 +01:00
Łukasz Magiera	d2e9d21031	Gather graphsync metrics on provider side as well	2021-10-19 19:45:25 +02:00
Łukasz Magiera	32a855b984	Fix lint	2021-10-19 19:22:32 +02:00
Łukasz Magiera	9a993d25d0	Collect and expose graphsync metrics	2021-10-19 19:20:00 +02:00
Jiaying Wang	afe54ab5f3	Merge pull request #7511 from filecoin-project/frrist/fix-lily fix: support node instantiation in external packages	2021-10-13 20:33:17 -04:00
frrist	6cd6732225	fix: support node instantiation in external packages - implement ChainGetPath on ChainModuleAPI	2021-10-13 15:24:28 -07:00
Anton Evangelatov	6e5ccc87cf	cli: add retry for deals stuck in Publish with no funds	2021-10-11 14:03:55 +02:00
Peter Rabbitson	a40aa9f90b	Normlize selector use within lotus	2021-10-07 11:01:29 +02:00
Peter Rabbitson	f8ac98bf46	Adjust error wording	2021-10-05 18:16:14 +02:00
Peter Rabbitson	5e6aceeb0e	Merge remote-tracking branch 'origin/master' into feat/datamodel-selector-retrieval	2021-10-05 18:15:51 +02:00
Jiaying Wang	cbb147de23	Merge pull request #7405 from filecoin-project/feat/update-graphsync-0.10.0 Update to graphsync to v0.10.0, enable seperate storage and retrieval transfer limits	2021-10-05 11:50:59 -04:00
Rod Vagg	97c2cdb8d6	traversals: limit link load count for SelectiveCar use as well	2021-10-05 10:47:49 +02:00
Rod Vagg	43f7fd5e10	traversals: limit maximum number of DAG links to traverse Impacts CommP and graphsync transfers	2021-10-05 10:47:49 +02:00
Dirk McCormick	a4e2fce9ca	feat: update to go-fil-markets v1.13.1	2021-10-05 10:40:01 +02:00
hannahhoward	368d72ebfe	feat(graphsync): update to v0.10.0-rc1 also add config changes	2021-10-05 14:13:58 +11:00
Peter Rabbitson	af0d9b650d	Error on corner cases with corresponding negative tests	2021-10-04 23:21:24 +02:00
Peter Rabbitson	2a1094fc17	Merge banch feat/go-fil-markets-v1.13.0 into feat/datamodel-selector-retrieval	2021-10-04 21:56:11 +02:00
Dirk McCormick	16973bc2e8	feat: use go-fil-markets@v1.13.0	2021-10-04 15:41:40 +02:00
Jennifer Wang	38e01cba83	Merge branch 'release/v1.12.0' into jen/12tomaster	2021-10-04 02:38:00 -04:00
Aayush Rajasekaran	f004d036dc	Set BatchPreCommitAboveBaseFee correctly	2021-10-01 14:23:41 -04:00
Łukasz Magiera	6fd9d5f28b	Precommit batch balancer support/config	2021-10-01 14:23:41 -04:00
Anton Evangelatov	809289f5ef	add Dealmaking.StartEpochSealingBuffer config	2021-10-01 17:44:15 +02:00
Łukasz Magiera	a8a9818043	Expose storage states on the metrics endpoint	2021-10-01 14:45:01 +02:00
Łukasz Magiera	7c84e22c3e	Add envconfig docs to the config	2021-09-30 18:23:05 +02:00
Aayush Rajasekaran	c3c46e9097	Fix Drand fetching around null tipsets	2021-09-29 11:57:59 -04:00
Mak Muftic	530df4298e	Fix linter errors	2021-09-29 16:49:03 +02:00
Mak Muftic	ea7673618d	Add unit tests for lotus tracer	2021-09-29 16:39:44 +02:00
Matija Petrunic	53c8e68fba	Remove extra tracer dtypes	2021-09-29 16:23:08 +02:00
Matija Petrunic	181b35dc5e	Extend peer score event with score components	2021-09-29 16:20:43 +02:00
Mak Muftic	7b19beaf5d	Fix linter errors	2021-09-29 13:12:42 +02:00
Mak Muftic	3433fa4a6e	Add stringdocs to config props and generate docs	2021-09-29 12:51:43 +02:00
Jennifer Wang	d7690aa719	Merge branch 'releases' into jen/backport	2021-09-29 03:21:19 -04:00
Mak Muftic	36913f4efc	Fix elasticsearch index setup	2021-09-27 13:28:28 +02:00
Matija Petrunic	6c680e306d	Rename json tracer flag	2021-09-24 16:25:41 +02:00
Mak Muftic	30fc2387be	Merge branch 'mpetrun5/lotus-extended-tracer' of github.com:ChainSafe/lotus into mpetrun5/lotus-extended-tracer	2021-09-24 14:16:31 +02:00
Mak Muftic	9332b9f399	Change json config naming	2021-09-24 13:53:31 +02:00
Mak Muftic	4438c4bd87	Move index name to config only and add default value	2021-09-24 13:43:25 +02:00
Mak Muftic	22bbb113e7	Fix schema creation	2021-09-24 12:58:32 +02:00
Mak Muftic	94a1601ab2	Remove hardcoded url schema	2021-09-24 12:40:33 +02:00
Mak Muftic	e46da5a181	Add cli flags	2021-09-24 12:37:16 +02:00
Łukasz Magiera	8e0314b3a3	Merge pull request #7362 from filecoin-project/chore/update-libp2p Update go-libp2p to v0.15.0	2021-09-23 14:24:12 +01:00
Aayush Rajasekaran	b9bfcc4b69	Randomness: Move getters from ChainAPI to StateAPI	2021-09-21 14:20:15 -04:00
Anton Evangelatov	c7e4217693	fix staging area path buildup	2021-09-21 16:45:00 +02:00
Aarsh Shah	89ac8b09b5	update libp2p	2021-09-21 15:10:04 +04:00
Matija Petrunic	0c390d12f7	Append source auth flag to lotus tracer event	2021-09-21 12:54:07 +02:00
Matija Petrunic	d65d9aa6e0	Send peerID peer score as separate events to enable usage of kibana runtime fields	2021-09-21 12:30:50 +02:00
Mak Muftic	082f70757c	Revert additional logging in elastic client	2021-09-20 12:29:12 +02:00
Mak Muftic	a685ccc955	Remove document id	2021-09-17 15:14:39 +02:00
Mak Muftic	0962e73e3e	Add loger to elastic search client	2021-09-17 14:22:57 +02:00
Matija Petrunic	7d0aefacbb	Change peer score event type to avoid exceeding elastisearch field limit	2021-09-17 13:46:26 +02:00
Matija Petrunic	a91ffa60d6	Set peer id event as string in lotus trace event	2021-09-17 12:11:14 +02:00
Mak Muftic	dc79ea363d	Change index name	2021-09-17 11:35:36 +02:00
ldoublewood	eb2bbfebe3	format code by make gen	2021-09-17 00:20:24 +08:00
Mak Muftic	f09f5c2d11	Change elastic document name	2021-09-16 16:49:47 +02:00
Mak Muftic	8e2b474fc7	Use only one document on elastic	2021-09-16 16:44:35 +02:00
Matija Petrunic	9efa495b0e	Write every trace to new line when using json transport	2021-09-16 16:42:18 +02:00
Matija Petrunic	941a0f7153	Update peer score tracker creation to include lotus tracer	2021-09-16 16:31:36 +02:00
Matija Petrunic	9cdc5261c4	Fix lotus tracer being nil when remote tracer is not configured	2021-09-16 15:53:39 +02:00
Matija Petrunic	d067bc9f01	Move creating and opening file to json transport constructor	2021-09-16 15:22:18 +02:00
Matija Petrunic	e2206147cc	Add source auth token to lotus traces	2021-09-16 12:21:57 +02:00
Mak Muftic	c25a5e06cf	Use config string for elasticsearch client	2021-09-16 12:15:42 +02:00
ldoublewood	3a1fc8e7da	fix bug for CommittedCapacitySectorLifetime	2021-09-16 09:25:02 +08:00
Mak Muftic	5cbd2519da	Remove processing elasticsearch result	2021-09-15 16:20:36 +02:00
Mak Muftic	67c1d633c6	Rename jsonEvent to evt	2021-09-15 16:16:12 +02:00
Mak Muftic	add13c2646	Code cleanup	2021-09-15 16:05:44 +02:00
Mak Muftic	71a2ca9e7c	Add new wrapper structure in tracer transport	2021-09-15 16:03:51 +02:00
Mak Muftic	707faf57b5	Code cleanup	2021-09-15 15:11:34 +02:00
Mak Muftic	e20cd0ef27	Propagate error on creating transport	2021-09-15 14:58:36 +02:00
Mak Muftic	cf06096565	Fix tracer initialization	2021-09-15 14:54:20 +02:00
Mak Muftic	bb85ca031d	Add elasticsearch transport	2021-09-15 14:50:27 +02:00
Matija Petrunic	c064ce4852	Add generated documentation	2021-09-15 14:22:48 +02:00
Matija Petrunic	8a657b90a7	Refactor lotus tracer to behave like lp2p tracer	2021-09-15 13:41:56 +02:00
Matija Petrunic	ab86c5419d	Implement json tracer transport	2021-09-15 13:41:03 +02:00
Matija Petrunic	8e6a01de6d	Add configuration in pubsub to trace locally to json file	2021-09-15 13:40:54 +02:00
Aayush Rajasekaran	1d6db306ec	Randomness: Move getters from ChainAPI to StateAPI	2021-09-15 00:03:13 +02:00
Matija Petrunic	d5a4b0637a	Add lotus tracer transport calls	2021-09-10 15:37:38 +02:00
Mak Muftic	c367533a3c	Add transport interface	2021-09-10 12:36:38 +02:00
Peter Rabbitson	0444435589	Expose basic text-based datamodel selector on retrieval Syntaxt of selection is located at https://pkg.go.dev/github.com/ipld/go-ipld-selector-text-lite#SelectorSpecFromPath Example use, assuming that: - The root of the deal is a plain dag-pb unixfs directory - The directory is not sharded - The user wants to retrieve the first entry in that directory lotus client retrieve --miner f0XXXXX --datamodel-path-selector 'Links/0/Hash' bafyROOTCID ~/output For a much more elaborate example see the top of ./itests/deals_partial_retrieval_test.go	2021-09-10 09:44:11 +02:00
Peter Rabbitson	2c5f4386ba	Prepare the retrieval codepath for selectors Slight import-name shuffle, no functional changes	2021-09-09 12:42:58 +02:00
Łukasz Magiera	3a3a4fc413	Merge pull request #7276 from filecoin-project/nonsense/reject-deal-too-many-staging-deals introduce MaxStagingDealsBytes - reject new deals if our staging deals area is full	2021-09-06 10:01:44 -07:00
Anton Evangelatov	9e2129338a	clarify that 0 is unlimited	2021-09-06 17:07:19 +02:00
Anton Evangelatov	0b7793a927	GiB to Bytes	2021-09-06 15:52:25 +02:00
Anton Evangelatov	c69d1db1eb	introduce MaxStagingDealsGiB - reject new deals if our staging deals area is "full"	2021-09-06 15:01:06 +02:00
hannahhoward	b4e9bc50f8	feat(deps): update go-graphsync v0.8.0 Update to go-graphsync v0.8.0 with go-ipld-prime linksystem branch & trusted store.	2021-09-03 16:21:55 +02:00
Łukasz Magiera	31efe8a690	Merge pull request #7255 from filecoin-project/cleanup/consensus chain: Cleanup consensus logic	2021-09-02 19:48:37 +02:00
Łukasz Magiera	30fccaa0bd	fix lint	2021-09-02 18:45:18 +02:00
Łukasz Magiera	95b128b7bc	chain: Cleanup consensus logic	2021-09-02 18:09:37 +02:00
Łukasz Magiera	52e93371bd	builder: Handle chainstore config in ConfigFullNode	2021-08-31 13:25:18 +02:00
Matija Petrunic	b1dafd81b8	Add peer score tracing on pubsub peer score inspect	2021-08-30 14:49:10 +02:00
Matija Petrunic	99fbd7039e	Update pubsub tracking to trace from lotus and lp2p tracers	2021-08-30 14:38:15 +02:00
Matija Petrunic	522858c871	Add lotus tracer interface	2021-08-30 14:35:15 +02:00
Dirk McCormick	7b7a5b0b21	revert: changes to OnDealExpiredOrChanged in #5431 #7201	2021-08-30 10:42:41 +02:00
Łukasz Magiera	3708a7ac30	config: Port some docstrings from go-ipfs	2021-08-27 19:17:32 +02:00
Łukasz Magiera	17b7dcef6f	config for disabling NAT port mapping	2021-08-27 19:14:29 +02:00
dirkmc	77a19774cf	fix events API timeout handling for nil blocks (#7184 )	2021-08-27 09:05:00 +02:00
Łukasz Magiera	1ba427f638	alerting: Address review	2021-08-26 16:09:18 +02:00
Łukasz Magiera	b094e0913d	untangle fsjournal dependencies	2021-08-26 15:44:45 +02:00
Łukasz Magiera	5366821144	api for getting alert states	2021-08-26 15:44:45 +02:00
Łukasz Magiera	81b1dd12f8	Simple alert system; FD limit alerts	2021-08-26 15:44:45 +02:00
Aarsh Shah	928ac2191b	fix throttling bug	2021-08-25 16:19:02 +05:30
Aarsh Shah	e1a056db46	ignore nil throttler	2021-08-24 10:42:00 +05:30
Łukasz Magiera	12875a9664	api/command for encoding actor params	2021-08-20 17:32:57 +02:00
Łukasz Magiera	0beb48f11d	Fix error handling in SectorAddPieceToAny api impl	2021-08-19 12:56:40 +02:00
Jennifer Wang	7ff5844aa2	Merge branch 'releases' into jen/mas	2021-08-17 10:38:36 -04:00
Frrist	518b6f1d41	fix(miner): always create miner deal staging directory (#7098 ) - fixes #7097 Co-authored-by: Raúl Kripalani <raul@protocol.ai> Co-authored-by: Raúl Kripalani <raul@protocol.ai>	2021-08-17 00:16:06 +01:00
Aarsh Shah	d7076778e2	integrate DAG store and CARv2 in deal-making (#6671 ) This commit removes badger from the deal-making processes, and moves to a new architecture with the dagstore as the cental component on the miner-side, and CARv2s on the client-side. Every deal that has been handed off to the sealing subsystem becomes a shard in the dagstore. Shards are mounted via the LotusMount, which teaches the dagstore how to load the related piece when serving retrievals. When the miner starts the Lotus for the first time with this patch, we will perform a one-time migration of all active deals into the dagstore. This is a lightweight process, and it consists simply of registering the shards in the dagstore. Shards are backed by the unsealed copy of the piece. This is currently a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so when it's time to acquire a shard to serve a retrieval, the unsealed CARv1 is joined with its index (safeguarded by the dagstore), to form a read-only blockstore, thus taking the place of the monolithic badger. Data transfers have been adjusted to interface directly with CARv2 files. On inbound transfers (client retrievals, miner storage deals), we stream the received data into a CARv2 ReadWrite blockstore. On outbound transfers (client storage deals, miner retrievals), we serve the data off a CARv2 ReadOnly blockstore. Client-side imports are managed by the refactored imports.Manager component (when not using IPFS integration). Just like it before, we use the go-filestore library to avoid duplicating the data from the original file in the resulting UnixFS DAG (concretely the leaves). However, the target of those imports are what we call "ref-CARv2s": CARv2 files placed under the `$LOTUS_PATH/imports` directory, containing the intermediate nodes in full, and the leaves as positional references to the original file on disk. Client-side retrievals are placed into CARv2 files in the location: `$LOTUS_PATH/retrievals`. A new set of `Dagstore` JSON-RPC operations and `lotus-miner dagstore` subcommands have been introduced on the miner-side to inspect and manage the dagstore. Despite moving to a CARv2-backed system, the IPFS integration has been respected, and it continues to be possible to make storage deals with data held in an IPFS node, and to perform retrievals directly into an IPFS node. NOTE: because the "staging" and "client" Badger blockstores are no longer used, existing imports on the client will be rendered useless. On startup, Lotus will enumerate all imports and print WARN statements on the log for each import that needs to be reimported. These log lines contain these messages: - import lacks carv2 path; import will not work; please reimport - import has missing/broken carv2; please reimport At the end, we will print a "sanity check completed" message indicating the count of imports found, and how many were deemed broken. Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com> Co-authored-by: Raúl Kripalani <raul@protocol.ai> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>	2021-08-16 23:34:32 +01:00
ZenGround0	10615b53ad	Merge pull request #6853 from filecoin-project/frrist/miner-temp-file-path feat(miner deals): create subdir to miner repo for staged deals	2021-08-16 16:19:05 -04:00
ZenGround0	eaff65c00f	Remove network version panic	2021-08-12 23:50:30 -04:00
Raúl Kripalani	f9ea2e834c	address nits.	2021-08-12 22:11:09 -04:00
Raúl Kripalani	85a4f8325f	fix docs and nits.	2021-08-12 22:08:20 -04:00
Anton Evangelatov	45166e760b	fixup	2021-08-12 22:06:11 -04:00
Anton Evangelatov	e90e71456b	add RuntimeSubsystems API method; use it in `lotus-miner info`	2021-08-12 22:05:08 -04:00
ZenGround0	6da26dcefc	Remove network version panic	2021-08-10 13:07:30 -04:00
Steven Allen	57bf5c2143	feat: add ChainGetTipSetAfterHeight This is identical to ChainGetTipSetByHeight, but returns the tipset following any null tipsets. This is what the user usually wants anyways. (and I need it for another PR)	2021-08-05 12:07:14 -07:00
Peter Rabbitson	eefdcb0454	Remove forgotten non-functioning config from the pre-mainnet days	2021-08-02 17:04:23 +02:00
frrist	ca697c9aa2	feat(miner deals): create subdir to miner repo for staged deals	2021-07-29 21:08:38 -07:00
vyzo	1d50dadc18	fix bug in message exclusion	2021-07-29 21:58:59 +03:00
Raúl Kripalani	b3c951c924	add RepoType#String; adjust repo parsing logic.	2021-07-29 12:22:27 -04:00
Raúl Kripalani	8442bac5b2	support MARKETS_API_INFO env var; support markets-repo, markets-api-url flags.	2021-07-29 12:20:05 -04:00
Raúl Kripalani	3144da86f3	add RepoType#String; adjust repo parsing logic.	2021-07-29 13:49:51 +01:00
Raúl Kripalani	4e19d8d562	support MARKETS_API_INFO env var; support markets-repo, markets-api-url flags.	2021-07-29 10:55:37 +01:00
Raúl Kripalani	d8c90b91be	address nits.	2021-07-29 04:41:27 -04:00
Raúl Kripalani	c119ab6ed9	fix docs and nits.	2021-07-29 04:39:28 -04:00
Anton Evangelatov	0dd83c6755	fixup	2021-07-29 04:38:25 -04:00
Anton Evangelatov	9b7a7713a7	add RuntimeSubsystems API method; use it in `lotus-miner info`	2021-07-29 04:38:19 -04:00
Raúl Kripalani	d3742048f9	address nits.	2021-07-28 23:56:18 +01:00
Raúl Kripalani	a61ba7b09c	Merge branch 'master' into nonsense/get-enabled-subsystems-api	2021-07-28 23:55:59 +01:00
Aayush Rajasekaran	21244704fc	Merge branch 'master' into mg/feat/configurable-sector-expirations	2021-07-28 16:45:27 -04:00
raulk	80aedcd276	Merge branch 'master' into nonsense/get-enabled-subsystems-api	2021-07-28 20:09:43 +01:00
Raúl Kripalani	07f40b9488	fix docs and nits.	2021-07-28 20:03:25 +01:00
Łukasz Magiera	389f71251c	Remove townhall	2021-07-28 17:58:46 +02:00
Anton Evangelatov	4cf5c8f656	fixup	2021-07-28 16:23:04 +03:00
Anton Evangelatov	de4a847078	add RuntimeSubsystems API method; use it in `lotus-miner info`	2021-07-28 16:02:05 +03:00
Jiaying Wang	d45b4a7a1f	Merge branch 'master' into chore/cleanup-chain	2021-07-27 17:11:36 -04:00
Łukasz Magiera	9bd312881d	VMSys doesn't belong in chainstore	2021-07-27 15:30:23 +02:00
vyzo	7c195245a1	make cfgdoc-gen	2021-07-27 12:13:26 +03:00
vyzo	9d25464703	use functional options for hotstore gc, rename MovingGC to FullGC	2021-07-27 09:53:22 +03:00
vyzo	aa0bd51b2c	thread GCFrequency option into the splitstore config	2021-07-26 15:05:57 +03:00
vyzo	20f93a520f	make cfgdoc-gen	2021-07-26 15:05:57 +03:00
vyzo	4cdb34e448	add docstrings for splitstore config	2021-07-26 15:05:57 +03:00
vyzo	5acae50e07	add config option for splitstore moving gc frequency	2021-07-26 15:05:57 +03:00
vyzo	a0d6fdba33	add ChainBlockstoreInfo APIv1 endpoint	2021-07-26 08:30:07 +03:00
vyzo	2dc72d5849	satisfy linter who wants to be a spell checker in comments	2021-07-25 11:47:21 +03:00
vyzo	3d2ae433ee	add ChainCheckBlockstore API	2021-07-25 11:14:48 +03:00
Mike Greenberg	e68c59f984	fix: Update node config docs	2021-07-23 21:05:50 -04:00
Mike Greenberg	64d2b7a122	Merge branch 'origin/master' into mg/feat/configurable-sector-expirations * origin/master: (47 commits) Fix links in issue templates Update issue templates and add templates for M1 config: Add newlines consistently config: Add note on codegen to types.go fix lint cli docsgen config: Bring some docs from filecoin-docs add comment about the necessity of FileIO improve README config doc gen: generate formatted code config: Fix doc finding logic Nicer lotus-miner config commands Move doc-comment logic to the config pkg config: doc struct codegen lotus daemon config utils adjust compaction test with badger to hit the db add splitstore compaction test with badger markset update README with configuration options tweak badger options some tweaks in badger markset ... Conflicts: node/config/def.go	2021-07-23 20:36:54 -04:00
Łukasz Magiera	91c6aca7e7	config: Add newlines consistently	2021-07-23 16:59:55 +02:00
Łukasz Magiera	80af6de316	config: Add note on codegen to types.go	2021-07-23 16:02:45 +02:00
Łukasz Magiera	ac2a1bfeab	fix lint	2021-07-23 15:53:31 +02:00
Łukasz Magiera	a26712ebc4	config: Bring some docs from filecoin-docs	2021-07-23 15:40:40 +02:00
Łukasz Magiera	ed7f70e68e	config doc gen: generate formatted code	2021-07-23 15:18:20 +02:00
Łukasz Magiera	3f4973cb41	config: Fix doc finding logic	2021-07-23 15:17:22 +02:00
Łukasz Magiera	8b1f19e94c	Move doc-comment logic to the config pkg	2021-07-23 14:55:19 +02:00
Łukasz Magiera	97f5bb66c7	config: doc struct codegen	2021-07-23 13:55:50 +02:00
Mike Greenberg	adb62a3fff	chore: Move cfg getter into PCPolicy; Clamp values on get	2021-07-22 15:07:03 -04:00
Mike Greenberg	9e34cce1f2	feat(config): support configurable CC sector expiration	2021-07-22 15:07:03 -04:00
Łukasz Magiera	660829703a	Merge remote-tracking branch 'origin/master' into feat/split-net-api	2021-07-22 15:38:06 +02:00
Łukasz Magiera	da5aeda197	Merge branch 'master' into feat/splitstore-hot-messages	2021-07-22 12:58:06 +02:00
Peter Rabbitson	4830a3406b	Introduce the LOTUS_CHAIN_BADGERSTORE_DISABLE_FSYNC envvar Allows the user to control the opts.SyncWrites option of BadgerDs	2021-07-21 22:38:14 +02:00
Anton Evangelatov	fe31956713	fetch peer id from chain	2021-07-21 11:43:19 +02:00
vyzo	839b00ab40	rename messagepool ProtectMessages to ForEachPendingMessage	2021-07-20 09:23:36 +03:00
vyzo	ebbaf23af8	support out-of-chain reference protection	2021-07-20 09:02:40 +03:00
vyzo	1b77361301	add option for hotstore message retention	2021-07-17 08:35:35 +03:00
vyzo	e003203bea	implement exposed splitstore	2021-07-15 13:12:10 +03:00
Łukasz Magiera	812dc266cf	builder: Don't require specific NetAPI impl in StorageMinerAPI	2021-07-15 12:01:13 +02:00
Łukasz Magiera	49e26cce7d	api: Separate the Net interface from Common	2021-07-15 11:41:30 +02:00
hunjixin	bdaa73a413	add ChainGetMessagesInTipset api	2021-07-15 14:52:09 +08:00
Łukasz Magiera	583a8a13d9	Merge pull request #6629 from filecoin-project/feat/pledge-from-miner-balance Config for collateral from miner available balance	2021-07-13 17:22:00 +02:00
Łukasz Magiera	837322ea59	Merge pull request #6356 from filecoin-project/nonsense/split-market-miner-processes Support standalone miner-market process	2021-07-13 17:16:07 +02:00
Łukasz Magiera	7526a074d9	sealing: collateral buffer / falback config	2021-07-13 17:04:58 +02:00
Łukasz Magiera	c37401a1a3	Merge pull request #6474 from filecoin-project/feat/splitstore-redux Splitstore Enhanchements	2021-07-13 12:43:57 +02:00
hannahhoward	92ec239202	fix(node): ClientRetrieve stops on cancel When a deal is cancelled, any in progress calls to ClientRetrieve or ClientRetrieveWithEvents should return	2021-07-12 14:46:18 -07:00
Łukasz Magiera	83f2368507	Add CollateralFromMinerBalance config	2021-07-12 14:11:58 +02:00
Anton Evangelatov	4bc9fa04ba	rename cfg.Subsystems.EnableStorageMarket to EnableMarkets	2021-07-12 12:12:29 +02:00
Anton Evangelatov	056136ef5a	rename argument from url to apiInfo	2021-07-12 11:36:22 +02:00
Anton Evangelatov	a423c46f2a	rename mock.go to net.go	2021-07-12 11:35:03 +02:00
Anton Evangelatov	d89ddb9315	resolve conflicts	2021-07-12 11:34:37 +02:00
Anton Evangelatov	16784aa2cc	remove pieceProvider from DI; small refactors	2021-07-12 11:30:26 +02:00
vyzo	c0a1cfffa1	rename noopstore to discardstore	2021-07-09 19:19:37 +03:00
vyzo	abdf4a161a	explicitly switch marksets for concurrent marking this has very noticeable impact in initial marking time; it also allows us to get rid of the confusing ts monikers.	2021-07-09 04:26:36 +03:00
Łukasz Magiera	ed0cc0b56d	Handle bad peerIDs in MarketListRetrievalDeals	2021-07-08 19:27:03 +02:00
Łukasz Magiera	8f5c0c60f5	Config for deal publishing control addresses	2021-07-07 18:00:54 +02:00
Anton Evangelatov	82efe01c70	fixup	2021-07-07 18:00:28 +02:00
Anton Evangelatov	6b014f57e5	pass Subsystems to StorageMiner option; add enableLibp2p bool in Settings	2021-07-07 13:56:37 +02:00
Anton Evangelatov	42ae876711	remove ApplyIfEnableLibP2P	2021-07-07 13:31:56 +02:00
Anton Evangelatov	7a78527fde	remove not used sa; cleanup	2021-07-06 17:46:21 +02:00
Anton Evangelatov	10dc90f7fe	addressing comments	2021-07-06 17:33:47 +02:00
Anton Evangelatov	898687ea2e	Update node/modules/storageminer.go Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>	2021-07-06 17:28:52 +02:00
Anton Evangelatov	ff2772a58c	resolved conflicts	2021-07-06 16:00:41 +02:00
Łukasz Magiera	97207df705	Merge pull request #6639 from filecoin-project/frrist/context-in-statetree-diff polish(statetree): accept a context in statetree diff for timeouts	2021-07-05 17:30:31 +02:00
Anton Evangelatov	4be0a7a215	resolve conflicts with master	2021-07-05 13:13:32 +02:00
vyzo	190cb18ab0	housekeeping - remove defunct tracking store implementations - update splitstore node config - use mark set type config option (defaulting to mapts); a memory constrained node may want to use an on-disk one	2021-07-04 18:38:28 +03:00
vyzo	5f7ae1f489	update splistore DI constructor	2021-07-04 18:38:28 +03:00
vyzo	68bc5d2291	skip moving cold blocks when running with a noop coldstore it is a noop but it still takes (a lot of) time because it has to read all the cold blocks.	2021-07-04 18:38:28 +03:00
vyzo	649b7dd162	add config option for hot headers	2021-07-04 18:38:28 +03:00
vyzo	997f2c098b	keep headers hot when running with a noop splitstore	2021-07-04 18:38:28 +03:00
vyzo	04f2e102a1	kill full splitstore compaction, simplify splitstore configuration	2021-07-04 18:38:28 +03:00
vyzo	5cca29d1db	hook noop blockstore for splitstore in DI	2021-07-04 18:38:28 +03:00
Łukasz Magiera	8a94ab676e	storage: Fix FinalizeSector with sectors in stoage paths	2021-07-02 19:54:45 +02:00
Anton Evangelatov	9e85492b85	fix test	2021-07-02 12:24:07 +02:00
Anton Evangelatov	96245c3cfa	lint	2021-07-02 12:02:36 +02:00
Anton Evangelatov	4194c7fa24	revert back optional for Miner; remove ActorAddress call in node.MinerHandler	2021-07-02 11:58:31 +02:00
Raúl Kripalani	1869de7662	unwire host from storage.Miner.	2021-07-01 13:53:20 +01:00
Łukasz Magiera	c094aa82ec	commit batch: AggregateAboveBaseFee config	2021-07-01 13:33:54 +02:00
frrist	d02a2a2cfa	polish(statetree): accept a context in statetree diff for timeouts - this operation can take a while, this changes gives users the option to abort if it takes "too long".	2021-06-30 16:40:43 -07:00
Anton Evangelatov	3aa6d03cc1	connect miner to full node , only when running markets	2021-06-30 16:45:11 +02:00
Anton Evangelatov	4f0a96c9c7	resolved conflicts	2021-06-30 13:16:52 +02:00
Anton Evangelatov	9ec7cd7239	enable libp2p options	2021-06-30 12:55:43 +02:00
Łukasz Magiera	e9dd3e8650	Test Miner SimultaneousTransfers	2021-06-28 18:17:29 +02:00
Łukasz Magiera	37c5dd5afc	Miner SimultaneousTransfers config	2021-06-28 14:24:14 +02:00
Łukasz Magiera	3f3336340f	Fix wallet error messages	2021-06-25 10:48:47 +02:00
Łukasz Magiera	c3480dc0e8	Merge remote-tracking branch 'origin/releases' into chore/merge-release	2021-06-24 09:39:11 +02:00
Łukasz Magiera	9521c0b773	Add miner-side MaxDealStartDelay config	2021-06-23 19:45:13 +02:00
Łukasz Magiera	0e4cd3c599	Merge pull request #6563 from filecoin-project/asr/batch-starter Always flush when timer goes off	2021-06-23 19:19:30 +02:00
Aayush Rajasekaran	616e5688fc	Remove MinPreCommitBatch	2021-06-23 12:30:32 -04:00
Łukasz Magiera	fa2be42eb0	Merge remote-tracking branch 'origin/release/v1.10.0' into chore/merge-1.10	2021-06-23 15:51:00 +02:00
Anton Evangelatov	4217ec5308	make auth on remote rpc server optional	2021-06-23 13:27:03 +02:00
Anton Evangelatov	21b51328f9	adding TestDealWithMarketAndMinerNode	2021-06-23 12:44:52 +02:00
Jakub Sztandera	bf82b96fd8	Update default fees for aggregates Resolves #6422 Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-06-22 20:20:01 +02:00
Anton Evangelatov	6720463799	resolve merge conflicts	2021-06-22 11:28:23 +02:00
Raúl Kripalani	b6147fb27f	extern/storage: retype resource filtering strategy to enum.	2021-06-21 20:28:15 +01:00
Anton Evangelatov	577b730415	wip	2021-06-18 18:42:34 +02:00
Łukasz Magiera	097baeb9b0	Make batch deal input test less flaky	2021-06-17 20:52:09 -04:00
Łukasz Magiera	5c8498b603	storagefsm: Fix batch deal packing behavior	2021-06-17 20:50:14 -04:00
Łukasz Magiera	44de67cf7d	Merge pull request #6175 from filecoin-project/feat/dynamic-retreival-pricing Dynamic Retrieval pricing	2021-06-17 10:25:48 +02:00
Anton Evangelatov	d45bb14015	Merge branch 'master' into nonsense/split-market-miner-processes	2021-06-16 18:49:15 +02:00
Aayush Rajasekaran	520a0091f5	Merge branch 'releases' into release/v1.10.0	2021-06-15 17:40:36 -04:00
Łukasz Magiera	0514f38dc4	Merge remote-tracking branch 'origin/master' into chore/merge-1.10	2021-06-14 13:27:53 +02:00
Anton Evangelatov	31e6fb154e	resolve conflicts	2021-06-14 11:27:54 +02:00
aarshkshah1992	4419205b0e	fix compilation	2021-06-14 10:10:29 +05:30
Aarsh Shah	bed2bc73a3	Update node/config/def.go Co-authored-by: raulk <raul@protocol.ai>	2021-06-14 09:42:22 +05:30
Łukasz Magiera	94be3a973a	Don't enable early finalization by default	2021-06-11 11:45:20 +02:00
Łukasz Magiera	ec06f086ef	sealing: Early finalization option	2021-06-11 11:41:28 +02:00
aarshkshah1992	fed5afa704	merge master	2021-06-11 09:35:20 +05:30
Łukasz Magiera	6f80d9f3c2	Merge pull request #6311 from filecoin-project/raulk/itests consolidate integration tests into `itests` package; create test kit; cleanup	2021-06-10 12:33:29 +02:00
Jennifer Wang	8f426b49ec	updated configuration comments for docs	2021-06-09 23:36:44 -04:00
Jennifer	3a1ad50da5	Merge pull request #6440 from filecoin-project/jen/miner-config-doc updated configuration comments for docs	2021-06-09 23:35:04 -04:00
Jennifer Wang	95ad74a156	updated configuration comments for docs	2021-06-09 20:44:20 -04:00
Raúl Kripalani	6d46be53bd	make tests no longer create auth tokens.	2021-06-10 00:15:29 +01:00
Łukasz Magiera	f0a2e97cb5	fee config for sector batching	2021-06-09 14:23:06 -04:00
Raúl Kripalani	81b3c6e2ab	Merge branch 'master' into raulk/itests	2021-06-09 17:34:46 +01:00
Anton Evangelatov	beb5bffaf6	fix log.Error; rename RemoteStore	2021-06-09 13:05:54 +02:00
Łukasz Magiera	78171055e7	fee config for sector batching	2021-06-08 15:45:32 +02:00
Anton Evangelatov	9f3ec82cc1	initial resolution of conflicts	2021-06-08 13:01:22 +02:00
aarshkshah1992	4e9bb16532	changes as per review	2021-06-08 10:09:27 +05:30
aarshkshah1992	16166504f4	merged master	2021-06-08 09:17:40 +05:30
Raúl Kripalani	a274fcc533	Merge branch 'master' into raulk/itests	2021-06-07 23:44:22 +01:00
Raúl Kripalani	db1a61852d	Merge branch 'master' into raulk/itests	2021-06-07 23:17:44 +01:00
Jakub Sztandera	f9acd07987	Update libp2p to 0.14.1 This does not resolve the CI issue but prepares us for it after https://github.com/libp2p/go-libp2p/pull/1116 Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-06-07 18:51:48 +02:00
Łukasz Magiera	fadc79a487	Merge pull request #6280 from filecoin-project/feat/pieceread-outside-scheduler Bypass task scheduler for reading unsealed pieces	2021-06-07 18:11:44 +02:00
Łukasz Magiera	8625da3479	Bump miner/worker api versions	2021-06-07 16:57:24 +02:00
aarshkshah1992	21e6b50294	finished rebasing PR	2021-06-07 16:02:15 +05:30
Anton Evangelatov	de646a5b93	fix node builder	2021-06-07 12:16:10 +02:00
aarshkshah1992	4efaa8d688	unit tests for the remote store Reader	2021-06-07 15:02:52 +05:30
aarshkshah1992	65eb610ec3	docs, logs and green ci	2021-06-07 15:02:49 +05:30
aarshkshah1992	670835fca0	bypass task scheduler for reading unsealed pieces	2021-06-07 15:02:04 +05:30
Lion	e4588aed5c	Fix the doc errors of the sealing config funcs	2021-06-07 12:35:43 +08:00
Dirk McCormick	52b90e4c9d	test: offline deals	2021-06-04 22:42:07 +02:00
Łukasz Magiera	a7746961fb	Merge remote-tracking branch 'origin/master' into feat/list-retrievals	2021-06-04 20:49:01 +02:00
Anton Evangelatov	ded5341da0	fixup test node builder	2021-06-04 17:22:36 +02:00
Anton Evangelatov	ed634bc3a4	rebuild docs	2021-06-04 17:06:55 +02:00
Anton Evangelatov	c4d10bc44e	apply messagepool.NewProviderLite to lite nodes	2021-06-04 16:51:07 +02:00
Anton Evangelatov	6f125cc129	add prover to fullnode and miner builders	2021-06-04 16:26:32 +02:00
Anton Evangelatov	d9a7348ae1	use masters filecoin-ffi	2021-06-04 16:20:40 +02:00
Anton Evangelatov	8bd09e39ca	resolve merge conflicts	2021-06-04 16:17:00 +02:00
Anton Evangelatov	c877166b4a	add messagepool to test node constructor	2021-06-04 15:53:33 +02:00
Anton Evangelatov	88756f3ebf	fix TestAPIDeal tests	2021-06-04 15:41:38 +02:00
aarshkshah1992	303ef15a83	fix ci	2021-06-04 17:33:11 +05:30
Łukasz Magiera	6d9779ecae	Merge remote-tracking branch 'origin/master' into feat/nv13-1.11	2021-06-02 15:07:13 +02:00
Łukasz Magiera	a676da8a7e	Merge remote-tracking branch 'origin/master' into feat/nv13-1.11	2021-06-01 21:07:04 +02:00
Łukasz Magiera	dc642d0b7b	Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11	2021-06-01 21:06:58 +02:00
Dirk McCormick	8997ed508b	feat: update to markets-v1.4.0	2021-06-01 10:16:12 -06:00
Łukasz Magiera	d69032b2f0	Merge pull request #6367 from filecoin-project/fix/robustify-commit-batcher Robustify commit batcher	2021-06-01 17:49:46 +02:00
Łukasz Magiera	5a7f97210f	Merge pull request #6317 from filecoin-project/jen/test-datacap-topup Add test for AddVerifiedClient	2021-06-01 14:41:05 +02:00
Łukasz Magiera	5a9e6c8142	Merge pull request #6364 from filecoin-project/asr/resolve-messages Improve address resolution for messages	2021-06-01 14:38:31 +02:00
Łukasz Magiera	482e1110c2	precommit batcher: Improve error propagation	2021-06-01 14:35:30 +02:00
Łukasz Magiera	cb4eb487f4	commit batcher: Fix min aggregate size check	2021-06-01 12:27:22 +02:00
Łukasz Magiera	9fcb564bef	Make commit batcher more robust	2021-06-01 11:56:19 +02:00
Anton Evangelatov	e4136b0d42	add modules.StorageAuthWithURL to set correct token	2021-06-01 11:42:28 +02:00
Jennifer Wang	fcfc214ed1	Use MockSbBuilder	2021-05-31 20:14:11 -04:00
Aayush Rajasekaran	9ceee6028b	More tweaking of context handling in the messagepool	2021-05-31 18:13:23 -04:00
Aayush Rajasekaran	ed93d0725f	Protect mp.localAddrs and mp.pending behind helper functions	2021-05-31 18:13:23 -04:00
Aayush Rajasekaran	1f03a618f9	Plumb contexts through	2021-05-31 18:13:23 -04:00
Łukasz Magiera	ffa47659a1	Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11	2021-05-31 21:38:34 +02:00
Łukasz Magiera	c3e8eddb9b	Merge remote-tracking branch 'origin/master' into feat/nv13-1.11	2021-05-31 21:24:56 +02:00
Łukasz Magiera	2a1b35e694	Merge pull request #6361 from filecoin-project/asr/fip-0015 Implement FIP-0015	2021-05-31 20:58:21 +02:00
Łukasz Magiera	2d6a159dcc	Merge pull request #6041 from filecoin-project/fix/batch-deal-packing storagefsm: Fix batch deal packing behavior	2021-05-31 20:45:12 +02:00
Aayush Rajasekaran	7fca1c1ee7	Implement FIP-0015	2021-05-31 14:40:54 -04:00
Peter Rabbitson	1e72d8f14c	Reduce noise from 'peer has different genesis' messages After the unification of all networks behind a build-tag, the amount of these warnings has risen sharply. Reduce it to a debug-level, since it isn't actionable to an operator.	2021-05-31 13:43:24 +02:00
Anton Evangelatov	d195a12565	Merge branch 'master' into nonsense/split-market-miner-processes	2021-05-31 11:26:11 +02:00
Łukasz Magiera	9475079b97	Make batch deal input test less flaky	2021-05-30 17:46:22 +02:00
Łukasz Magiera	c8d603557b	storagefsm: Fix batch deal packing behavior	2021-05-30 17:46:22 +02:00
Jennifer Wang	2b08e9f3a6	Add test AddVerifiedClient	2021-05-28 15:41:57 -04:00
Aayush Rajasekaran	fba5c65ffd	Extend the default deal start epoch delay	2021-05-28 12:35:48 -04:00
hannahhoward	19b6dc8d1e	feat(cli): add a list retrievals command Currently, there is no way to inspect retrievals on a client. This adds said command, allow with corresponding APIs	2021-05-27 11:48:25 -07:00
Łukasz Magiera	1e4456138e	Merge master into feat/nv13	2021-05-27 12:28:20 +02:00
Anton Evangelatov	c311520968	add missing messagepool override	2021-05-27 10:28:41 +02:00
Aayush Rajasekaran	d9e86afa24	Tweak client calcDealExpiration to consider genesis miners	2021-05-26 21:46:37 -04:00
Aayush Rajasekaran	21b4741e30	Fix randomness fetching around null blocks	2021-05-26 21:06:06 -04:00
Aayush Rajasekaran	7714537239	Allow starting networks from arbitrary actor versions	2021-05-26 12:48:14 -04:00
Anton Evangelatov	9979de24db	logs and defaults	2021-05-26 12:47:21 +02:00
Łukasz Magiera	43c62f4406	Revert "Allow starting networks from arbitrary actor versions"	2021-05-26 12:33:08 +02:00
Aayush Rajasekaran	cf574ca9a1	Allow starting networks from arbitrary actor versions	2021-05-25 19:30:20 -04:00
Łukasz Magiera	4c6c9a0edb	Fix windowpost/deadline tests	2021-05-25 21:27:32 +02:00
Łukasz Magiera	04658e1cae	fix lint	2021-05-25 16:26:59 +02:00
Łukasz Magiera	d90ab2b81f	config: Document batching fields	2021-05-25 16:20:49 +02:00
aarshkshah1992	78255eac54	test unsealing prices for default pricing strategy	2021-05-24 15:11:50 +05:30
Raúl Kripalani	c3f45cc836	Merge branch 'raulk/refactor-rpc' into raulk/itests	2021-05-23 14:56:54 +01:00
Raúl Kripalani	2337248aa5	rpc: separate handlers/server; refactor miner rpc.	2021-05-23 14:48:42 +01:00
Raúl Kripalani	627248d610	Merge branch 'raulk/refactor-rpc' into raulk/itests	2021-05-23 12:41:25 +01:00
Raúl Kripalani	de454b8e26	add godocs to ServeRPC.	2021-05-23 12:40:33 +01:00
aarshkshah1992	dc6dbc9a11	dpr changes and test based on new unsealing PR	2021-05-22 22:40:21 +05:30
Raúl Kripalani	5b77dbd06f	decouple rpc server and shutdown logic from cmd.	2021-05-21 21:34:17 +01:00
aarshkshah1992	bd9959070e	unit tests for the remote store Reader	2021-05-20 15:25:46 -06:00
aarshkshah1992	73613ee883	docs, logs and green ci	2021-05-20 15:25:46 -06:00
aarshkshah1992	2a40c802ea	bypass task scheduler for reading unsealed pieces	2021-05-20 15:25:46 -06:00
Dirk McCormick	885564fe24	Revert "chore: update go-libp2p" This reverts commit `f7fbaef361`.	2021-05-20 11:22:19 -06:00
Łukasz Magiera	13ff6ed462	Test pledging before nv13 upgrade	2021-05-20 18:38:33 +02:00
Anton Evangelatov	4693c61305	re-arrange NodeBuilder and add storageminer_svc	2021-05-20 13:10:14 +02:00
Anton Evangelatov	76bb424de0	update StorageMinerAPI	2021-05-20 13:01:14 +02:00
Raúl Kripalani	06b8ee5b95	Merge branch 'master' into raulk/itests	2021-05-20 11:56:43 +01:00
Anton Evangelatov	cb603c62d9	update retrievaladapter ; add piece_provider	2021-05-20 12:49:53 +02:00
Anton Evangelatov	2aad7b6979	update sectorstorage.New (Manager) interface	2021-05-20 12:32:29 +02:00
Dirk McCormick	706fdcbb80	feat: increase data transfer timeouts	2021-05-19 12:56:07 -06:00
Łukasz Magiera	3a99f95f8e	Merge pull request #6149 from filecoin-project/feat/markets-1.3 Update to markets 1.3	2021-05-19 20:51:33 +02:00
Raúl Kripalani	c46d4ae529	wip	2021-05-19 17:30:43 +01:00
Łukasz Magiera	2a1b359ede	config: fmt	2021-05-19 15:20:34 +02:00
Łukasz Magiera	a5677d1b7a	ffiwrapper: Separate Prover interface	2021-05-19 15:20:23 +02:00
Łukasz Magiera	2a0c0e379b	Working default batching config	2021-05-19 14:32:41 +02:00
Anton Evangelatov	a989f60e27	add SectorAddPieceToAny and SectorUnsealPiece to StorageMiner iface; model moved to api package - PieceDealInfo, DealSchedule	2021-05-19 13:05:07 +02:00
Łukasz Magiera	5112b9fe2b	Lower default batch slack	2021-05-18 21:04:47 +02:00
Łukasz Magiera	9690bc882c	Test to trigger batching logic	2021-05-18 18:41:42 +02:00
Łukasz Magiera	81b5d8c671	Make things build with both batchers	2021-05-18 16:53:49 +02:00
Łukasz Magiera	c7ba083fa4	Import precommit batcher	2021-05-18 16:51:06 +02:00
vyzo	7512748b56	wire in sealing config values	2021-05-18 15:31:52 +03:00
vyzo	74bad49068	correctly handle commit batch timer	2021-05-18 14:30:47 +03:00
vyzo	357c0868b7	proper config for termination batching and commit wait	2021-05-18 12:20:19 +03:00
vyzo	febf7cf28f	sane config defaults	2021-05-18 12:02:53 +03:00
Aayush Rajasekaran	9a6e601754	Merge pull request #5961 from filecoin-project/feat/stateless-offline-dealflow Introduce stateless offline dealflow, bypassing the FSM/deallists	2021-05-17 12:35:39 -04:00
Łukasz Magiera	506f39b294	WIP: Integrate FIP0013	2021-05-11 22:10:29 -04:00
Peter Rabbitson	d54ed1b0c6	Merge remote-tracking branch 'origin/master' into feat/stateless-offline-dealflow	2021-05-11 04:54:22 +02:00
Steven Allen	f7fbaef361	chore: update go-libp2p From 0.12.0 to 0.14.0. Headline for lotus: faster yamux.	2021-05-10 19:41:57 -07:00
Peter Rabbitson	6f71eab8ce	Adjust lp2p retry params	2021-05-11 04:26:04 +02:00
Jakub Sztandera	18cbdcfc81	Disable checks API on Lotus Lite Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 17:19:17 +02:00
Jakub Sztandera	d777680449	Fix mpool.GetActor for lite node Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:06 +02:00
Jakub Sztandera	8d75da1629	Use MessagePrototype for check API Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:05 +02:00
vyzo	e2d0047a2a	introduce message prototypes This introduces message prototypes to applicable API endpoints, which allows us to invert control of message sending and give the user a chance to intervene with an interactive ui. Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:05 +02:00
vyzo	d782250aba	implement MessagePool.CheckReplaceMessages Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:04 +02:00
vyzo	91e774063e	implement MessagePool.CheckMessages Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:04 +02:00
vyzo	ed61642b3a	implement NodeStatus API Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:04 +02:00
Peter Rabbitson	e648b72d8d	Merge remote-tracking branch 'origin/master' into feat/stateless-offline-dealflow	2021-05-06 15:57:10 +02:00
Dirk McCormick	5b1fab8ffa	feat: markets v1.3	2021-05-06 09:49:44 +02:00
Steven Allen	eb10918470	Merge pull request #6107 from filecoin-project/feat/checkpoint-sync feat: allow checkpointing to forks	2021-04-29 16:30:52 -07:00
Łukasz Magiera	9b87566dbf	Merge pull request #6141 from filecoin-project/split-TestBatchDealInput-from-TestAPIDealFlow flaky tests improvement: separate TestBatchDealInput from TestAPIDealFlow	2021-04-29 19:01:35 +02:00
Steven Allen	f983fa8cd5	Merge pull request #6134 from filecoin-project/asr/emptytsk Use EmptyTSK where appropriate	2021-04-29 09:35:43 -07:00
Anton Evangelatov	c7d74ef16e	separate TestBatchDealInput from TestAPIDealFlow	2021-04-29 17:39:00 +02:00
Łukasz Magiera	a456f394ac	Merge pull request #6007 from filecoin-project/asr/state-list-messages Speed up StateListMessages in some cases	2021-04-29 07:47:26 +02:00
Aayush Rajasekaran	d794b49df3	Use EmptyTSK where appropriate	2021-04-29 01:00:03 -04:00
Steven Allen	69da6a2a29	feat: allow checkpointing to forks Previously, `lotus sync checkpoint` would only checkpoint on the current chain. Now, it can switch to a new fork.	2021-04-28 13:54:26 -07:00
Łukasz Magiera	a24a73bb52	Merge tag 'v1.8.0' into release/v1.9.0	2021-04-27 10:30:33 +02:00
Łukasz Magiera	c4137a438d	Merge commit 'f1ded63d608e1b191d79c780b8065f8cff367c98' into feat/nv12-1.11	2021-04-27 08:22:55 +02:00
Aayush Rajasekaran	f1ded63d60	v4 specs-actors integration, nv12 migration	2021-04-27 02:02:44 -04:00
Reskorey	58bee65bcb	go fmt	2021-04-23 16:10:51 +08:00
Reskorey	7738886936	fix full node painc fix: full node painc when calling RPC Filecoin.ClientRetrieve without the order.Tatol pecified.	2021-04-23 15:43:41 +08:00
Łukasz Magiera	c72eb0ccaf	Merge pull request #6048 from filecoin-project/fix/nonce-getting Fix nonce getting on Lotus lite	2021-04-18 16:25:47 +02:00
Peter Rabbitson	dab2a41792	Raise client MaxFee default The original value is too conservative for the current state of mainnet Resolves (temporarily) #5543	2021-04-17 00:35:40 +02:00
Jakub Sztandera	66c408938b	Fix signature in messagepool, wire in context Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-04-16 18:43:28 +02:00
Jakub Sztandera	77d004ec06	Fix nonce getting on Lotus lite Resolves #5593 #5995 Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-04-15 19:57:09 +02:00
Łukasz Magiera	e76935147f	Merge remote-tracking branch 'origin/master' into feat/v1-api	2021-04-13 14:24:22 +02:00
Aayush Rajasekaran	ff48a70ed9	Speed up StateListMessages in some cases	2021-04-10 02:33:34 -04:00
Łukasz Magiera	88bdb591d5	Merge pull request #6003 from filecoin-project/fix/fbs-di Fix fallback chainstore	2021-04-10 01:58:38 +02:00
Łukasz Magiera	7e4cb9da84	Fix fallback chainstore	2021-04-09 16:11:02 +02:00
Dirk McCormick	4a3261a986	feat: go-fil-markets-v1.2.5-rc1	2021-04-08 14:34:35 +02:00
Peter Rabbitson	25a77d905d	Centralize everything on ipfs/go-log/v2 I am not entirely sure this is right, but everything seems to build...	2021-04-06 15:04:32 +02:00
Łukasz Magiera	e8f28d7b9f	Fix tests	2021-04-06 12:24:58 +02:00
Łukasz Magiera	81bd27911f	Propagate StateMsg api changes	2021-04-05 19:56:53 +02:00
Peter Rabbitson	b79be2a2c2	Merge remote-tracking branch 'origin/master' into feat/stateless-offline-dealflow	2021-04-05 16:06:44 +02:00
Łukasz Magiera	deb2b90b6a	Fix lotus/miner build	2021-04-05 13:23:46 +02:00
Peter Rabbitson	7fddbb528d	Introduce stateless offline dealflow, bypassing the FSM/deallists This is aproposal for an additional flag --manual-stateless-deal and a corresponding API endpoint ClientStatelessDeal. This allows firing off an offline-style deal against a miner without keeping further track of it locally. Not keeping any local state introduces the limitation of requiring free storage deals, as there is nothing to tie the payment channel setup to. Rationale/need for this type of flow is the case of incredibly large sets of data nd deals, where the client and providers have prearranged payment ahead of time, and the client has a separate-from-lotus database of deal inventory. This way the client can use their lotus node merely as a network gateway, without running into any limitations currently present in both lotus as a whole and go-fil-markets in particular. Specific context for this work is filecoin-discover, where the requirement is to onboard ~ 12,000,000 individual deals against a pool of miners with whom the client has prearranged a relationship.	2021-04-05 13:11:10 +02:00
Łukasz Magiera	f4e46c9003	implement v1 api with api modules	2021-04-03 13:20:50 +02:00
Łukasz Magiera	1b32d7f52f	cli: fix build with v1 api changes	2021-04-03 12:55:29 +02:00
Łukasz Magiera	0103d2f621	v1 api: Cleanup message wait/search apis	2021-04-02 13:52:24 +02:00
Łukasz Magiera	c9866da688	Merge remote-tracking branch 'origin/master' into feat/v1-api	2021-04-02 13:15:33 +02:00
Łukasz Magiera	4538550999	Merge remote-tracking branch 'origin/master' into feat/local-retrieval	2021-04-02 12:36:28 +02:00

... 13 14 15 16 17 ...

3410 Commits