lotus

Author	SHA1	Message	Date
Łukasz Magiera	54cb55a7ae	feat: miner: API/CLI to compute window-post	2022-03-28 17:06:51 -04:00
Łukasz Magiera	c951e348b4	feat: config: Move MakeNewSectorForDeals config into the Sealing sectoin	2022-03-26 23:45:48 -04:00
Łukasz Magiera	b78892934b	feat: config: Move MakeNewSectorForDeals config into the Sealing sectoin	2022-03-26 15:50:21 -04:00
Łukasz Magiera	82343a06e7	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-03-25 16:49:46 -04:00
Łukasz Magiera	7c4d3a457c	Merge pull request #8343 from filecoin-project/feat/more-snap-config feat: sealing: More SnapDeals config knobs	2022-03-25 15:07:00 -04:00
Łukasz Magiera	945b93590d	ping: Match go-ipfs in cmd output	2022-03-21 15:11:19 +01:00
Łukasz Magiera	701d0a111e	Merge branch 'feat/net-ping' of github.com:ychiaoli18/lotus into feat/net-ping	2022-03-21 13:06:11 +01:00
Łukasz Magiera	92e4caa151	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-03-21 12:09:47 +01:00
Łukasz Magiera	433e9c351d	config: Fix doc on PreferNewSectorsForDeals; gofmt	2022-03-21 12:07:00 +01:00
Łukasz Magiera	97a73f1c29	sealing: More SnapDeals config knobs	2022-03-21 12:04:23 +01:00
Łukasz Magiera	fc34d9ba1c	Merge pull request #8086 from filecoin-project/nonsense/refactor-nodetype refactor: convert RepoType from int to interface	2022-03-21 11:57:39 +01:00
Anton Evangelatov	0ed8647b6f	extract unixfs filestore into lib	2022-03-21 10:48:52 +01:00
Łukasz Magiera	5365ccfdb1	cli: Show separate worker types in miner info	2022-03-18 21:31:15 +01:00
Łukasz Magiera	7b81369c8c	wdpost: Don't attempt to snark with skipped sectors	2022-03-18 19:45:25 +01:00
Anton Evangelatov	0d6493ec3f	resolve merge conflicts with master	2022-03-18 12:28:22 +01:00
Łukasz Magiera	6ddbe41376	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-03-18 10:54:44 +01:00
Aayush Rajasekaran	2ed2ee1da7	Merge branch 'release/v1.15.1' into feat/fvm	2022-03-17 11:31:56 -04:00
Łukasz Magiera	037e80c70c	gofmt	2022-03-17 12:10:24 +01:00
Łukasz Magiera	1c055fe83b	Merge branch 'master' into test/t.TempDir	2022-03-17 12:06:52 +01:00
Łukasz Magiera	d3cec2f0d1	sealing: Add MakeCCSectorsAvailable config	2022-03-16 22:29:40 +01:00
Łukasz Magiera	a3bdd29615	Merge pull request #8330 from filecoin-project/feat/snap-queue feat: sealing: Sector upgrade queue	2022-03-16 22:24:45 +01:00
Aayush Rajasekaran	90d78f924f	Merge pull request #8293 from filecoin-project/feat/fvm Filecoin Virtual Machine integration	2022-03-16 15:51:56 -04:00
Łukasz Magiera	4ded059a1c	sealing: Add MakeCCSectorsAvailable config	2022-03-16 19:29:47 +01:00
Darko Brdareski	79453663b3	feat: Add additional test annotations (#8272 ) * Annotate api,proxy_util,blockstore_badger, policy tests * Annotate splitstore: bsbadger / markset * Annotate splitstore feature * Annotate union/timed blockstore tests * Annotate openrpc, diff_adt tests * Annotate error,drand,events tests * Annotate predicates_test * Fix annotations * Annotate tscache, gen tests * Annotate fundmanager test * Annotate repub and selection tests * Annotate statetree_test * Annotate forks_test * Annotate searchwait_test.go * Fix duplicated @@ symbols * Annotate chain stmgr/store tests * Annotate more (types) tests * More tests annotated * Annotate conformance chaos actor tests * Annotate more integration tests * Annotate journal system tests * Annotate more tests. * Annotate gas,head buffer behaviors * Fix markset annotations * doc: test annotations for the markets dagstore wrapper * Annotate miner_api test in dagstore * Annotate more test files * Remove bad annotations from fsrepo * Annotate wdpost system * Remove bad annotations * Renamce "conformance" to "chaos_actor" tests * doc: stm annotations for blockheader & election proof tests * Annotate remaining "A" tests * annotate: stm for error_test * memrepo_test.go * Annotate "b" file tests * message_test.go * doc: stm annotate for fsrepo_test * Annotate "c" file tests * Annotate "D" test files * message_test.go * doc: stm annotate for chain, node/config & client * docs: stm annotate node_test * Annotate u,v,wl tests * doc: stm annotations for various test files * Annotate "T" test files * doc: stm annotate for proxy_util_test & policy_test * doc: stm annotate for various tests * doc: final few stm annotations * Add mempool unit tests * Add two more memPool Add tests * Update submodules * Add check function tests * Add stm annotations, refactor test helper * Annotate api,proxy_util,blockstore_badger, policy tests * Annotate splitstore: bsbadger / markset solving merge conflicts * Annotate splitstore feature * Annotate union/timed blockstore tests * Annotate openrpc, diff_adt tests * Annotate error,drand,events tests * Annotate predicates_test * Fix annotations * Annotate tscache, gen tests * Annotate fundmanager test * Annotate statetree_test * Annotate forks_test * Annotate searchwait_test.go * Fix duplicated @@ symbols * Annotate chain stmgr/store tests * Annotate more (types) tests * More tests annotated * Annotate conformance chaos actor tests * Annotate more integration tests * Annotate journal system tests * Annotate more tests. * Annotate gas,head buffer behaviors solve merge conflict * Fix markset annotations * Annotate miner_api test in dagstore * Annotate more test files * doc: test annotations for the markets dagstore wrapper * Annotate wdpost system * Renamce "conformance" to "chaos_actor" tests * Annotate remaining "A" tests * doc: stm annotations for blockheader & election proof tests * annotate: stm for error_test * Annotate "b" file tests * memrepo_test.go * Annotate "c" file tests * message_test.go * Annotate "D" test files * doc: stm annotate for fsrepo_test * Annotate u,v,wl tests * message_test.go * doc: stm annotate for chain, node/config & client * docs: stm annotate node_test * Annotate "T" test files * doc: stm annotations for various test files * Add mempool unit tests solve merge conflict * doc: stm annotate for proxy_util_test & policy_test * doc: stm annotate for various tests * doc: final few stm annotations * Add two more memPool Add tests * Update submodules * Add check function tests solve conflict * Add stm annotations, refactor test helper solve merge conflict * Change CLI test kinds to "unit" * Fix double merged test * Fix ccupgrade_test merge * Fix lint issues * Add stm annotation to types_Test * Test vectors submodule * Add file annotation to burn_test Co-authored-by: Nikola Divic <divicnikola@gmail.com> Co-authored-by: TheMenko <themenkoprojects@gmail.com>	2022-03-16 18:37:34 +01:00
Aayush	16128a0212	Merge branch 'master' into feat/fvm	2022-03-16 11:57:46 -04:00
Aayush	9ea623e125	Rename the Lotus VM to LegacyVM	2022-03-15 19:40:17 -04:00
Jiaying Wang	ada499ef14	Merge pull request #8314 from filecoin-project/jen/indexsetting feat: config: enable indexer providing by default	2022-03-15 19:33:49 -04:00
Jiaying Wang	c98ef0aaf8	Merge pull request #8317 from filecoin-project/jen/backport chore: backport: release to master	2022-03-15 19:12:20 -04:00
vyzo	9fcafff847	adjust conns to 2x maxconns for duplicate connections	2022-03-15 10:02:27 +02:00
vyzo	53c525f0ed	improve resource manager integration - add opt-in env var to control instantation, until we are comfortable with testing to enble by default. - adjust default limits if the connection manager high mark is higher than the default inbound conn limit.	2022-03-15 09:43:18 +02:00
Jennifer Wang	8a2a2d065d	Merge branch 'releases' into jen/backport	2022-03-15 02:07:35 -04:00
Jennifer Wang	6240729ee4	enable indexer providing by default	2022-03-14 23:24:52 -04:00
Eng Zer Jun	c229842f57	test: use `T.TempDir` to create temporary test directory The directory created by `T.TempDir` is automatically removed when the test and all its subtests complete. Reference: https://pkg.go.dev/testing#T.TempDir Signed-off-by: Eng Zer Jun <engzerjun@gmail.com>	2022-03-15 00:21:31 +08:00
Łukasz Magiera	135aef78d7	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-03-11 17:04:58 +01:00
Łukasz Magiera	81f0cfdd1e	Merge pull request #8276 from filecoin-project/feat/faster-clientcalccommp feat: api: Make ClientCalcCommP multithreaded	2022-03-10 23:22:48 +01:00
Łukasz Magiera	0d9e34d651	client impl: Fix typo in ClientCalcCommP CopyBuffer error	2022-03-10 22:41:45 +01:00
Łukasz Magiera	4421bf2fa1	feat: config: Persistent subsystem log level config	2022-03-10 14:44:56 +01:00
Łukasz Magiera	9c4f8254ee	post workers: Fix build post-merge	2022-03-10 14:20:07 +01:00
Łukasz Magiera	a715fb3063	feat: Make ClientCalcCommP multithreaded	2022-03-09 23:25:29 +01:00
hannahhoward	49742f8fdc	feat(deps): update to graphsync v0.13.0 with 2.0 protocol	2022-03-09 18:06:35 +00:00
Łukasz Magiera	046a9f8af0	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-03-09 16:27:03 +01:00
Anton Evangelatov	7cdf6dc680	remove .Type() now that we have singletons for repos	2022-03-07 13:00:26 +01:00
Anton Evangelatov	b9fe61f9da	Merge branch 'master' into nonsense/refactor-nodetype	2022-03-07 11:51:28 +01:00
Anton Evangelatov	5734a39e8e	move DealmakingConfiger and SealingConfiger away from config/types.go	2022-03-03 16:51:47 +01:00
Anton Evangelatov	62e352ef88	move APIInfoEnvVars, RepoFlags, APIFlags under RepoType	2022-03-03 16:45:11 +01:00
Anton Evangelatov	acbfc879aa	config: extract DealmakingConfig and Sealing config from Lotus for Boost (#8172 ) * refactor sealingCfg and dealmakingCfg to be able to abstract Config * fix error messages	2022-03-03 15:37:23 +01:00
Anton Evangelatov	066d0edf51	add SupportsStagingDeals trait	2022-03-03 15:37:23 +01:00
Anton Evangelatov	949ec65118	drop defConfForType	2022-03-03 15:37:23 +01:00
Anton Evangelatov	f1f878a000	unexport repo types; use a global var for every repo type	2022-03-03 15:37:23 +01:00
Anton Evangelatov	e3edab66e3	refactor RepoType	2022-03-03 15:37:23 +01:00
Łukasz Magiera	7dcfeee0ea	use old go-path	2022-03-03 14:17:25 +00:00
Masih H. Derkani	1bb65189af	Upgrade to go-path `v0.2.1` and remove `replace` directive The dependency was used in the chain API. Integrated the new fetcher factory.	2022-03-03 13:29:32 +00:00
Aayush	85886287b4	Revert "use the libp2p connmgr" This reverts commit `826cdb2186`.	2022-03-02 15:39:23 -05:00
Aayush	b5912d1543	Revert "instantiate resource manager in DI" This reverts commit `8d3f98fe38`.	2022-03-02 15:39:22 -05:00
Aayush	e9e28f75e2	Revert "add resource manager NetStat api" This reverts commit `d05d5bcb70`.	2022-03-02 15:39:22 -05:00
Aayush	b8747e3afc	Revert "add resource manager Net(Set)Limit api" This reverts commit `554848fe6b`.	2022-03-02 15:39:21 -05:00
Aayush	48e28898a6	Revert "fix lint" This reverts commit `325a4c0e5b`.	2022-03-02 15:29:27 -05:00
Aayush	17ac583508	Revert "close the rcmgr on shutdown" This reverts commit `73ec10a49e`.	2022-03-02 15:29:26 -05:00
Masih H. Derkani	aacc246ba3	Merge branch 'master' into feat/cid-to-piece-idx	2022-03-02 14:06:30 +00:00
Masih H. Derkani	444a2bf355	Integrate the latest index-provider with config to disable announcements Integrate the latest `index-provider` and reflect the changes to engine configuration. Note that this commit disables announcements of indices on the network by default as requested for initial merge to master. Introduce dedicated index provider configuration parameters with documentation and defaults that match the defaults in index-provider. Re-generate code as needed.	2022-03-02 13:45:09 +00:00
Łukasz Magiera	e9a6f5f287	Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse	2022-03-02 12:56:56 +00:00
Masih H. Derkani	769b0f30ef	Address review comments - Add comment to clarify the reason for loop in testkit - Trim common prefix in state printed in CLI commands for better readability - Upgrade to a tagged release of `go-fil-markets` that includes indexing work; see: https://github.com/filecoin-project/go-fil-markets/pull/673 - Fix typo in CLI usage. - Add comments to note that it is safe to use fx `OnStart` context when starting the provider engine. - Fix string concatenation in error message formatting.	2022-03-02 11:04:50 +00:00
zenground0	77a954c7c3	Merge branch 'master' into feat/cid-to-piece-idx	2022-02-28 12:57:11 -07:00
jennijuju	c6361b391c	Merge branch 'releases' into jen/v15	2022-02-25 21:01:31 -05:00
zenground0	e5de237990	Fix fault tracker to handle snap deals	2022-02-24 17:53:21 -05:00
ZenGround0	d711775cdf	Merge pull request #8177 from filecoin-project/fix/fault-tracker-snap-deals fix:proving:post check sector handles snap deals replica faults	2022-02-24 11:30:10 -07:00
Aayush	fd8cb44e67	fix: client: calculate commps for pieces bigger than 32GB	2022-02-24 10:27:37 -05:00
zenground0	abe04c33c1	Fix fault tracker to handle snap deals	2022-02-23 09:56:47 -07:00
Aarsh Shah	e3f5b7c59f	update dagstore top level index db	2022-02-18 14:00:01 +04:00
Kevin Li	ba72eff3e6	feat: cli/net: implement 'net ping' command	2022-02-17 17:22:52 +08:00
Masih H. Derkani	a09f386537	Remove redundant `IndexProvider` config now that host is shared Now that host is shared having separate config for listen and announce addresses along with graphsync limit makes no sense since all of that is shared with the markets host and datatransfer manager.	2022-02-16 13:46:44 +00:00
Masih H. Derkani	2ebc111b70	Use the same host and datatransfer as markets for index provider Remove the bespoke instantiation of libp2p host and datatransfer manager for index provider and reuse the existing instances used by markets. The rationale for reuse is the following: 1. Separation of host introduces a discovery problem, where without gossipsub the index provider endpoint will not be discoverable. Using the same host as markets would mean the chain can be used to discover addresses, putting less empassis on criticality of gossipsub considering its set-up cost and lack of message delivery guarantees. 2. Only a single instance of graphsync/datatransfer can be instantiated per libp2p host; therefore, if the host is shared, so should datatransfer manager. 3. it is not clear if the assumptions under which separation was decided still hold.	2022-02-16 13:46:44 +00:00
Łukasz Magiera	681b90768a	Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse	2022-02-16 10:06:36 +01:00
vyzo	9c00af1b86	don't track peer ids in rcmgr metrics	2022-02-15 16:12:17 +02:00
Łukasz Magiera	48dea93c7e	Merge pull request #8089 from filecoin-project/feat/rcmgr-metrics Resource Manager Metrics	2022-02-15 14:40:44 +01:00
Aayush	75b8e4c981	fix: sealer: allow enable/disabling ReplicaUpdate tasks	2022-02-14 15:25:52 -05:00
Aayush	56df886b55	fix: sealer: allow enable/disabling ReplicaUpdate tasks	2022-02-14 15:24:13 -05:00
Łukasz Magiera	60b9acc09d	gen	2022-02-14 21:16:41 +01:00
Aayush	eebe784191	fix: sealer: allow enable/disabling ReplicaUpdate tasks	2022-02-14 15:08:28 -05:00
Łukasz Magiera	36a1934845	paychmgr: Fix tests after api changes	2022-02-14 20:16:30 +01:00
Łukasz Magiera	df5bd147fa	paychmgr: Review comments Co-authored-by: Hannah Howard <hannah@hannahhoward.net> Co-authored-by: dirkmc <dirkmdev@gmail.com>	2022-02-14 20:07:13 +01:00
Łukasz Magiera	f61eb23f8f	api: separate method for paych funding	2022-02-14 19:56:02 +01:00
Łukasz Magiera	10af768c60	Merge commit 'origin/release/v1.15.0~2' into feat/paych-avail-reuse	2022-02-14 19:27:12 +01:00
vyzo	a9ec408844	collect resource manager metrics	2022-02-14 19:46:05 +02:00
Masih H. Derkani	7e7844f169	Fix log message formatting when instantiating index provider	2022-02-11 12:15:21 +00:00
Will	296eab3045	Merge pull request #8045 from gammazero/feat/cid-to-piece-idx Add indexer pubsub message authentication and rate limiting	2022-02-10 13:49:58 -08:00
gammazero	eedcf910eb	change scoring based on review	2022-02-10 10:53:07 -08:00
gammazero	b72cd90c19	Add indexer message scoring	2022-02-10 09:21:21 -08:00
gammazero	3f3d61b043	Service creation takes interface, not implementation	2022-02-10 08:41:18 -08:00
Jennifer Wang	de1f4792cd	Merge remote-tracking branch 'origin/master' into jen/v15	2022-02-09 19:27:34 -05:00
Łukasz Magiera	44fd0e3349	Merge pull request #8008 from filecoin-project/feat/splitstore-sortless-compaction splitstore sortless compaction	2022-02-09 17:17:57 +00:00
Jennifer Wang	6ed97f0d04	Merge branch 'release/v1.14.0' into jen/v15	2022-02-09 03:17:38 -05:00
zenground0	e32704e7f6	Update node doc gen	2022-02-08 12:56:32 -05:00
zenground0	1e524b7210	Review Response	2022-02-08 12:39:35 -05:00
zenground0	5a09b6496d	Actually read config value	2022-02-08 12:39:35 -05:00
zenground0	2aa5911fb9	config gen	2022-02-08 12:39:35 -05:00
zenground0	4d8b912acc	New config for turning off auto deal creation	2022-02-08 12:39:35 -05:00
Masih H. Derkani	cc61650f86	Upgrade to latest `index-provider` and set miner ID as extra gossip data Upgrade to the latest `index-provider` which upgrades the go-legs protocol to allow the inclusion of extra gossip data that may be used for gossip validation purposes. In the case of lotus gossip message validators the miner ID is used to verify the sender's peer ID on chain. Relates to: - https://github.com/filecoin-project/lotus/pull/8045	2022-02-08 14:32:03 +00:00
gammazero	b2805823ce	Pass to validator the interfaces needed to get miner info	2022-02-08 04:55:59 -08:00
gammazero	1dc6a2fea6	Add indexer pubsub message authentication and rate limiting	2022-02-08 02:53:25 -08:00
Aarsh Shah	399a1afd94	Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx	2022-02-08 12:37:57 +04:00
vyzo	1221c0ba3f	make gen	2022-02-06 12:28:21 +02:00
vyzo	03352ea863	make badger the default splitstore markset type	2022-02-06 11:21:16 +02:00
Aarsh Shah	e155eb9fa3	Merge pull request #8016 from filecoin-project/nonsense/add-fullnode-api add indexProvider host to markets	2022-02-04 12:56:36 +04:00
Aarsh Shah	4691b2b809	changes to the indexer message relay PR	2022-02-04 12:15:01 +04:00
Aarsh Shah	3ecf478ff0	Merge pull request #8026 from gammazero/feat/cid-to-piece-idx Lotus chain nodes relay indexer pubsub messages	2022-02-04 12:05:46 +04:00
gammazero	c084130d3e	Lotus chain nodes relay indexer pubsub messages Content providers announce the availability of indexer data using gossip pubsub. The content providers are not connected directly to indexers, so the pubsub messages are relayed to indexers via chain nodes. This PR makes chain nodes relay gossip pubsub messages, on the /indexer/ingest/<netname> topic.	2022-02-03 14:56:21 -08:00
Will Scott	7247f8e41f	add net protect api methods fix #8015	2022-02-03 17:24:49 +01:00
Anton Evangelatov	dde2ab59ae	fixup	2022-02-03 15:53:46 +01:00
Anton Evangelatov	595ad44ee7	refactor indexprovider libp2p host connection to fullnode with meshcreator	2022-02-03 15:44:18 +01:00
Masih H. Derkani	575e6add43	Rename `IndexerProvider` to `IndexProvider` and add more docs Rename the config section corresponding to indexing to `IndexProvider` for better readability. Update existing docs for better clarity and add docs for config parameters embedded from `index-provider` `Ingest` config library.	2022-02-03 13:10:47 +00:00
Masih H. Derkani	235377585d	Clarify the purpose of `MaxSimultaneousTransfers` config Clarify the documentation of `MaxSimultaneousTransfers` to state that the "transfer" refers to multihash syncing only and not the data itself.	2022-02-03 11:51:01 +00:00
Anton Evangelatov	faa3a79a5f	use NetAddrListener iface	2022-02-02 16:01:19 +01:00
Anton Evangelatov	2790cf35af	change fullnodeApi to listen addrs	2022-02-02 13:06:53 +01:00
Anton Evangelatov	f9b63c5f8c	add idxProvHost to StorageProvider	2022-02-02 13:03:41 +01:00
Anton Evangelatov	db5e4da268	add fullnodeApi to StorageProvider	2022-02-02 13:03:41 +01:00
Łukasz Magiera	98f4d37f1a	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-01-31 19:50:52 +00:00
Darko Brdareski	e51ce5c508	Merge remote-tracking branch 'upstream/master' into bloxico/system-test-matrix	2022-01-27 10:57:56 +01:00
Darko Brdareski	8c41e17c93	Fix typo in client_test annotations	2022-01-27 10:56:33 +01:00
Masih H. Derkani	5d72ebe0cc	Upgrade `index-provider` and update topic name Update the topic name to be consistent with default configured in storetheindex. Upgrade to the latest `index-provider`.	2022-01-27 09:48:10 +00:00
Aayush	1b18236f91	feat: sealer: allow users to abort in-flight snap upgrades	2022-01-25 13:01:51 -05:00
Łukasz Magiera	4235a97cf4	retrieval: OffChainRetrieval config	2022-01-20 18:19:27 +01:00
Łukasz Magiera	8b19b84140	paych: option to force off-chain get	2022-01-20 18:19:26 +01:00
Łukasz Magiera	8e46b9ea5d	paych: API to pre-fund channels	2022-01-20 18:15:46 +01:00
Aarsh Shah	b8bf2af3e3	upgrade on master	2022-01-20 16:34:13 +04:00
Aarsh Shah	9ee940098d	Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx	2022-01-20 16:21:48 +04:00
Łukasz Magiera	e476cf7968	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-01-20 13:15:48 +01:00
vyzo	73ec10a49e	close the rcmgr on shutdown	2022-01-20 11:47:39 +02:00
vyzo	325a4c0e5b	fix lint	2022-01-20 11:46:39 +02:00
vyzo	554848fe6b	add resource manager Net(Set)Limit api	2022-01-20 11:36:11 +02:00
vyzo	d05d5bcb70	add resource manager NetStat api	2022-01-20 11:36:11 +02:00
vyzo	8d3f98fe38	instantiate resource manager in DI	2022-01-20 11:36:11 +02:00
vyzo	826cdb2186	use the libp2p connmgr	2022-01-20 11:36:11 +02:00
Łukasz Magiera	b38141601c	Untangle ffi from api	2022-01-18 11:57:04 +01:00
Łukasz Magiera	03fa955e9e	Cleanup imports	2022-01-18 11:37:24 +01:00
Łukasz Magiera	f2496d87c9	post workers: Snapdeals updates	2022-01-18 11:25:04 +01:00
Łukasz Magiera	0c062f9c08	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-01-18 10:49:42 +01:00
hannahhoward	92d56d82f4	feat(deps): update markets stack update go-fil-markets, go-data-transfer 1.13.0, go-graphsync 0.12.0	2022-01-14 17:21:04 -08:00
vyzo	7e43bfc109	hook the upgrade schedule to splitstore start	2022-01-14 17:14:32 -05:00
zenground0	d6aa17e21f	Snap Deals Integration - FSM handles the actual cc upgrade process including error states - PoSting (winning and window) works over upgraded and upgrading sectors - Integration test and changes to itest framework to reduce flakes - Update CLI to handle new upgrade - Update dependencies	2022-01-14 17:14:32 -05:00
Łukasz Magiera	4a874eff70	post workers: Cleanup, tests	2022-01-14 14:17:52 +01:00
c r	da6752eccb	feat: #7747 sealing: Adding conf variable for capping number of concurrent unsealing jobs (#7884 ) * adding the new variables- now time for logic * putting parameters into right placeS * adding unsealing throttle * fixing linter issues * removing one last thing...	2022-01-13 12:26:13 -06:00
Aarsh Shah	8459d75597	update index provider config	2022-01-13 16:15:43 +04:00
Aarsh Shah	b780a64ed9	send markets multiaddrs to the engine	2022-01-13 15:31:15 +04:00
Aarsh Shah	8c8652e7bb	add a new cli	2022-01-12 18:06:48 +04:00
Aarsh Shah	dec79c70a1	fix compilation	2022-01-12 15:03:46 +04:00
Aarsh Shah	21a8f71859	compiles	2022-01-12 14:59:09 +04:00
Aarsh Shah	edfc2c215c	Merge remote-tracking branch 'origin/main' into feat/cid-to-piece-idx	2022-01-12 12:48:54 +04:00
zenground0	33f2d24f54	Snap Deals Integration - FSM handles the actual cc upgrade process including error states - PoSting (winning and window) works over upgraded and upgrading sectors - Integration test and changes to itest framework to reduce flakes - Update CLI to handle new upgrade - Update dependencies	2022-01-10 15:39:38 +05:30
hannahhoward	1078dff3a9	feat(storageminer): also list data transfers with no id	2021-12-22 13:41:56 -08:00
hannahhoward	8a923fb420	fix(storageminer): fix conditional	2021-12-22 13:41:56 -08:00
hannahhoward	177282d9f9	fix(storageminer): lint/mod-tidy/extra checks	2021-12-22 13:41:55 -08:00
hannahhoward	cddf63efe9	feat(storageminer): add api for transfer diagnostics Add API + CLI for inspecting in depth diagnostics on graphsync transfers with a given peer	2021-12-22 13:41:29 -08:00
Aayush Rajasekaran	1bc987772c	Merge pull request #7818 from filecoin-project/asr/vm-nvfactor Refactor: VM: Remove the NetworkVersionGetter	2021-12-21 12:19:53 -05:00
Darko Brdareski	a54ac2ef79	Remove FS, MEM, GAS leftover test anotations.	2021-12-20 17:00:01 +01:00
Darko Brdareski	86cf17c1f4	Remove bad annotation from gas_test.go	2021-12-20 16:51:09 +01:00
Darko Brdareski	dda1a42a2a	Merge branch 'bloxico/system-test-matrix' of https://github.com/filecoin-project/lotus into merge_lotus	2021-12-20 15:48:16 +01:00
Aayush Rajasekaran	3f4eaa99d5	Refactor: State: Rename stmgr::GetNtwkVersion to GetNetworkVersion	2021-12-17 18:43:39 -05:00
Dirk McCormick	b1734f84b3	feat: retrieval ask CLI command	2021-12-17 15:55:12 +01:00
Łukasz Magiera	a4728d3c72	Merge pull request #7578 from filecoin-project/rvagg/SimultaneousTransfersForStoragePerClient feat(graphsync): allow setting of per-peer incoming requests for miners	2021-12-17 14:27:15 +01:00
vyzo	dd327f0b22	plumb more contexts	2021-12-17 11:42:09 +02:00
Rod Vagg	9e7d9affbe	feat(graphsync): allow setting of per-peer incoming requests for miners	2021-12-17 15:04:16 +11:00
Darko Brdareski	3e32aa896c	Annotate client storage deals feature	2021-12-16 14:36:02 +01:00
vyzo	22ce395130	fix some unit tests	2021-12-14 17:05:10 +02:00
vyzo	716b4a3b09	hook a great context in the sky to satisfy DI; sigh...	2021-12-14 16:57:09 +02:00
vyzo	eb48dc9b68	fix issues with new peerstore constructor signature in DI	2021-12-14 16:21:55 +02:00
vyzo	f157ac9b68	update for new ConnManager constructor	2021-12-14 15:30:29 +02:00
Dirk McCormick	a51b08e386	feat: add IndexerProvider AnnounceAddress config	2021-12-13 18:24:05 +01:00
Masih H. Derkani	6b7b9793ab	Set default index provider ingest config values Set default values for index-provider ingest config. Leave a TODO to use provider config APIs once defaults exposed. Run `go mod tidy`	2021-12-13 14:44:56 +00:00
Darko Brdareski	0addca1070	Fix bad annotations	2021-12-13 13:41:04 +01:00
vyzo	84710cf27e	plumb more contexts in lotus	2021-12-13 14:26:59 +02:00
Aayush Rajasekaran	dfb65ed89f	Plumb contexts through	2021-12-11 17:04:00 -05:00
vyzo	5d6398f20e	hook the upgrade schedule to splitstore start	2021-12-03 12:11:54 +02:00
Łukasz Magiera	f39942283b	Merge pull request #7706 from filecoin-project/feat/ret-matchding-cars retrieval: Only output matching nodes, MatchPath dagspec	2021-11-29 22:14:25 +01:00
Łukasz Magiera	d019853687	review: Cleanup some comments Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>	2021-11-29 21:29:00 +01:00
Łukasz Magiera	5b5e6b9e44	retrieval: DagSpec.MatchPath -> ExportMerkleProof	2021-11-29 21:14:00 +01:00
Łukasz Magiera	61791b90ea	retrieval: Only output matching nodes, MatchPath dagspec	2021-11-29 20:40:55 +01:00
Steven Allen	797147097c	Merge pull request #7689 from filecoin-project/disable-mplex disable mplex stream muxer	2021-11-29 09:46:24 -08:00
Łukasz Magiera	77d75b7739	Merge pull request #7688 from filecoin-project/chore/partret_cleanup_comment_and_flow Cleanup partial retrieval codepaths ( zero functional changes )	2021-11-29 16:39:55 +01:00
Łukasz Magiera	8d955d5f30	dagstore mount: Add random access support	2021-11-26 17:40:53 +01:00
Marten Seemann	cf0faf58dc	remove muxer config via LIBP2P_MUX_PREFS env	2021-11-26 11:01:54 +04:00
Marten Seemann	12dc3baff9	disable mplex stream muxer	2021-11-25 09:38:57 -08:00
Peter Rabbitson	d5b4d93575	Cleanup partial retrieval codepaths ( zero functional changes ) - Adjust the ExportRef comment to reflect change from #7673 - Simplify control flow in parseDagSpec() ( read diff under -w )	2021-11-25 10:35:05 +01:00
Łukasz Magiera	0c884734a3	Merge pull request #7673 from filecoin-project/chore/DM-level-tests_plus_merkle-proof-cars Chore/dm level tests plus merkle proof cars	2021-11-24 20:43:39 +01:00
Łukasz Magiera	bd4927b494	retrieval: Cleanup some comments	2021-11-24 20:13:49 +01:00
Peter Rabbitson	46937a1b9f	Add tests cementing the DM-level behavior on simple paths	2021-11-24 15:31:15 +01:00
Łukasz Magiera	78949d6c05	retrieval: Fix deadlock in ClientGetRetrievalUpdates	2021-11-23 18:32:56 +01:00
Łukasz Magiera	407c2ed114	retrieval: Drop the RootSelector hack	2021-11-23 17:42:43 +01:00
Łukasz Magiera	25e89d3a7a	retrieval: require unixfs exports to be aligned on block boundary	2021-11-22 13:29:09 +01:00
Łukasz Magiera	9ea229ed5a	retrieval: fix defult ask	2021-11-22 13:04:12 +01:00
Łukasz Magiera	2c583b03ff	retrieval: Support DM-paths in ls	2021-11-22 12:46:29 +01:00
Łukasz Magiera	74f645a098	retrieval: Fix parseDagSpec root check	2021-11-22 12:46:29 +01:00
Łukasz Magiera	597b72e286	retrieval: Fix lint, cli docsgen	2021-11-22 12:46:29 +01:00
Łukasz Magiera	9c119bfdad	retrieval: Make the ls command work	2021-11-22 12:46:17 +01:00
Łukasz Magiera	450d0687da	retrieval: REST export endpoint	2021-11-22 12:46:17 +01:00
Łukasz Magiera	b83a9b902a	gofmt	2021-11-22 12:46:17 +01:00
Łukasz Magiera	b26906963b	retrieval: Support multi-root export	2021-11-22 12:46:17 +01:00
Łukasz Magiera	b9bd061bdd	fix unixfs selector root node selection	2021-11-22 12:46:04 +01:00
Łukasz Magiera	b868769ec8	more retrieval api work	2021-11-22 12:46:02 +01:00
Łukasz Magiera	89138bab4d	Simplify retrieval APIs	2021-11-22 12:45:42 +01:00
Łukasz Magiera	60ea33b1c7	client: cleanup clientRetrieve	2021-11-22 12:45:42 +01:00
Łukasz Magiera	08e297a217	client: Support json selectors in retrieval	2021-11-22 12:45:42 +01:00
Aayush Rajasekaran	32fc03886d	CLI: Add a lotus multisig cancel command	2021-11-18 19:21:17 -05:00
Aarsh Shah	7655e660f3	integrate store-the-hash	2021-11-18 12:07:12 +04:00
Anton Evangelatov	5454aebf13	CLI tools for the DAGStore inverted index (#7361 ) * cli commands for dagstore * address comments from Dirk * rename inverted index to piece index	2021-11-17 15:39:45 +02:00
Dirk McCormick	501308239f	feat: CLI command to announce deal to indexers	2021-11-17 12:16:22 +01:00
Aarsh Shah	8d3017febb	make gen should work	2021-11-17 15:02:11 +04:00
Aarsh Shah	caf3cfa7d8	Merge branch 'feat/cid-to-piece-idx' into feat/indexer-provider	2021-11-17 11:43:49 +04:00
Aarsh Shah	bfee6699d2	update go mod and inject store the hash index in dagstore	2021-11-17 11:42:14 +04:00
Dirk McCormick	627d4fbf74	wip: storage miner index provider	2021-11-16 17:45:42 +01:00
Matija Petrunić	df17ea06f5	Merge branch 'master' into libp2p-pubsub-tracer	2021-11-10 12:47:09 +00:00
Matija Petrunic	59040d4419	Trace reject messages in pubsub	2021-11-10 13:45:40 +01:00
TheMenko	b45924a234	annotated repo_test	2021-11-01 10:49:30 +01:00
TheMenko	fb911a4542	annotated repo_test	2021-11-01 01:24:26 +01:00
TheMenko	9297e1b90d	annotated repo_test	2021-10-31 12:24:07 +01:00
Łukasz Magiera	d2e9d21031	Gather graphsync metrics on provider side as well	2021-10-19 19:45:25 +02:00
Łukasz Magiera	32a855b984	Fix lint	2021-10-19 19:22:32 +02:00
Łukasz Magiera	9a993d25d0	Collect and expose graphsync metrics	2021-10-19 19:20:00 +02:00
Jiaying Wang	afe54ab5f3	Merge pull request #7511 from filecoin-project/frrist/fix-lily fix: support node instantiation in external packages	2021-10-13 20:33:17 -04:00
frrist	6cd6732225	fix: support node instantiation in external packages - implement ChainGetPath on ChainModuleAPI	2021-10-13 15:24:28 -07:00
Anton Evangelatov	6e5ccc87cf	cli: add retry for deals stuck in Publish with no funds	2021-10-11 14:03:55 +02:00
Peter Rabbitson	a40aa9f90b	Normlize selector use within lotus	2021-10-07 11:01:29 +02:00
Peter Rabbitson	f8ac98bf46	Adjust error wording	2021-10-05 18:16:14 +02:00
Peter Rabbitson	5e6aceeb0e	Merge remote-tracking branch 'origin/master' into feat/datamodel-selector-retrieval	2021-10-05 18:15:51 +02:00
Jiaying Wang	cbb147de23	Merge pull request #7405 from filecoin-project/feat/update-graphsync-0.10.0 Update to graphsync to v0.10.0, enable seperate storage and retrieval transfer limits	2021-10-05 11:50:59 -04:00
Rod Vagg	97c2cdb8d6	traversals: limit link load count for SelectiveCar use as well	2021-10-05 10:47:49 +02:00
Rod Vagg	43f7fd5e10	traversals: limit maximum number of DAG links to traverse Impacts CommP and graphsync transfers	2021-10-05 10:47:49 +02:00
Dirk McCormick	a4e2fce9ca	feat: update to go-fil-markets v1.13.1	2021-10-05 10:40:01 +02:00
hannahhoward	368d72ebfe	feat(graphsync): update to v0.10.0-rc1 also add config changes	2021-10-05 14:13:58 +11:00
Peter Rabbitson	af0d9b650d	Error on corner cases with corresponding negative tests	2021-10-04 23:21:24 +02:00
Peter Rabbitson	2a1094fc17	Merge banch feat/go-fil-markets-v1.13.0 into feat/datamodel-selector-retrieval	2021-10-04 21:56:11 +02:00
Dirk McCormick	16973bc2e8	feat: use go-fil-markets@v1.13.0	2021-10-04 15:41:40 +02:00
Jennifer Wang	38e01cba83	Merge branch 'release/v1.12.0' into jen/12tomaster	2021-10-04 02:38:00 -04:00
Aayush Rajasekaran	f004d036dc	Set BatchPreCommitAboveBaseFee correctly	2021-10-01 14:23:41 -04:00
Łukasz Magiera	6fd9d5f28b	Precommit batch balancer support/config	2021-10-01 14:23:41 -04:00
Anton Evangelatov	809289f5ef	add Dealmaking.StartEpochSealingBuffer config	2021-10-01 17:44:15 +02:00
Łukasz Magiera	a8a9818043	Expose storage states on the metrics endpoint	2021-10-01 14:45:01 +02:00
Łukasz Magiera	7c84e22c3e	Add envconfig docs to the config	2021-09-30 18:23:05 +02:00
Aayush Rajasekaran	c3c46e9097	Fix Drand fetching around null tipsets	2021-09-29 11:57:59 -04:00
Mak Muftic	530df4298e	Fix linter errors	2021-09-29 16:49:03 +02:00
Mak Muftic	ea7673618d	Add unit tests for lotus tracer	2021-09-29 16:39:44 +02:00
Matija Petrunic	53c8e68fba	Remove extra tracer dtypes	2021-09-29 16:23:08 +02:00
Matija Petrunic	181b35dc5e	Extend peer score event with score components	2021-09-29 16:20:43 +02:00
Mak Muftic	7b19beaf5d	Fix linter errors	2021-09-29 13:12:42 +02:00
Mak Muftic	3433fa4a6e	Add stringdocs to config props and generate docs	2021-09-29 12:51:43 +02:00
Jennifer Wang	d7690aa719	Merge branch 'releases' into jen/backport	2021-09-29 03:21:19 -04:00
Mak Muftic	36913f4efc	Fix elasticsearch index setup	2021-09-27 13:28:28 +02:00
Matija Petrunic	6c680e306d	Rename json tracer flag	2021-09-24 16:25:41 +02:00
Mak Muftic	30fc2387be	Merge branch 'mpetrun5/lotus-extended-tracer' of github.com:ChainSafe/lotus into mpetrun5/lotus-extended-tracer	2021-09-24 14:16:31 +02:00
Mak Muftic	9332b9f399	Change json config naming	2021-09-24 13:53:31 +02:00
Mak Muftic	4438c4bd87	Move index name to config only and add default value	2021-09-24 13:43:25 +02:00
Mak Muftic	22bbb113e7	Fix schema creation	2021-09-24 12:58:32 +02:00
Mak Muftic	94a1601ab2	Remove hardcoded url schema	2021-09-24 12:40:33 +02:00
Mak Muftic	e46da5a181	Add cli flags	2021-09-24 12:37:16 +02:00
Łukasz Magiera	8e0314b3a3	Merge pull request #7362 from filecoin-project/chore/update-libp2p Update go-libp2p to v0.15.0	2021-09-23 14:24:12 +01:00
Aayush Rajasekaran	b9bfcc4b69	Randomness: Move getters from ChainAPI to StateAPI	2021-09-21 14:20:15 -04:00
Anton Evangelatov	c7e4217693	fix staging area path buildup	2021-09-21 16:45:00 +02:00
Aarsh Shah	89ac8b09b5	update libp2p	2021-09-21 15:10:04 +04:00
Matija Petrunic	0c390d12f7	Append source auth flag to lotus tracer event	2021-09-21 12:54:07 +02:00
Matija Petrunic	d65d9aa6e0	Send peerID peer score as separate events to enable usage of kibana runtime fields	2021-09-21 12:30:50 +02:00
Mak Muftic	082f70757c	Revert additional logging in elastic client	2021-09-20 12:29:12 +02:00
Mak Muftic	a685ccc955	Remove document id	2021-09-17 15:14:39 +02:00
Mak Muftic	0962e73e3e	Add loger to elastic search client	2021-09-17 14:22:57 +02:00
Matija Petrunic	7d0aefacbb	Change peer score event type to avoid exceeding elastisearch field limit	2021-09-17 13:46:26 +02:00
Matija Petrunic	a91ffa60d6	Set peer id event as string in lotus trace event	2021-09-17 12:11:14 +02:00
Mak Muftic	dc79ea363d	Change index name	2021-09-17 11:35:36 +02:00
ldoublewood	eb2bbfebe3	format code by make gen	2021-09-17 00:20:24 +08:00
Mak Muftic	f09f5c2d11	Change elastic document name	2021-09-16 16:49:47 +02:00
Mak Muftic	8e2b474fc7	Use only one document on elastic	2021-09-16 16:44:35 +02:00
Matija Petrunic	9efa495b0e	Write every trace to new line when using json transport	2021-09-16 16:42:18 +02:00
Matija Petrunic	941a0f7153	Update peer score tracker creation to include lotus tracer	2021-09-16 16:31:36 +02:00
Matija Petrunic	9cdc5261c4	Fix lotus tracer being nil when remote tracer is not configured	2021-09-16 15:53:39 +02:00
Matija Petrunic	d067bc9f01	Move creating and opening file to json transport constructor	2021-09-16 15:22:18 +02:00
Matija Petrunic	e2206147cc	Add source auth token to lotus traces	2021-09-16 12:21:57 +02:00
Mak Muftic	c25a5e06cf	Use config string for elasticsearch client	2021-09-16 12:15:42 +02:00
ldoublewood	3a1fc8e7da	fix bug for CommittedCapacitySectorLifetime	2021-09-16 09:25:02 +08:00
Mak Muftic	5cbd2519da	Remove processing elasticsearch result	2021-09-15 16:20:36 +02:00
Mak Muftic	67c1d633c6	Rename jsonEvent to evt	2021-09-15 16:16:12 +02:00
Mak Muftic	add13c2646	Code cleanup	2021-09-15 16:05:44 +02:00
Mak Muftic	71a2ca9e7c	Add new wrapper structure in tracer transport	2021-09-15 16:03:51 +02:00
Mak Muftic	707faf57b5	Code cleanup	2021-09-15 15:11:34 +02:00
Mak Muftic	e20cd0ef27	Propagate error on creating transport	2021-09-15 14:58:36 +02:00
Mak Muftic	cf06096565	Fix tracer initialization	2021-09-15 14:54:20 +02:00
Mak Muftic	bb85ca031d	Add elasticsearch transport	2021-09-15 14:50:27 +02:00
Matija Petrunic	c064ce4852	Add generated documentation	2021-09-15 14:22:48 +02:00
Matija Petrunic	8a657b90a7	Refactor lotus tracer to behave like lp2p tracer	2021-09-15 13:41:56 +02:00
Matija Petrunic	ab86c5419d	Implement json tracer transport	2021-09-15 13:41:03 +02:00
Matija Petrunic	8e6a01de6d	Add configuration in pubsub to trace locally to json file	2021-09-15 13:40:54 +02:00
Aayush Rajasekaran	1d6db306ec	Randomness: Move getters from ChainAPI to StateAPI	2021-09-15 00:03:13 +02:00
Matija Petrunic	d5a4b0637a	Add lotus tracer transport calls	2021-09-10 15:37:38 +02:00
Mak Muftic	c367533a3c	Add transport interface	2021-09-10 12:36:38 +02:00
Peter Rabbitson	0444435589	Expose basic text-based datamodel selector on retrieval Syntaxt of selection is located at https://pkg.go.dev/github.com/ipld/go-ipld-selector-text-lite#SelectorSpecFromPath Example use, assuming that: - The root of the deal is a plain dag-pb unixfs directory - The directory is not sharded - The user wants to retrieve the first entry in that directory lotus client retrieve --miner f0XXXXX --datamodel-path-selector 'Links/0/Hash' bafyROOTCID ~/output For a much more elaborate example see the top of ./itests/deals_partial_retrieval_test.go	2021-09-10 09:44:11 +02:00
Peter Rabbitson	2c5f4386ba	Prepare the retrieval codepath for selectors Slight import-name shuffle, no functional changes	2021-09-09 12:42:58 +02:00
Łukasz Magiera	3a3a4fc413	Merge pull request #7276 from filecoin-project/nonsense/reject-deal-too-many-staging-deals introduce MaxStagingDealsBytes - reject new deals if our staging deals area is full	2021-09-06 10:01:44 -07:00
Anton Evangelatov	9e2129338a	clarify that 0 is unlimited	2021-09-06 17:07:19 +02:00
Anton Evangelatov	0b7793a927	GiB to Bytes	2021-09-06 15:52:25 +02:00
Anton Evangelatov	c69d1db1eb	introduce MaxStagingDealsGiB - reject new deals if our staging deals area is "full"	2021-09-06 15:01:06 +02:00
hannahhoward	b4e9bc50f8	feat(deps): update go-graphsync v0.8.0 Update to go-graphsync v0.8.0 with go-ipld-prime linksystem branch & trusted store.	2021-09-03 16:21:55 +02:00
Łukasz Magiera	31efe8a690	Merge pull request #7255 from filecoin-project/cleanup/consensus chain: Cleanup consensus logic	2021-09-02 19:48:37 +02:00
Łukasz Magiera	30fccaa0bd	fix lint	2021-09-02 18:45:18 +02:00
Łukasz Magiera	95b128b7bc	chain: Cleanup consensus logic	2021-09-02 18:09:37 +02:00
Łukasz Magiera	52e93371bd	builder: Handle chainstore config in ConfigFullNode	2021-08-31 13:25:18 +02:00
Matija Petrunic	b1dafd81b8	Add peer score tracing on pubsub peer score inspect	2021-08-30 14:49:10 +02:00
Matija Petrunic	99fbd7039e	Update pubsub tracking to trace from lotus and lp2p tracers	2021-08-30 14:38:15 +02:00
Matija Petrunic	522858c871	Add lotus tracer interface	2021-08-30 14:35:15 +02:00
Dirk McCormick	7b7a5b0b21	revert: changes to OnDealExpiredOrChanged in #5431 #7201	2021-08-30 10:42:41 +02:00
Łukasz Magiera	3708a7ac30	config: Port some docstrings from go-ipfs	2021-08-27 19:17:32 +02:00
Łukasz Magiera	17b7dcef6f	config for disabling NAT port mapping	2021-08-27 19:14:29 +02:00
dirkmc	77a19774cf	fix events API timeout handling for nil blocks (#7184 )	2021-08-27 09:05:00 +02:00
Łukasz Magiera	1ba427f638	alerting: Address review	2021-08-26 16:09:18 +02:00
Łukasz Magiera	b094e0913d	untangle fsjournal dependencies	2021-08-26 15:44:45 +02:00
Łukasz Magiera	5366821144	api for getting alert states	2021-08-26 15:44:45 +02:00
Łukasz Magiera	81b1dd12f8	Simple alert system; FD limit alerts	2021-08-26 15:44:45 +02:00
Aarsh Shah	928ac2191b	fix throttling bug	2021-08-25 16:19:02 +05:30
Aarsh Shah	e1a056db46	ignore nil throttler	2021-08-24 10:42:00 +05:30
Łukasz Magiera	12875a9664	api/command for encoding actor params	2021-08-20 17:32:57 +02:00
Łukasz Magiera	0beb48f11d	Fix error handling in SectorAddPieceToAny api impl	2021-08-19 12:56:40 +02:00
Jennifer Wang	7ff5844aa2	Merge branch 'releases' into jen/mas	2021-08-17 10:38:36 -04:00
Frrist	518b6f1d41	fix(miner): always create miner deal staging directory (#7098 ) - fixes #7097 Co-authored-by: Raúl Kripalani <raul@protocol.ai> Co-authored-by: Raúl Kripalani <raul@protocol.ai>	2021-08-17 00:16:06 +01:00
Aarsh Shah	d7076778e2	integrate DAG store and CARv2 in deal-making (#6671 ) This commit removes badger from the deal-making processes, and moves to a new architecture with the dagstore as the cental component on the miner-side, and CARv2s on the client-side. Every deal that has been handed off to the sealing subsystem becomes a shard in the dagstore. Shards are mounted via the LotusMount, which teaches the dagstore how to load the related piece when serving retrievals. When the miner starts the Lotus for the first time with this patch, we will perform a one-time migration of all active deals into the dagstore. This is a lightweight process, and it consists simply of registering the shards in the dagstore. Shards are backed by the unsealed copy of the piece. This is currently a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so when it's time to acquire a shard to serve a retrieval, the unsealed CARv1 is joined with its index (safeguarded by the dagstore), to form a read-only blockstore, thus taking the place of the monolithic badger. Data transfers have been adjusted to interface directly with CARv2 files. On inbound transfers (client retrievals, miner storage deals), we stream the received data into a CARv2 ReadWrite blockstore. On outbound transfers (client storage deals, miner retrievals), we serve the data off a CARv2 ReadOnly blockstore. Client-side imports are managed by the refactored imports.Manager component (when not using IPFS integration). Just like it before, we use the go-filestore library to avoid duplicating the data from the original file in the resulting UnixFS DAG (concretely the leaves). However, the target of those imports are what we call "ref-CARv2s": CARv2 files placed under the `$LOTUS_PATH/imports` directory, containing the intermediate nodes in full, and the leaves as positional references to the original file on disk. Client-side retrievals are placed into CARv2 files in the location: `$LOTUS_PATH/retrievals`. A new set of `Dagstore` JSON-RPC operations and `lotus-miner dagstore` subcommands have been introduced on the miner-side to inspect and manage the dagstore. Despite moving to a CARv2-backed system, the IPFS integration has been respected, and it continues to be possible to make storage deals with data held in an IPFS node, and to perform retrievals directly into an IPFS node. NOTE: because the "staging" and "client" Badger blockstores are no longer used, existing imports on the client will be rendered useless. On startup, Lotus will enumerate all imports and print WARN statements on the log for each import that needs to be reimported. These log lines contain these messages: - import lacks carv2 path; import will not work; please reimport - import has missing/broken carv2; please reimport At the end, we will print a "sanity check completed" message indicating the count of imports found, and how many were deemed broken. Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com> Co-authored-by: Raúl Kripalani <raul@protocol.ai> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>	2021-08-16 23:34:32 +01:00
ZenGround0	10615b53ad	Merge pull request #6853 from filecoin-project/frrist/miner-temp-file-path feat(miner deals): create subdir to miner repo for staged deals	2021-08-16 16:19:05 -04:00
ZenGround0	eaff65c00f	Remove network version panic	2021-08-12 23:50:30 -04:00
Raúl Kripalani	f9ea2e834c	address nits.	2021-08-12 22:11:09 -04:00
Raúl Kripalani	85a4f8325f	fix docs and nits.	2021-08-12 22:08:20 -04:00
Anton Evangelatov	45166e760b	fixup	2021-08-12 22:06:11 -04:00
Anton Evangelatov	e90e71456b	add RuntimeSubsystems API method; use it in `lotus-miner info`	2021-08-12 22:05:08 -04:00
ZenGround0	6da26dcefc	Remove network version panic	2021-08-10 13:07:30 -04:00
Steven Allen	57bf5c2143	feat: add ChainGetTipSetAfterHeight This is identical to ChainGetTipSetByHeight, but returns the tipset following any null tipsets. This is what the user usually wants anyways. (and I need it for another PR)	2021-08-05 12:07:14 -07:00
Peter Rabbitson	eefdcb0454	Remove forgotten non-functioning config from the pre-mainnet days	2021-08-02 17:04:23 +02:00
frrist	ca697c9aa2	feat(miner deals): create subdir to miner repo for staged deals	2021-07-29 21:08:38 -07:00
vyzo	1d50dadc18	fix bug in message exclusion	2021-07-29 21:58:59 +03:00
Raúl Kripalani	b3c951c924	add RepoType#String; adjust repo parsing logic.	2021-07-29 12:22:27 -04:00
Raúl Kripalani	8442bac5b2	support MARKETS_API_INFO env var; support markets-repo, markets-api-url flags.	2021-07-29 12:20:05 -04:00
Raúl Kripalani	3144da86f3	add RepoType#String; adjust repo parsing logic.	2021-07-29 13:49:51 +01:00
Raúl Kripalani	4e19d8d562	support MARKETS_API_INFO env var; support markets-repo, markets-api-url flags.	2021-07-29 10:55:37 +01:00
Raúl Kripalani	d8c90b91be	address nits.	2021-07-29 04:41:27 -04:00
Raúl Kripalani	c119ab6ed9	fix docs and nits.	2021-07-29 04:39:28 -04:00
Anton Evangelatov	0dd83c6755	fixup	2021-07-29 04:38:25 -04:00
Anton Evangelatov	9b7a7713a7	add RuntimeSubsystems API method; use it in `lotus-miner info`	2021-07-29 04:38:19 -04:00
Raúl Kripalani	d3742048f9	address nits.	2021-07-28 23:56:18 +01:00
Raúl Kripalani	a61ba7b09c	Merge branch 'master' into nonsense/get-enabled-subsystems-api	2021-07-28 23:55:59 +01:00
Aayush Rajasekaran	21244704fc	Merge branch 'master' into mg/feat/configurable-sector-expirations	2021-07-28 16:45:27 -04:00
raulk	80aedcd276	Merge branch 'master' into nonsense/get-enabled-subsystems-api	2021-07-28 20:09:43 +01:00
Raúl Kripalani	07f40b9488	fix docs and nits.	2021-07-28 20:03:25 +01:00
Łukasz Magiera	389f71251c	Remove townhall	2021-07-28 17:58:46 +02:00
Anton Evangelatov	4cf5c8f656	fixup	2021-07-28 16:23:04 +03:00
Anton Evangelatov	de4a847078	add RuntimeSubsystems API method; use it in `lotus-miner info`	2021-07-28 16:02:05 +03:00
Jiaying Wang	d45b4a7a1f	Merge branch 'master' into chore/cleanup-chain	2021-07-27 17:11:36 -04:00
Łukasz Magiera	9bd312881d	VMSys doesn't belong in chainstore	2021-07-27 15:30:23 +02:00
vyzo	7c195245a1	make cfgdoc-gen	2021-07-27 12:13:26 +03:00
vyzo	9d25464703	use functional options for hotstore gc, rename MovingGC to FullGC	2021-07-27 09:53:22 +03:00
vyzo	aa0bd51b2c	thread GCFrequency option into the splitstore config	2021-07-26 15:05:57 +03:00
vyzo	20f93a520f	make cfgdoc-gen	2021-07-26 15:05:57 +03:00
vyzo	4cdb34e448	add docstrings for splitstore config	2021-07-26 15:05:57 +03:00
vyzo	5acae50e07	add config option for splitstore moving gc frequency	2021-07-26 15:05:57 +03:00
vyzo	a0d6fdba33	add ChainBlockstoreInfo APIv1 endpoint	2021-07-26 08:30:07 +03:00
vyzo	2dc72d5849	satisfy linter who wants to be a spell checker in comments	2021-07-25 11:47:21 +03:00
vyzo	3d2ae433ee	add ChainCheckBlockstore API	2021-07-25 11:14:48 +03:00
Mike Greenberg	e68c59f984	fix: Update node config docs	2021-07-23 21:05:50 -04:00
Mike Greenberg	64d2b7a122	Merge branch 'origin/master' into mg/feat/configurable-sector-expirations * origin/master: (47 commits) Fix links in issue templates Update issue templates and add templates for M1 config: Add newlines consistently config: Add note on codegen to types.go fix lint cli docsgen config: Bring some docs from filecoin-docs add comment about the necessity of FileIO improve README config doc gen: generate formatted code config: Fix doc finding logic Nicer lotus-miner config commands Move doc-comment logic to the config pkg config: doc struct codegen lotus daemon config utils adjust compaction test with badger to hit the db add splitstore compaction test with badger markset update README with configuration options tweak badger options some tweaks in badger markset ... Conflicts: node/config/def.go	2021-07-23 20:36:54 -04:00
Łukasz Magiera	91c6aca7e7	config: Add newlines consistently	2021-07-23 16:59:55 +02:00
Łukasz Magiera	80af6de316	config: Add note on codegen to types.go	2021-07-23 16:02:45 +02:00
Łukasz Magiera	ac2a1bfeab	fix lint	2021-07-23 15:53:31 +02:00
Łukasz Magiera	a26712ebc4	config: Bring some docs from filecoin-docs	2021-07-23 15:40:40 +02:00
Łukasz Magiera	ed7f70e68e	config doc gen: generate formatted code	2021-07-23 15:18:20 +02:00
Łukasz Magiera	3f4973cb41	config: Fix doc finding logic	2021-07-23 15:17:22 +02:00
Łukasz Magiera	8b1f19e94c	Move doc-comment logic to the config pkg	2021-07-23 14:55:19 +02:00
Łukasz Magiera	97f5bb66c7	config: doc struct codegen	2021-07-23 13:55:50 +02:00
Mike Greenberg	adb62a3fff	chore: Move cfg getter into PCPolicy; Clamp values on get	2021-07-22 15:07:03 -04:00
Mike Greenberg	9e34cce1f2	feat(config): support configurable CC sector expiration	2021-07-22 15:07:03 -04:00
Łukasz Magiera	660829703a	Merge remote-tracking branch 'origin/master' into feat/split-net-api	2021-07-22 15:38:06 +02:00
Łukasz Magiera	da5aeda197	Merge branch 'master' into feat/splitstore-hot-messages	2021-07-22 12:58:06 +02:00
Peter Rabbitson	4830a3406b	Introduce the LOTUS_CHAIN_BADGERSTORE_DISABLE_FSYNC envvar Allows the user to control the opts.SyncWrites option of BadgerDs	2021-07-21 22:38:14 +02:00
Anton Evangelatov	fe31956713	fetch peer id from chain	2021-07-21 11:43:19 +02:00
vyzo	839b00ab40	rename messagepool ProtectMessages to ForEachPendingMessage	2021-07-20 09:23:36 +03:00
vyzo	ebbaf23af8	support out-of-chain reference protection	2021-07-20 09:02:40 +03:00
vyzo	1b77361301	add option for hotstore message retention	2021-07-17 08:35:35 +03:00
vyzo	e003203bea	implement exposed splitstore	2021-07-15 13:12:10 +03:00
Łukasz Magiera	812dc266cf	builder: Don't require specific NetAPI impl in StorageMinerAPI	2021-07-15 12:01:13 +02:00
Łukasz Magiera	49e26cce7d	api: Separate the Net interface from Common	2021-07-15 11:41:30 +02:00
hunjixin	bdaa73a413	add ChainGetMessagesInTipset api	2021-07-15 14:52:09 +08:00
Łukasz Magiera	583a8a13d9	Merge pull request #6629 from filecoin-project/feat/pledge-from-miner-balance Config for collateral from miner available balance	2021-07-13 17:22:00 +02:00
Łukasz Magiera	837322ea59	Merge pull request #6356 from filecoin-project/nonsense/split-market-miner-processes Support standalone miner-market process	2021-07-13 17:16:07 +02:00
Łukasz Magiera	7526a074d9	sealing: collateral buffer / falback config	2021-07-13 17:04:58 +02:00
Łukasz Magiera	c37401a1a3	Merge pull request #6474 from filecoin-project/feat/splitstore-redux Splitstore Enhanchements	2021-07-13 12:43:57 +02:00
hannahhoward	92ec239202	fix(node): ClientRetrieve stops on cancel When a deal is cancelled, any in progress calls to ClientRetrieve or ClientRetrieveWithEvents should return	2021-07-12 14:46:18 -07:00
Łukasz Magiera	83f2368507	Add CollateralFromMinerBalance config	2021-07-12 14:11:58 +02:00
Anton Evangelatov	4bc9fa04ba	rename cfg.Subsystems.EnableStorageMarket to EnableMarkets	2021-07-12 12:12:29 +02:00
Anton Evangelatov	056136ef5a	rename argument from url to apiInfo	2021-07-12 11:36:22 +02:00
Anton Evangelatov	a423c46f2a	rename mock.go to net.go	2021-07-12 11:35:03 +02:00
Anton Evangelatov	d89ddb9315	resolve conflicts	2021-07-12 11:34:37 +02:00
Anton Evangelatov	16784aa2cc	remove pieceProvider from DI; small refactors	2021-07-12 11:30:26 +02:00
vyzo	c0a1cfffa1	rename noopstore to discardstore	2021-07-09 19:19:37 +03:00
vyzo	abdf4a161a	explicitly switch marksets for concurrent marking this has very noticeable impact in initial marking time; it also allows us to get rid of the confusing ts monikers.	2021-07-09 04:26:36 +03:00
Łukasz Magiera	ed0cc0b56d	Handle bad peerIDs in MarketListRetrievalDeals	2021-07-08 19:27:03 +02:00
Łukasz Magiera	8f5c0c60f5	Config for deal publishing control addresses	2021-07-07 18:00:54 +02:00
Anton Evangelatov	82efe01c70	fixup	2021-07-07 18:00:28 +02:00
Anton Evangelatov	6b014f57e5	pass Subsystems to StorageMiner option; add enableLibp2p bool in Settings	2021-07-07 13:56:37 +02:00
Anton Evangelatov	42ae876711	remove ApplyIfEnableLibP2P	2021-07-07 13:31:56 +02:00
Anton Evangelatov	7a78527fde	remove not used sa; cleanup	2021-07-06 17:46:21 +02:00
Anton Evangelatov	10dc90f7fe	addressing comments	2021-07-06 17:33:47 +02:00
Anton Evangelatov	898687ea2e	Update node/modules/storageminer.go Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>	2021-07-06 17:28:52 +02:00
Anton Evangelatov	ff2772a58c	resolved conflicts	2021-07-06 16:00:41 +02:00
Łukasz Magiera	97207df705	Merge pull request #6639 from filecoin-project/frrist/context-in-statetree-diff polish(statetree): accept a context in statetree diff for timeouts	2021-07-05 17:30:31 +02:00
Anton Evangelatov	4be0a7a215	resolve conflicts with master	2021-07-05 13:13:32 +02:00
vyzo	190cb18ab0	housekeeping - remove defunct tracking store implementations - update splitstore node config - use mark set type config option (defaulting to mapts); a memory constrained node may want to use an on-disk one	2021-07-04 18:38:28 +03:00
vyzo	5f7ae1f489	update splistore DI constructor	2021-07-04 18:38:28 +03:00
vyzo	68bc5d2291	skip moving cold blocks when running with a noop coldstore it is a noop but it still takes (a lot of) time because it has to read all the cold blocks.	2021-07-04 18:38:28 +03:00
vyzo	649b7dd162	add config option for hot headers	2021-07-04 18:38:28 +03:00
vyzo	997f2c098b	keep headers hot when running with a noop splitstore	2021-07-04 18:38:28 +03:00
vyzo	04f2e102a1	kill full splitstore compaction, simplify splitstore configuration	2021-07-04 18:38:28 +03:00
vyzo	5cca29d1db	hook noop blockstore for splitstore in DI	2021-07-04 18:38:28 +03:00
Łukasz Magiera	8a94ab676e	storage: Fix FinalizeSector with sectors in stoage paths	2021-07-02 19:54:45 +02:00
Anton Evangelatov	9e85492b85	fix test	2021-07-02 12:24:07 +02:00
Anton Evangelatov	96245c3cfa	lint	2021-07-02 12:02:36 +02:00
Anton Evangelatov	4194c7fa24	revert back optional for Miner; remove ActorAddress call in node.MinerHandler	2021-07-02 11:58:31 +02:00
Raúl Kripalani	1869de7662	unwire host from storage.Miner.	2021-07-01 13:53:20 +01:00
Łukasz Magiera	c094aa82ec	commit batch: AggregateAboveBaseFee config	2021-07-01 13:33:54 +02:00
frrist	d02a2a2cfa	polish(statetree): accept a context in statetree diff for timeouts - this operation can take a while, this changes gives users the option to abort if it takes "too long".	2021-06-30 16:40:43 -07:00
Anton Evangelatov	3aa6d03cc1	connect miner to full node , only when running markets	2021-06-30 16:45:11 +02:00
Anton Evangelatov	4f0a96c9c7	resolved conflicts	2021-06-30 13:16:52 +02:00
Anton Evangelatov	9ec7cd7239	enable libp2p options	2021-06-30 12:55:43 +02:00
Łukasz Magiera	e9dd3e8650	Test Miner SimultaneousTransfers	2021-06-28 18:17:29 +02:00
Łukasz Magiera	37c5dd5afc	Miner SimultaneousTransfers config	2021-06-28 14:24:14 +02:00
Łukasz Magiera	3f3336340f	Fix wallet error messages	2021-06-25 10:48:47 +02:00
Łukasz Magiera	c3480dc0e8	Merge remote-tracking branch 'origin/releases' into chore/merge-release	2021-06-24 09:39:11 +02:00
Łukasz Magiera	9521c0b773	Add miner-side MaxDealStartDelay config	2021-06-23 19:45:13 +02:00
Łukasz Magiera	0e4cd3c599	Merge pull request #6563 from filecoin-project/asr/batch-starter Always flush when timer goes off	2021-06-23 19:19:30 +02:00
Aayush Rajasekaran	616e5688fc	Remove MinPreCommitBatch	2021-06-23 12:30:32 -04:00
Łukasz Magiera	fa2be42eb0	Merge remote-tracking branch 'origin/release/v1.10.0' into chore/merge-1.10	2021-06-23 15:51:00 +02:00
Anton Evangelatov	4217ec5308	make auth on remote rpc server optional	2021-06-23 13:27:03 +02:00
Anton Evangelatov	21b51328f9	adding TestDealWithMarketAndMinerNode	2021-06-23 12:44:52 +02:00
Jakub Sztandera	bf82b96fd8	Update default fees for aggregates Resolves #6422 Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-06-22 20:20:01 +02:00
Anton Evangelatov	6720463799	resolve merge conflicts	2021-06-22 11:28:23 +02:00
Raúl Kripalani	b6147fb27f	extern/storage: retype resource filtering strategy to enum.	2021-06-21 20:28:15 +01:00
Anton Evangelatov	577b730415	wip	2021-06-18 18:42:34 +02:00
Łukasz Magiera	097baeb9b0	Make batch deal input test less flaky	2021-06-17 20:52:09 -04:00
Łukasz Magiera	5c8498b603	storagefsm: Fix batch deal packing behavior	2021-06-17 20:50:14 -04:00
Łukasz Magiera	44de67cf7d	Merge pull request #6175 from filecoin-project/feat/dynamic-retreival-pricing Dynamic Retrieval pricing	2021-06-17 10:25:48 +02:00
Anton Evangelatov	d45bb14015	Merge branch 'master' into nonsense/split-market-miner-processes	2021-06-16 18:49:15 +02:00
Aayush Rajasekaran	520a0091f5	Merge branch 'releases' into release/v1.10.0	2021-06-15 17:40:36 -04:00
Łukasz Magiera	0514f38dc4	Merge remote-tracking branch 'origin/master' into chore/merge-1.10	2021-06-14 13:27:53 +02:00
Anton Evangelatov	31e6fb154e	resolve conflicts	2021-06-14 11:27:54 +02:00
aarshkshah1992	4419205b0e	fix compilation	2021-06-14 10:10:29 +05:30
Aarsh Shah	bed2bc73a3	Update node/config/def.go Co-authored-by: raulk <raul@protocol.ai>	2021-06-14 09:42:22 +05:30
Łukasz Magiera	94be3a973a	Don't enable early finalization by default	2021-06-11 11:45:20 +02:00
Łukasz Magiera	ec06f086ef	sealing: Early finalization option	2021-06-11 11:41:28 +02:00
aarshkshah1992	fed5afa704	merge master	2021-06-11 09:35:20 +05:30
Łukasz Magiera	6f80d9f3c2	Merge pull request #6311 from filecoin-project/raulk/itests consolidate integration tests into `itests` package; create test kit; cleanup	2021-06-10 12:33:29 +02:00
Jennifer Wang	8f426b49ec	updated configuration comments for docs	2021-06-09 23:36:44 -04:00
Jennifer	3a1ad50da5	Merge pull request #6440 from filecoin-project/jen/miner-config-doc updated configuration comments for docs	2021-06-09 23:35:04 -04:00
Jennifer Wang	95ad74a156	updated configuration comments for docs	2021-06-09 20:44:20 -04:00
Raúl Kripalani	6d46be53bd	make tests no longer create auth tokens.	2021-06-10 00:15:29 +01:00
Łukasz Magiera	f0a2e97cb5	fee config for sector batching	2021-06-09 14:23:06 -04:00
Raúl Kripalani	81b3c6e2ab	Merge branch 'master' into raulk/itests	2021-06-09 17:34:46 +01:00
Anton Evangelatov	beb5bffaf6	fix log.Error; rename RemoteStore	2021-06-09 13:05:54 +02:00
Łukasz Magiera	78171055e7	fee config for sector batching	2021-06-08 15:45:32 +02:00
Anton Evangelatov	9f3ec82cc1	initial resolution of conflicts	2021-06-08 13:01:22 +02:00
aarshkshah1992	4e9bb16532	changes as per review	2021-06-08 10:09:27 +05:30
aarshkshah1992	16166504f4	merged master	2021-06-08 09:17:40 +05:30
Raúl Kripalani	a274fcc533	Merge branch 'master' into raulk/itests	2021-06-07 23:44:22 +01:00
Raúl Kripalani	db1a61852d	Merge branch 'master' into raulk/itests	2021-06-07 23:17:44 +01:00
Jakub Sztandera	f9acd07987	Update libp2p to 0.14.1 This does not resolve the CI issue but prepares us for it after https://github.com/libp2p/go-libp2p/pull/1116 Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-06-07 18:51:48 +02:00
Łukasz Magiera	fadc79a487	Merge pull request #6280 from filecoin-project/feat/pieceread-outside-scheduler Bypass task scheduler for reading unsealed pieces	2021-06-07 18:11:44 +02:00
Łukasz Magiera	8625da3479	Bump miner/worker api versions	2021-06-07 16:57:24 +02:00
aarshkshah1992	21e6b50294	finished rebasing PR	2021-06-07 16:02:15 +05:30
Anton Evangelatov	de646a5b93	fix node builder	2021-06-07 12:16:10 +02:00
aarshkshah1992	4efaa8d688	unit tests for the remote store Reader	2021-06-07 15:02:52 +05:30
aarshkshah1992	65eb610ec3	docs, logs and green ci	2021-06-07 15:02:49 +05:30
aarshkshah1992	670835fca0	bypass task scheduler for reading unsealed pieces	2021-06-07 15:02:04 +05:30
Lion	e4588aed5c	Fix the doc errors of the sealing config funcs	2021-06-07 12:35:43 +08:00
Dirk McCormick	52b90e4c9d	test: offline deals	2021-06-04 22:42:07 +02:00
Łukasz Magiera	a7746961fb	Merge remote-tracking branch 'origin/master' into feat/list-retrievals	2021-06-04 20:49:01 +02:00
Anton Evangelatov	ded5341da0	fixup test node builder	2021-06-04 17:22:36 +02:00
Anton Evangelatov	ed634bc3a4	rebuild docs	2021-06-04 17:06:55 +02:00
Anton Evangelatov	c4d10bc44e	apply messagepool.NewProviderLite to lite nodes	2021-06-04 16:51:07 +02:00
Anton Evangelatov	6f125cc129	add prover to fullnode and miner builders	2021-06-04 16:26:32 +02:00
Anton Evangelatov	d9a7348ae1	use masters filecoin-ffi	2021-06-04 16:20:40 +02:00
Anton Evangelatov	8bd09e39ca	resolve merge conflicts	2021-06-04 16:17:00 +02:00
Anton Evangelatov	c877166b4a	add messagepool to test node constructor	2021-06-04 15:53:33 +02:00
Anton Evangelatov	88756f3ebf	fix TestAPIDeal tests	2021-06-04 15:41:38 +02:00
aarshkshah1992	303ef15a83	fix ci	2021-06-04 17:33:11 +05:30
Łukasz Magiera	6d9779ecae	Merge remote-tracking branch 'origin/master' into feat/nv13-1.11	2021-06-02 15:07:13 +02:00
Łukasz Magiera	a676da8a7e	Merge remote-tracking branch 'origin/master' into feat/nv13-1.11	2021-06-01 21:07:04 +02:00
Łukasz Magiera	dc642d0b7b	Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11	2021-06-01 21:06:58 +02:00
Dirk McCormick	8997ed508b	feat: update to markets-v1.4.0	2021-06-01 10:16:12 -06:00
Łukasz Magiera	d69032b2f0	Merge pull request #6367 from filecoin-project/fix/robustify-commit-batcher Robustify commit batcher	2021-06-01 17:49:46 +02:00
Łukasz Magiera	5a7f97210f	Merge pull request #6317 from filecoin-project/jen/test-datacap-topup Add test for AddVerifiedClient	2021-06-01 14:41:05 +02:00
Łukasz Magiera	5a9e6c8142	Merge pull request #6364 from filecoin-project/asr/resolve-messages Improve address resolution for messages	2021-06-01 14:38:31 +02:00
Łukasz Magiera	482e1110c2	precommit batcher: Improve error propagation	2021-06-01 14:35:30 +02:00
Łukasz Magiera	cb4eb487f4	commit batcher: Fix min aggregate size check	2021-06-01 12:27:22 +02:00
Łukasz Magiera	9fcb564bef	Make commit batcher more robust	2021-06-01 11:56:19 +02:00
Anton Evangelatov	e4136b0d42	add modules.StorageAuthWithURL to set correct token	2021-06-01 11:42:28 +02:00
Jennifer Wang	fcfc214ed1	Use MockSbBuilder	2021-05-31 20:14:11 -04:00
Aayush Rajasekaran	9ceee6028b	More tweaking of context handling in the messagepool	2021-05-31 18:13:23 -04:00
Aayush Rajasekaran	ed93d0725f	Protect mp.localAddrs and mp.pending behind helper functions	2021-05-31 18:13:23 -04:00
Aayush Rajasekaran	1f03a618f9	Plumb contexts through	2021-05-31 18:13:23 -04:00
Łukasz Magiera	ffa47659a1	Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11	2021-05-31 21:38:34 +02:00
Łukasz Magiera	c3e8eddb9b	Merge remote-tracking branch 'origin/master' into feat/nv13-1.11	2021-05-31 21:24:56 +02:00
Łukasz Magiera	2a1b35e694	Merge pull request #6361 from filecoin-project/asr/fip-0015 Implement FIP-0015	2021-05-31 20:58:21 +02:00
Łukasz Magiera	2d6a159dcc	Merge pull request #6041 from filecoin-project/fix/batch-deal-packing storagefsm: Fix batch deal packing behavior	2021-05-31 20:45:12 +02:00
Aayush Rajasekaran	7fca1c1ee7	Implement FIP-0015	2021-05-31 14:40:54 -04:00
Peter Rabbitson	1e72d8f14c	Reduce noise from 'peer has different genesis' messages After the unification of all networks behind a build-tag, the amount of these warnings has risen sharply. Reduce it to a debug-level, since it isn't actionable to an operator.	2021-05-31 13:43:24 +02:00
Anton Evangelatov	d195a12565	Merge branch 'master' into nonsense/split-market-miner-processes	2021-05-31 11:26:11 +02:00
Łukasz Magiera	9475079b97	Make batch deal input test less flaky	2021-05-30 17:46:22 +02:00
Łukasz Magiera	c8d603557b	storagefsm: Fix batch deal packing behavior	2021-05-30 17:46:22 +02:00
Jennifer Wang	2b08e9f3a6	Add test AddVerifiedClient	2021-05-28 15:41:57 -04:00
Aayush Rajasekaran	fba5c65ffd	Extend the default deal start epoch delay	2021-05-28 12:35:48 -04:00
hannahhoward	19b6dc8d1e	feat(cli): add a list retrievals command Currently, there is no way to inspect retrievals on a client. This adds said command, allow with corresponding APIs	2021-05-27 11:48:25 -07:00
Łukasz Magiera	1e4456138e	Merge master into feat/nv13	2021-05-27 12:28:20 +02:00
Anton Evangelatov	c311520968	add missing messagepool override	2021-05-27 10:28:41 +02:00
Aayush Rajasekaran	d9e86afa24	Tweak client calcDealExpiration to consider genesis miners	2021-05-26 21:46:37 -04:00
Aayush Rajasekaran	21b4741e30	Fix randomness fetching around null blocks	2021-05-26 21:06:06 -04:00
Aayush Rajasekaran	7714537239	Allow starting networks from arbitrary actor versions	2021-05-26 12:48:14 -04:00
Anton Evangelatov	9979de24db	logs and defaults	2021-05-26 12:47:21 +02:00
Łukasz Magiera	43c62f4406	Revert "Allow starting networks from arbitrary actor versions"	2021-05-26 12:33:08 +02:00
Aayush Rajasekaran	cf574ca9a1	Allow starting networks from arbitrary actor versions	2021-05-25 19:30:20 -04:00
Łukasz Magiera	4c6c9a0edb	Fix windowpost/deadline tests	2021-05-25 21:27:32 +02:00
Łukasz Magiera	04658e1cae	fix lint	2021-05-25 16:26:59 +02:00
Łukasz Magiera	d90ab2b81f	config: Document batching fields	2021-05-25 16:20:49 +02:00
aarshkshah1992	78255eac54	test unsealing prices for default pricing strategy	2021-05-24 15:11:50 +05:30
Raúl Kripalani	c3f45cc836	Merge branch 'raulk/refactor-rpc' into raulk/itests	2021-05-23 14:56:54 +01:00
Raúl Kripalani	2337248aa5	rpc: separate handlers/server; refactor miner rpc.	2021-05-23 14:48:42 +01:00
Raúl Kripalani	627248d610	Merge branch 'raulk/refactor-rpc' into raulk/itests	2021-05-23 12:41:25 +01:00
Raúl Kripalani	de454b8e26	add godocs to ServeRPC.	2021-05-23 12:40:33 +01:00
aarshkshah1992	dc6dbc9a11	dpr changes and test based on new unsealing PR	2021-05-22 22:40:21 +05:30
Raúl Kripalani	5b77dbd06f	decouple rpc server and shutdown logic from cmd.	2021-05-21 21:34:17 +01:00
aarshkshah1992	bd9959070e	unit tests for the remote store Reader	2021-05-20 15:25:46 -06:00
aarshkshah1992	73613ee883	docs, logs and green ci	2021-05-20 15:25:46 -06:00
aarshkshah1992	2a40c802ea	bypass task scheduler for reading unsealed pieces	2021-05-20 15:25:46 -06:00
Dirk McCormick	885564fe24	Revert "chore: update go-libp2p" This reverts commit `f7fbaef361`.	2021-05-20 11:22:19 -06:00
Łukasz Magiera	13ff6ed462	Test pledging before nv13 upgrade	2021-05-20 18:38:33 +02:00
Anton Evangelatov	4693c61305	re-arrange NodeBuilder and add storageminer_svc	2021-05-20 13:10:14 +02:00
Anton Evangelatov	76bb424de0	update StorageMinerAPI	2021-05-20 13:01:14 +02:00
Raúl Kripalani	06b8ee5b95	Merge branch 'master' into raulk/itests	2021-05-20 11:56:43 +01:00
Anton Evangelatov	cb603c62d9	update retrievaladapter ; add piece_provider	2021-05-20 12:49:53 +02:00
Anton Evangelatov	2aad7b6979	update sectorstorage.New (Manager) interface	2021-05-20 12:32:29 +02:00
Dirk McCormick	706fdcbb80	feat: increase data transfer timeouts	2021-05-19 12:56:07 -06:00
Łukasz Magiera	3a99f95f8e	Merge pull request #6149 from filecoin-project/feat/markets-1.3 Update to markets 1.3	2021-05-19 20:51:33 +02:00
Raúl Kripalani	c46d4ae529	wip	2021-05-19 17:30:43 +01:00
Łukasz Magiera	2a1b359ede	config: fmt	2021-05-19 15:20:34 +02:00
Łukasz Magiera	a5677d1b7a	ffiwrapper: Separate Prover interface	2021-05-19 15:20:23 +02:00
Łukasz Magiera	2a0c0e379b	Working default batching config	2021-05-19 14:32:41 +02:00
Anton Evangelatov	a989f60e27	add SectorAddPieceToAny and SectorUnsealPiece to StorageMiner iface; model moved to api package - PieceDealInfo, DealSchedule	2021-05-19 13:05:07 +02:00
Łukasz Magiera	5112b9fe2b	Lower default batch slack	2021-05-18 21:04:47 +02:00
Łukasz Magiera	9690bc882c	Test to trigger batching logic	2021-05-18 18:41:42 +02:00
Łukasz Magiera	81b5d8c671	Make things build with both batchers	2021-05-18 16:53:49 +02:00
Łukasz Magiera	c7ba083fa4	Import precommit batcher	2021-05-18 16:51:06 +02:00
vyzo	7512748b56	wire in sealing config values	2021-05-18 15:31:52 +03:00
vyzo	74bad49068	correctly handle commit batch timer	2021-05-18 14:30:47 +03:00
vyzo	357c0868b7	proper config for termination batching and commit wait	2021-05-18 12:20:19 +03:00
vyzo	febf7cf28f	sane config defaults	2021-05-18 12:02:53 +03:00
Aayush Rajasekaran	9a6e601754	Merge pull request #5961 from filecoin-project/feat/stateless-offline-dealflow Introduce stateless offline dealflow, bypassing the FSM/deallists	2021-05-17 12:35:39 -04:00
Łukasz Magiera	506f39b294	WIP: Integrate FIP0013	2021-05-11 22:10:29 -04:00
Peter Rabbitson	d54ed1b0c6	Merge remote-tracking branch 'origin/master' into feat/stateless-offline-dealflow	2021-05-11 04:54:22 +02:00
Steven Allen	f7fbaef361	chore: update go-libp2p From 0.12.0 to 0.14.0. Headline for lotus: faster yamux.	2021-05-10 19:41:57 -07:00
Peter Rabbitson	6f71eab8ce	Adjust lp2p retry params	2021-05-11 04:26:04 +02:00
Jakub Sztandera	18cbdcfc81	Disable checks API on Lotus Lite Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 17:19:17 +02:00
Jakub Sztandera	d777680449	Fix mpool.GetActor for lite node Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:06 +02:00
Jakub Sztandera	8d75da1629	Use MessagePrototype for check API Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:05 +02:00
vyzo	e2d0047a2a	introduce message prototypes This introduces message prototypes to applicable API endpoints, which allows us to invert control of message sending and give the user a chance to intervene with an interactive ui. Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:05 +02:00
vyzo	d782250aba	implement MessagePool.CheckReplaceMessages Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:04 +02:00
vyzo	91e774063e	implement MessagePool.CheckMessages Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:04 +02:00
vyzo	ed61642b3a	implement NodeStatus API Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:04 +02:00
Peter Rabbitson	e648b72d8d	Merge remote-tracking branch 'origin/master' into feat/stateless-offline-dealflow	2021-05-06 15:57:10 +02:00
Dirk McCormick	5b1fab8ffa	feat: markets v1.3	2021-05-06 09:49:44 +02:00
Steven Allen	eb10918470	Merge pull request #6107 from filecoin-project/feat/checkpoint-sync feat: allow checkpointing to forks	2021-04-29 16:30:52 -07:00
Łukasz Magiera	9b87566dbf	Merge pull request #6141 from filecoin-project/split-TestBatchDealInput-from-TestAPIDealFlow flaky tests improvement: separate TestBatchDealInput from TestAPIDealFlow	2021-04-29 19:01:35 +02:00
Steven Allen	f983fa8cd5	Merge pull request #6134 from filecoin-project/asr/emptytsk Use EmptyTSK where appropriate	2021-04-29 09:35:43 -07:00
Anton Evangelatov	c7d74ef16e	separate TestBatchDealInput from TestAPIDealFlow	2021-04-29 17:39:00 +02:00
Łukasz Magiera	a456f394ac	Merge pull request #6007 from filecoin-project/asr/state-list-messages Speed up StateListMessages in some cases	2021-04-29 07:47:26 +02:00
Aayush Rajasekaran	d794b49df3	Use EmptyTSK where appropriate	2021-04-29 01:00:03 -04:00
Steven Allen	69da6a2a29	feat: allow checkpointing to forks Previously, `lotus sync checkpoint` would only checkpoint on the current chain. Now, it can switch to a new fork.	2021-04-28 13:54:26 -07:00
Łukasz Magiera	a24a73bb52	Merge tag 'v1.8.0' into release/v1.9.0	2021-04-27 10:30:33 +02:00
Łukasz Magiera	c4137a438d	Merge commit 'f1ded63d608e1b191d79c780b8065f8cff367c98' into feat/nv12-1.11	2021-04-27 08:22:55 +02:00
Aayush Rajasekaran	f1ded63d60	v4 specs-actors integration, nv12 migration	2021-04-27 02:02:44 -04:00
Reskorey	58bee65bcb	go fmt	2021-04-23 16:10:51 +08:00
Reskorey	7738886936	fix full node painc fix: full node painc when calling RPC Filecoin.ClientRetrieve without the order.Tatol pecified.	2021-04-23 15:43:41 +08:00
Łukasz Magiera	c72eb0ccaf	Merge pull request #6048 from filecoin-project/fix/nonce-getting Fix nonce getting on Lotus lite	2021-04-18 16:25:47 +02:00
Peter Rabbitson	dab2a41792	Raise client MaxFee default The original value is too conservative for the current state of mainnet Resolves (temporarily) #5543	2021-04-17 00:35:40 +02:00
Jakub Sztandera	66c408938b	Fix signature in messagepool, wire in context Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-04-16 18:43:28 +02:00
Jakub Sztandera	77d004ec06	Fix nonce getting on Lotus lite Resolves #5593 #5995 Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-04-15 19:57:09 +02:00
Łukasz Magiera	e76935147f	Merge remote-tracking branch 'origin/master' into feat/v1-api	2021-04-13 14:24:22 +02:00
Aayush Rajasekaran	ff48a70ed9	Speed up StateListMessages in some cases	2021-04-10 02:33:34 -04:00
Łukasz Magiera	88bdb591d5	Merge pull request #6003 from filecoin-project/fix/fbs-di Fix fallback chainstore	2021-04-10 01:58:38 +02:00
Łukasz Magiera	7e4cb9da84	Fix fallback chainstore	2021-04-09 16:11:02 +02:00
Dirk McCormick	4a3261a986	feat: go-fil-markets-v1.2.5-rc1	2021-04-08 14:34:35 +02:00
Peter Rabbitson	25a77d905d	Centralize everything on ipfs/go-log/v2 I am not entirely sure this is right, but everything seems to build...	2021-04-06 15:04:32 +02:00
Łukasz Magiera	e8f28d7b9f	Fix tests	2021-04-06 12:24:58 +02:00
Łukasz Magiera	81bd27911f	Propagate StateMsg api changes	2021-04-05 19:56:53 +02:00
Peter Rabbitson	b79be2a2c2	Merge remote-tracking branch 'origin/master' into feat/stateless-offline-dealflow	2021-04-05 16:06:44 +02:00
Łukasz Magiera	deb2b90b6a	Fix lotus/miner build	2021-04-05 13:23:46 +02:00
Peter Rabbitson	7fddbb528d	Introduce stateless offline dealflow, bypassing the FSM/deallists This is aproposal for an additional flag --manual-stateless-deal and a corresponding API endpoint ClientStatelessDeal. This allows firing off an offline-style deal against a miner without keeping further track of it locally. Not keeping any local state introduces the limitation of requiring free storage deals, as there is nothing to tie the payment channel setup to. Rationale/need for this type of flow is the case of incredibly large sets of data nd deals, where the client and providers have prearranged payment ahead of time, and the client has a separate-from-lotus database of deal inventory. This way the client can use their lotus node merely as a network gateway, without running into any limitations currently present in both lotus as a whole and go-fil-markets in particular. Specific context for this work is filecoin-discover, where the requirement is to onboard ~ 12,000,000 individual deals against a pool of miners with whom the client has prearranged a relationship.	2021-04-05 13:11:10 +02:00
Łukasz Magiera	f4e46c9003	implement v1 api with api modules	2021-04-03 13:20:50 +02:00
Łukasz Magiera	1b32d7f52f	cli: fix build with v1 api changes	2021-04-03 12:55:29 +02:00
Łukasz Magiera	0103d2f621	v1 api: Cleanup message wait/search apis	2021-04-02 13:52:24 +02:00
Łukasz Magiera	c9866da688	Merge remote-tracking branch 'origin/master' into feat/v1-api	2021-04-02 13:15:33 +02:00
Łukasz Magiera	4538550999	Merge remote-tracking branch 'origin/master' into feat/local-retrieval	2021-04-02 12:36:28 +02:00
Anton Evangelatov	df003d44d2	include Stages and DealStages only on ClientDealInfo, not on ClientListDeals	2021-04-01 17:08:01 +03:00
Łukasz Magiera	1a5c4d70f4	Merge remote-tracking branch 'origin/master' into feat/v1-api	2021-04-01 15:33:37 +02:00
Anton Evangelatov	967fbf52c6	resolve conflicts	2021-03-31 23:14:29 +03:00
Łukasz Magiera	dfe8a1c1e2	client: Set the client var in the correct scope	2021-03-31 19:38:02 +02:00
Łukasz Magiera	ee04f9b71c	Merge remote-tracking branch 'origin/master' into nonsense/add-cancel-retrieval-deal-cmd	2021-03-31 19:15:21 +02:00
Łukasz Magiera	10ac9039b4	Merge pull request #5853 from filecoin-project/feat/markets-v1.2.0 update to markets v1.2.x	2021-03-31 18:59:20 +02:00
Łukasz Magiera	f2ab316fb1	Local retrieval support	2021-03-30 21:32:06 +02:00
Anton Evangelatov	139ff4ae64	resolved conflicts	2021-03-30 16:15:42 +03:00
Anton Evangelatov	77bc8d6ca3	remove storedCounters	2021-03-29 18:39:07 +03:00
Anton Evangelatov	8ba0bcb375	Merge branch 'master' into nonsense/cli-show-deals-backup	2021-03-29 18:19:28 +03:00
hunjixin	896303c6a2	fix lint	2021-03-29 10:31:31 +08:00
Raúl Kripalani	fd91c095c4	rename command to cancel-retrieval; rename args to follow Lotus style.	2021-03-26 16:37:46 +00:00
Dirk McCormick	b13f6a3209	fix: cancel retrieval deal - disallow negative deal ID	2021-03-26 09:51:07 +01:00
hunjixin	449b33abee	remote calc winningpost proof	2021-03-26 13:32:03 +08:00
Łukasz Magiera	e003977559	apiclient: Version client funcs	2021-03-25 15:39:48 +01:00
Łukasz Magiera	7a23b411ae	Merge apistruct with the api package	2021-03-25 15:10:08 +01:00
Anton Evangelatov	a202f9d19e	add cancel-retrieval-deal cmd	2021-03-24 18:18:49 +02:00
Dirk McCormick	1359fb6cc4	fix: disable pull channel monitoring for now	2021-03-24 13:56:59 +01:00
Łukasz Magiera	c41777dcd2	API proxy struct codegen (#5854 ) * mostly working api proxy gen * api: Consistent api names * fix docsgen * regenerate api struct * api: expand external interfaces * Add missing gen files * apigen: fix perm detection * api: Move perm tags to the interface * gofmt * worker perms * docsgen * docsgen: ignore tag comments * apigen: add codegen warning * gofmt * missing actor type * docsgen * make linter happy * fix lint * apigen: use directives for tags * docsgen * regen openrpc docs	2021-03-23 13:42:56 +01:00
Dirk McCormick	aee9890ee7	feat: update go-data-transfer and go-fil-markets	2021-03-23 13:28:30 +01:00
Anton Evangelatov	22217b7cae	Merge branch 'master' into nonsense/cli-show-deals	2021-03-23 13:58:26 +02:00
frrist	6caafaeee3	polish(node): define settings opts for 3rd party dep injection - motivating use case running a lotus-sentinel node	2021-03-22 15:37:44 -07:00
Jakub Sztandera	5f672c2ed0	perf: add cache for gas permium estimation Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-03-22 18:06:14 +01:00
Dirk McCormick	9fb1dd43a9	feat: update to markets v1.2.0	2021-03-22 15:31:13 +01:00
Łukasz Magiera	43d9cc36a4	OpenRPC Support (#5843 ) * main: init implement rpc.Discover RPC method This implement the basic functionality for the method over HTTP RPC. Signed-off-by: meows <b5c6@protonmail.com> * main,go.mod,go.sum: init example with go-openrpc-reflect lib Signed-off-by: meows <b5c6@protonmail.com> Conflicts: go.mod go.sum * main: make variable name human-friendly Signed-off-by: meows <b5c6@protonmail.com> * main,go.mod,go.sum: init impl of go-openrp-reflect printing document Signed-off-by: meows <b5c6@protonmail.com> Conflicts: go.mod go.sum * go.mod,go.sum: use go-openrpc-reflect and open-rpc/meta-schema hackforks This is for development only. Versions need to be bumped when they're ready for use as canonical remotes. Signed-off-by: meows <b5c6@protonmail.com> * main,openrpc,main: refactor openrpc supporting code to own package This eliminates code duplication. Signed-off-by: meows <b5c6@protonmail.com> * main: add rpc.Discover to openrpc document Signed-off-by: meows <b5c6@protonmail.com> * openrpc: fix rpc.discover method name casing Also fixes casing stuff for the rest of Filecoin. methods. Signed-off-by: meows <b5c6@protonmail.com> * Revert "main: add rpc.Discover to openrpc document" This reverts commit 116898efb10f33e405ac74acb1aa6daefcd46a62. * main: fix document creation method name This fixes an issue caused with the latest reverting commit. Signed-off-by: meows <b5c6@protonmail.com> * main,docgen,openrpc: refactor to share api parsing, etc as docgen exported stuff Signed-off-by: meows <b5c6@protonmail.com> Makefile: fix docgen refactoring for makefile use of command Signed-off-by: meows <b5c6@protonmail.com> * openrpc: add schema.examples to app reflector There are quite of few of these already registered for the docgen command, so it makes sense to use those! Signed-off-by: meows <b5c6@protonmail.com> * openrpc: init method pairing examples Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump go.mod to use latest meta-schema and openrpc-reflect versions Signed-off-by: meows <b5c6@protonmail.com> * openrpc: init SchemaType mapper function This function will handle the manual configurations for app-specific data types w/r/t their json schema representation. This is useful for cases where the reflect library is unable to provide a sufficient representation automatically. Provided in this commit is an initial implementation for the integerD type (assuming number are represented in the API as hexs), and a commonly used cid.Cid type. Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: tame dependencies by bumping etclabscore/go-openrpc-reflect This removes a problematic dependency on github.com/ethereum/go-ethereum, which was imported as a dependency for a couple github.com/etclabscore/go-openrpc-reflect tests. etclabscore/go-openrpc-reflect v0.0.36 has removed this dependency, so this commit is the result of bumping that version and then running 'go mod tidy' This is in response to a review at https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205 Date: 2020-11-21 06:52:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: add 'miner' arg to openrpc gen cmd This allows the command to EITHER generate the doc for Full or Miner APIs. See comment for usage. Date: 2020-11-21 07:48:05-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgen: add missing examples for Miner API Generating the Miner API OpenRPC doc (via 'go run ./api/openrpc/cmd miner') caused the example logic to panic because some types were missing. This commit adds those missing types, although I'm not an expert in the API so I can't suggest that the example values provided are ideal or well representative. Date: 2020-11-21 07:50:21-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build/openrpc/full.json,build/openrpc/miner.json: add build/openrpc/[full/miner].json docs These will be used as static documents provided by the rpc.discover method. Date: 2020-11-21 07:51:39-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: init go-rice openrpc static assets Date: 2020-11-21 08:23:06-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: remove rpc.discover implementation from runtime plugin Instead of generating the doc on the fly, we're going to serve a static asset. Rel https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205 This removes the runtime implementation from the RPC server construction. Date: 2020-11-21 08:41:20-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,common: add Discover(ctx) method to CommonAPI interface and structs Date: 2020-11-21 08:41:56-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: use rpc server method aliasing for rpc.discover This depends on a currently-forked change at filecoin-project/go-jsonrpc 8350f9463ee451b187d35c492e32f1b999e80210 which establishes this new method RPCServer.AliasMethod. This solves the problem that the OpenRPC spec says that the document should be served at the system extension-prefixed endpoing rpc.discover (not Filecoin.Discover). In fact, the document will be available at BOTH endpoints, but that duplicity is harmless. Date: 2020-11-21 09:18:26-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,build,common: rpc.discover: return json object instead of string Instead of casting the JSON asset from bytes to string, unmarshal it to a map[string]interface{} so the server will provide it as a JSON object. Date: 2020-11-21 09:27:11-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile: merge resolve: docsgen command path Date: 2020-11-22 07:19:36-06:00 Signed-off-by: meows <b5c6@protonmail.com> * apistruct,main,docgen,openrpc: merge resolve: fix func exporteds, signatures Date: 2020-11-22 07:31:03-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: 'get get' auto-bumps version Date: 2020-11-22 07:31:44-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile,docgen,main,build/openrpc: refactor openrpc documentation generation This creates Makefile command docsgen-openrpc-json, and refactors the docsgen command to generate both the markdown and openrpc json documents, redirecting the output of the openrpc json documentation to the build/openrpc/ directory, where those json files will be compiled as static assets via go-rice boxes. The api/openrpc/cmd now uses usage argumentation congruent to that of the docgen command (switching on API context). Date: 2020-11-22 08:01:18-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main,docgen_openrpc: rename api/openrpc -> api/docgen-openrpc Renames the package as well. This is intended to parallel the existing docgen package and command namespacing. Date: 2020-11-22 10:34:46-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,docgen,build,build/openrpc: use typed Discover response Instead of using a map[string]interface{}, use a typed response for the Discover method implementation. This avoids having to set a docgen Example for the generic map[string]interface{} (as an openrpc document) which both pollutes the generic type and lacks useful information for the Discover method example. Date: 2020-11-22 08:31:16-06:00 Signed-off-by: meows <b5c6@protonmail.com> * apistruct,build,main,impl: implement Discover method for Worker and StorageMiner APIs Methods return static compiled assets respective to the APIs. Date: 2020-11-22 08:57:18-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgen_openrpc,build/openrpc: remove timestamping from openrpc doc info This should allow openrpc docs generated at different times to be equal. This is important because the CI (Circle) runs the docgen command and tests that the output and the source are unchanged (via git diff). Date: 2020-11-22 10:47:07-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main,docgen_openrpc,main,build: fix lint issues Fixes goimports, staticcheck, golint issues. Date: 2020-11-22 11:06:46-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc: fix: don't use an underscore in package name (golint) Date: 2020-11-22 11:07:53-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.sum: fix: mod-tidy-check (run 'go mod tidy') Date: 2020-11-22 11:09:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump filecoin-project/go-jsonrpc dep to latest This version includes the necessary RPCServer.AliasMethod method. Date: 2020-11-23 12:16:15-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile,main,build,build/openrpc: init gzipped openrpc static docs Date: 2020-11-24 06:15:06-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: refactor gzip reading Date: 2020-11-24 06:18:34-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: add basic test for openrpc doc from static assets Date: 2020-11-24 06:30:23-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: handle reader Close error This keeps the errcheck linter happy. Date: 2020-11-24 06:33:14-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.sum: run 'go mod tidy' Date: 2020-11-24 06:36:07-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: go mod tidy Tidying up after resolving the merge conflicts with master at go.mod Date: 2020-11-24 06:40:45-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump filecoin-project/go-jsonrpc to latest This is a repeat of 76e6fd2, since the latest merge to master seems to have reverted this. Date: 2020-11-24 06:42:30-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc,build/openrpc: remove method example pairings, improve schema examples Removing method example pairings since they were redundant to schema examples and were not implemented well. Improved schema examples by using the ExampleValue method instead of the map lookup. Made a note in the comment here that this is not ideal, since we have to make a shortcut assumption /workaround by using 'unknown' as the method name and the typea as its own parent. Luckily these values aren't heavily used by the method logic. Date: 2020-11-27 12:57:36-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc: use generic number jsonschema for number types Previously used an integer schema assuming hex encoding. It appears, based on review some of the examples, that this may not be the case. Obvioussly this schema could be more descriptive, but just shooting for mostly likely to be not wrong at this point. Date: 2020-12-15 14:44:37-06:00 Signed-off-by: meows <b5c6@protonmail.com> * cmd/lotus,go.mod,go.sum: maybe fix straggling merge resolution conflicts Date: 2021-01-19 12:30:42-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build/openrpc/full.json.gz,build/openrpc/miner.json.gz,build/openrpc/worker.json.gz: run 'make docsgen' Date: 2021-01-19 12:33:55-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/apistruct,node/impl: (lint) gofmt Date: 2021-01-19 12:39:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen: maybe fix parse error: open ./api: no such file or directory Date: 2021-01-19 12:52:04-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen,build/openrpc: maybe fix no such file error and run 'make docsgen' Date: 2021-01-19 12:55:52-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen: return if AST comment/groupdoc parsing encounters any error This will returns empty comments/docs maps. This should fix issues like: https://app.circleci.com/pipelines/github/filecoin-project/lotus/12445/workflows/4ebadce9-a298-4ad1-939b-f19ef4c0a5bf/jobs/107218 where the environment makes file lookups hard or impossible. Date: 2021-01-19 13:04:58-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api: Don't depend on build/ * make: support parallel docsgen * openrpc gen: Use simple build version * methodgen * goimports Co-authored-by: meows <b5c6@protonmail.com>	2021-03-19 19:22:46 +01:00
Anton Evangelatov	ac576c75d4	amend restart config	2021-03-19 14:08:50 +02:00
Anton Evangelatov	86772a09c7	use new channelmonitor.Config	2021-03-19 13:53:03 +02:00
Anton Evangelatov	9d6c77d93f	remove interactive UX; add `inspect-deal` cmd	2021-03-18 14:53:03 +02:00
Aayush Rajasekaran	a54c6bfb05	Merge pull request #4653 from filecoin-project/4621-delete-wallet Aftering importing a previously deleted key, be able to delete it again.	2021-03-14 20:36:56 -04:00
Aayush Rajasekaran	c3d1534044	Merge pull request #5743 from filecoin-project/feat/keep-unsealed-by-default Default the AlwaysKeepUnsealedCopy flag to true	2021-03-14 15:47:07 -04:00
Łukasz Magiera	1aea1ebb9d	Merge pull request #5801 from filecoin-project/fix/fsm-input-stuck storagefsm: Trigger input processing when below limits	2021-03-12 19:02:15 +01:00
Łukasz Magiera	08472f4fac	batch deal input processing test	2021-03-12 15:41:46 +01:00
Łukasz Magiera	e5226412fb	builder: set modules.PaychAPI correctly	2021-03-12 11:27:00 +01:00
whyrusleeping	cbc7f1c244	fix paychmgr constructor to take an easier to implement interface	2021-03-12 00:39:20 -08:00
whyrusleeping	e0b650d4ec	make rpcstatemanager its own package, for easy reuse	2021-03-11 18:21:18 -08:00
whyrusleeping	bf42a3be5e	move DI stuff for paychmgr into modules	2021-03-11 17:35:31 -08:00
Łukasz Magiera	bd8864ae43	Merge pull request #5755 from filecoin-project/feat/kvlog Metadata datastore log	2021-03-10 14:13:01 +01:00
Łukasz Magiera	05274943c9	Merge pull request #5620 from filecoin-project/feat/disable-owner-worker-fallback miner: Config to disable owner/worker address fallback	2021-03-10 11:21:39 +01:00
Łukasz Magiera	d11f2def6e	Merge branch 'feat/max-deal-collateral-multiplier' into next	2021-03-10 11:13:15 +01:00
Łukasz Magiera	0a513952c8	better DisableWorkerFallback doc	2021-03-09 23:42:08 +01:00
Łukasz Magiera	d4a981f327	fix lint	2021-03-09 23:38:28 +01:00
Łukasz Magiera	ca7e70bf3a	Metadata datastore log	2021-03-09 22:33:01 +01:00
Łukasz Magiera	260ddacc47	Fix wallet import cycle, better trash key backup	2021-03-09 16:52:51 +01:00
jennijuju	49c44b70c2	Aftering importing a previously deleted key, be able to delete it again. Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>	2021-03-09 16:43:01 +01:00
Łukasz Magiera	1c62d7a40f	Merge pull request #5729 from filecoin-project/feat/api-no-dep-build Feat/api no dep build	2021-03-09 16:01:53 +01:00
Łukasz Magiera	5388f3e6f8	Add connmgr metadata to NetPeerInfo	2021-03-09 00:22:48 +01:00
Łukasz Magiera	6591af960f	Merge pull request #4992 from filecoin-project/feat/splitstore hot/cold blockstore segregation (aka. splitstore)	2021-03-08 22:01:06 +01:00
Łukasz Magiera	305c2ec77d	miner: Config to disable owner/worker addcess fallback	2021-03-08 21:33:46 +01:00
Łukasz Magiera	b13046293f	Merge pull request #5735 from filecoin-project/feat/pubsub-ip-colocatio-whitelist Add configuration option for pubsub IPColocationWhitelist subnets	2021-03-08 21:01:39 +01:00
Dirk McCormick	a6c8b56ab6	feat: Default the AlwaysKeepUnsealedCopy flag to true	2021-03-08 16:35:46 +01:00
vyzo	64646de761	add configuration option for pubsub IPColocationWhitelist subnets	2021-03-06 20:05:32 +02:00
vyzo	4a74f752c0	implement extended peer info in net peers cli	2021-03-06 19:14:13 +02:00
whyrusleeping	e7a1d72ba8	extract build/version to api/version, remove api package dep on build	2021-03-05 14:28:13 -08:00
vyzo	6b8c60a659	don't ID wrap the hotstore	2021-03-05 14:46:18 +02:00
vyzo	dd0c308427	move Blockstore config to FullNode, rename to Chainstore and add default for HotStoreType	2021-03-05 14:46:18 +02:00
vyzo	6014273e69	storage miner doesn't need a splitstore	2021-03-05 14:46:18 +02:00
Raúl Kripalani	4b1e1f4b52	rename liveset => markset; rename snoop => tracking store; docs.	2021-03-05 14:46:18 +02:00
Raúl Kripalani	8cfba5b092	renames and polish.	2021-03-05 14:46:18 +02:00
Raúl Kripalani	b1b452bc0f	remove dependency from blockstore/splitstore => chain/store.	2021-03-05 14:46:18 +02:00
Raúl Kripalani	1a804fbdec	move splitstore into blockstore package.	2021-03-05 14:46:18 +02:00
Raúl Kripalani	1b51c10d78	split off lmdb support to a different branch.	2021-03-05 14:46:18 +02:00
vyzo	0fc2f3a26f	fix post-rebase compilation errors	2021-03-05 14:46:18 +02:00
vyzo	97abbe1eca	add (salted) bloom filter liveset	2021-03-05 14:46:17 +02:00
vyzo	73259aa350	add configuration for splitstore and default to a simple compaction algorithm	2021-03-05 14:46:17 +02:00
vyzo	b83994797a	separate LMDB options for hotstore and tracking stores	2021-03-05 14:46:17 +02:00
vyzo	68b6f913c7	propagate useLMDB option to splitstore through DI	2021-03-05 14:46:17 +02:00
vyzo	99c7d8e3eb	more informative names for the hotstore directories	2021-03-05 14:46:17 +02:00
vyzo	a586d42c3b	make hot store DI injectable in the split store, default to badger.	2021-03-05 14:46:17 +02:00
vyzo	622b4f7d9d	hook splitstore into DI	2021-03-05 14:46:17 +02:00
Łukasz Magiera	448813d2fe	Merge pull request #5695 from filecoin-project/feat/segregate-blockstores segregate chain and state blockstores	2021-03-03 10:53:46 +01:00
Dirk McCormick	ced455bba9	feat: add configurable max provider collateral	2021-03-02 10:24:57 +01:00
Łukasz Magiera	8bd5173a54	Merge branch 'next' into refactor/lib/blockstore	2021-03-01 19:04:40 +01:00
Łukasz Magiera	3f5b17dcfc	Merge pull request #5694 from filecoin-project/asr/nits Correct some logs	2021-03-01 18:22:40 +01:00
Raúl Kripalani	3795cc2bd2	segregate chain and state blockstores. This paves the way for better object lifetime management. Concretely, it makes it possible to: - have different stores backing chain and state data. - having the same datastore library, but using different parameters. - attach different caching layers/policies to each class of data, e.g. sizing caches differently. - specifying different retention policies for chain and state data. This separation is important because: - access patterns/frequency of chain and state data are different. - state is derivable from chain, so one could never expunge the chain store, and only retain state objects reachable from the last finality in the state store.	2021-02-28 22:49:44 +00:00
Raúl Kripalani	9f0c68bb70	fix unused import post-merge.	2021-02-28 19:57:37 +00:00
Raúl Kripalani	7f0f7d0b36	Merge branch 'master' into refactor/lib/blockstore	2021-02-28 19:55:23 +00:00
Aayush Rajasekaran	48bce386cd	Correct some logs	2021-02-28 01:18:29 -05:00
Łukasz Magiera	29c9fa3137	Merge pull request #5612 from filecoin-project/feat/cc-fsm-cleanup storagefsm: Cleanup CC sector creation	2021-02-26 11:39:28 +01:00
Łukasz Magiera	e49a412f6d	Merge pull request #5375 from filecoin-project/feat/refactor-fsm-input storagefsm: Rewrite input handling	2021-02-25 14:28:04 +01:00
Łukasz Magiera	26399dba70	Update markets, cbor-gen with soft map decoding	2021-02-19 20:11:43 +01:00
Łukasz Magiera	90b8612cf4	Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input	2021-02-16 22:39:06 +01:00
Łukasz Magiera	39ad3d3b27	Check format in client commP util	2021-02-16 20:48:31 +01:00
Łukasz Magiera	fd90c03018	Roturn SectorID from PledgeSector	2021-02-16 19:16:35 +01:00
Łukasz Magiera	f719765069	storageminer: exit PledgeSector after sectors enter sealing pipeline	2021-02-16 17:41:58 +01:00
Łukasz Magiera	46b880a6b9	storageminer: Skip uninitialized sectors in SectorsList	2021-02-16 17:34:06 +01:00
Łukasz Magiera	fc5e243c92	storagefsm: Cleanup CC sector creation	2021-02-16 17:14:59 +01:00
Łukasz Magiera	dd82729f60	Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input	2021-02-16 13:45:39 +01:00
Dirk McCormick	a6e3856776	Revert "feat: deals - show data transfer %" This reverts commit `b6c9ddccff`.	2021-02-16 12:32:45 +01:00
Łukasz Magiera	7111e924cc	Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input	2021-02-11 13:30:22 +01:00
Łukasz Magiera	95e47cf998	Merge pull request #5553 from filecoin-project/feat/data-xfer-percent show data transfer % for storage deals	2021-02-11 13:29:43 +01:00
Łukasz Magiera	0c6aef221d	Fix error logging format strings	2021-02-11 12:00:26 +01:00
Dirk McCormick	b6c9ddccff	feat: deals - show data transfer %	2021-02-10 09:56:23 +01:00
whyrusleeping	e387f3810e	add code cid to stateReadState output	2021-02-09 14:18:19 -08:00
Łukasz Magiera	e92b8b24b1	Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input	2021-02-09 14:04:21 +01:00
Łukasz Magiera	adeb4cefee	node builder: Cleanup and add some docstrings	2021-02-09 13:52:03 +01:00
Łukasz Magiera	6de62411c6	Merge pull request #5538 from filecoin-project/feat/deal-publisher-force add commands to list pending deals and force publish	2021-02-05 23:00:52 +01:00
Łukasz Magiera	b3f4e50c58	market: APIs to manage pending deals	2021-02-05 18:58:55 +01:00
Łukasz Magiera	2c772d6fda	skip TestDealMining in short mode	2021-02-05 13:18:41 +01:00
Łukasz Magiera	8f603717a6	Merge remote-tracking branch 'origin/master' into steb/refactor-consistent-tipset-methods	2021-02-05 13:08:49 +01:00
whyrusleeping	0efb4ca3a7	add an api endpoint to get pending transactions for multisigs	2021-02-03 20:46:10 -08:00
Łukasz Magiera	ba571794dc	Merge pull request #5309 from filecoin-project/feat/deal-batch-publish batch publish deal messages	2021-02-04 01:19:56 +01:00
Łukasz Magiera	6e7fcb7ee6	Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input	2021-02-03 14:01:56 +01:00

... 12 13 14 15 16 ...

3208 Commits