lotus

Author	SHA1	Message	Date
vyzo	554848fe6b	add resource manager Net(Set)Limit api	2022-01-20 11:36:11 +02:00
vyzo	d05d5bcb70	add resource manager NetStat api	2022-01-20 11:36:11 +02:00
vyzo	8d3f98fe38	instantiate resource manager in DI	2022-01-20 11:36:11 +02:00
vyzo	826cdb2186	use the libp2p connmgr	2022-01-20 11:36:11 +02:00
Łukasz Magiera	b38141601c	Untangle ffi from api	2022-01-18 11:57:04 +01:00
Łukasz Magiera	03fa955e9e	Cleanup imports	2022-01-18 11:37:24 +01:00
Łukasz Magiera	f2496d87c9	post workers: Snapdeals updates	2022-01-18 11:25:04 +01:00
Łukasz Magiera	0c062f9c08	Merge remote-tracking branch 'origin/master' into feat/post-worker	2022-01-18 10:49:42 +01:00
hannahhoward	92d56d82f4	feat(deps): update markets stack update go-fil-markets, go-data-transfer 1.13.0, go-graphsync 0.12.0	2022-01-14 17:21:04 -08:00
vyzo	7e43bfc109	hook the upgrade schedule to splitstore start	2022-01-14 17:14:32 -05:00
zenground0	d6aa17e21f	Snap Deals Integration - FSM handles the actual cc upgrade process including error states - PoSting (winning and window) works over upgraded and upgrading sectors - Integration test and changes to itest framework to reduce flakes - Update CLI to handle new upgrade - Update dependencies	2022-01-14 17:14:32 -05:00
Łukasz Magiera	4a874eff70	post workers: Cleanup, tests	2022-01-14 14:17:52 +01:00
c r	da6752eccb	feat: #7747 sealing: Adding conf variable for capping number of concurrent unsealing jobs (#7884 ) * adding the new variables- now time for logic * putting parameters into right placeS * adding unsealing throttle * fixing linter issues * removing one last thing...	2022-01-13 12:26:13 -06:00
Aarsh Shah	8459d75597	update index provider config	2022-01-13 16:15:43 +04:00
Aarsh Shah	b780a64ed9	send markets multiaddrs to the engine	2022-01-13 15:31:15 +04:00
Aarsh Shah	8c8652e7bb	add a new cli	2022-01-12 18:06:48 +04:00
Aarsh Shah	dec79c70a1	fix compilation	2022-01-12 15:03:46 +04:00
Aarsh Shah	21a8f71859	compiles	2022-01-12 14:59:09 +04:00
Aarsh Shah	edfc2c215c	Merge remote-tracking branch 'origin/main' into feat/cid-to-piece-idx	2022-01-12 12:48:54 +04:00
zenground0	33f2d24f54	Snap Deals Integration - FSM handles the actual cc upgrade process including error states - PoSting (winning and window) works over upgraded and upgrading sectors - Integration test and changes to itest framework to reduce flakes - Update CLI to handle new upgrade - Update dependencies	2022-01-10 15:39:38 +05:30
hannahhoward	1078dff3a9	feat(storageminer): also list data transfers with no id	2021-12-22 13:41:56 -08:00
hannahhoward	8a923fb420	fix(storageminer): fix conditional	2021-12-22 13:41:56 -08:00
hannahhoward	177282d9f9	fix(storageminer): lint/mod-tidy/extra checks	2021-12-22 13:41:55 -08:00
hannahhoward	cddf63efe9	feat(storageminer): add api for transfer diagnostics Add API + CLI for inspecting in depth diagnostics on graphsync transfers with a given peer	2021-12-22 13:41:29 -08:00
Aayush Rajasekaran	1bc987772c	Merge pull request #7818 from filecoin-project/asr/vm-nvfactor Refactor: VM: Remove the NetworkVersionGetter	2021-12-21 12:19:53 -05:00
Darko Brdareski	a54ac2ef79	Remove FS, MEM, GAS leftover test anotations.	2021-12-20 17:00:01 +01:00
Darko Brdareski	86cf17c1f4	Remove bad annotation from gas_test.go	2021-12-20 16:51:09 +01:00
Darko Brdareski	dda1a42a2a	Merge branch 'bloxico/system-test-matrix' of https://github.com/filecoin-project/lotus into merge_lotus	2021-12-20 15:48:16 +01:00
Aayush Rajasekaran	3f4eaa99d5	Refactor: State: Rename stmgr::GetNtwkVersion to GetNetworkVersion	2021-12-17 18:43:39 -05:00
Dirk McCormick	b1734f84b3	feat: retrieval ask CLI command	2021-12-17 15:55:12 +01:00
Łukasz Magiera	a4728d3c72	Merge pull request #7578 from filecoin-project/rvagg/SimultaneousTransfersForStoragePerClient feat(graphsync): allow setting of per-peer incoming requests for miners	2021-12-17 14:27:15 +01:00
vyzo	dd327f0b22	plumb more contexts	2021-12-17 11:42:09 +02:00
Rod Vagg	9e7d9affbe	feat(graphsync): allow setting of per-peer incoming requests for miners	2021-12-17 15:04:16 +11:00
Darko Brdareski	3e32aa896c	Annotate client storage deals feature	2021-12-16 14:36:02 +01:00
vyzo	22ce395130	fix some unit tests	2021-12-14 17:05:10 +02:00
vyzo	716b4a3b09	hook a great context in the sky to satisfy DI; sigh...	2021-12-14 16:57:09 +02:00
vyzo	eb48dc9b68	fix issues with new peerstore constructor signature in DI	2021-12-14 16:21:55 +02:00
vyzo	f157ac9b68	update for new ConnManager constructor	2021-12-14 15:30:29 +02:00
Dirk McCormick	a51b08e386	feat: add IndexerProvider AnnounceAddress config	2021-12-13 18:24:05 +01:00
Masih H. Derkani	6b7b9793ab	Set default index provider ingest config values Set default values for index-provider ingest config. Leave a TODO to use provider config APIs once defaults exposed. Run `go mod tidy`	2021-12-13 14:44:56 +00:00
Darko Brdareski	0addca1070	Fix bad annotations	2021-12-13 13:41:04 +01:00
vyzo	84710cf27e	plumb more contexts in lotus	2021-12-13 14:26:59 +02:00
Aayush Rajasekaran	dfb65ed89f	Plumb contexts through	2021-12-11 17:04:00 -05:00
vyzo	5d6398f20e	hook the upgrade schedule to splitstore start	2021-12-03 12:11:54 +02:00
Łukasz Magiera	f39942283b	Merge pull request #7706 from filecoin-project/feat/ret-matchding-cars retrieval: Only output matching nodes, MatchPath dagspec	2021-11-29 22:14:25 +01:00
Łukasz Magiera	d019853687	review: Cleanup some comments Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>	2021-11-29 21:29:00 +01:00
Łukasz Magiera	5b5e6b9e44	retrieval: DagSpec.MatchPath -> ExportMerkleProof	2021-11-29 21:14:00 +01:00
Łukasz Magiera	61791b90ea	retrieval: Only output matching nodes, MatchPath dagspec	2021-11-29 20:40:55 +01:00
Steven Allen	797147097c	Merge pull request #7689 from filecoin-project/disable-mplex disable mplex stream muxer	2021-11-29 09:46:24 -08:00
Łukasz Magiera	77d75b7739	Merge pull request #7688 from filecoin-project/chore/partret_cleanup_comment_and_flow Cleanup partial retrieval codepaths ( zero functional changes )	2021-11-29 16:39:55 +01:00
Łukasz Magiera	8d955d5f30	dagstore mount: Add random access support	2021-11-26 17:40:53 +01:00
Marten Seemann	cf0faf58dc	remove muxer config via LIBP2P_MUX_PREFS env	2021-11-26 11:01:54 +04:00
Marten Seemann	12dc3baff9	disable mplex stream muxer	2021-11-25 09:38:57 -08:00
Peter Rabbitson	d5b4d93575	Cleanup partial retrieval codepaths ( zero functional changes ) - Adjust the ExportRef comment to reflect change from #7673 - Simplify control flow in parseDagSpec() ( read diff under -w )	2021-11-25 10:35:05 +01:00
Łukasz Magiera	0c884734a3	Merge pull request #7673 from filecoin-project/chore/DM-level-tests_plus_merkle-proof-cars Chore/dm level tests plus merkle proof cars	2021-11-24 20:43:39 +01:00
Łukasz Magiera	bd4927b494	retrieval: Cleanup some comments	2021-11-24 20:13:49 +01:00
Peter Rabbitson	46937a1b9f	Add tests cementing the DM-level behavior on simple paths	2021-11-24 15:31:15 +01:00
Łukasz Magiera	78949d6c05	retrieval: Fix deadlock in ClientGetRetrievalUpdates	2021-11-23 18:32:56 +01:00
Łukasz Magiera	407c2ed114	retrieval: Drop the RootSelector hack	2021-11-23 17:42:43 +01:00
Łukasz Magiera	25e89d3a7a	retrieval: require unixfs exports to be aligned on block boundary	2021-11-22 13:29:09 +01:00
Łukasz Magiera	9ea229ed5a	retrieval: fix defult ask	2021-11-22 13:04:12 +01:00
Łukasz Magiera	2c583b03ff	retrieval: Support DM-paths in ls	2021-11-22 12:46:29 +01:00
Łukasz Magiera	74f645a098	retrieval: Fix parseDagSpec root check	2021-11-22 12:46:29 +01:00
Łukasz Magiera	597b72e286	retrieval: Fix lint, cli docsgen	2021-11-22 12:46:29 +01:00
Łukasz Magiera	9c119bfdad	retrieval: Make the ls command work	2021-11-22 12:46:17 +01:00
Łukasz Magiera	450d0687da	retrieval: REST export endpoint	2021-11-22 12:46:17 +01:00
Łukasz Magiera	b83a9b902a	gofmt	2021-11-22 12:46:17 +01:00
Łukasz Magiera	b26906963b	retrieval: Support multi-root export	2021-11-22 12:46:17 +01:00
Łukasz Magiera	b9bd061bdd	fix unixfs selector root node selection	2021-11-22 12:46:04 +01:00
Łukasz Magiera	b868769ec8	more retrieval api work	2021-11-22 12:46:02 +01:00
Łukasz Magiera	89138bab4d	Simplify retrieval APIs	2021-11-22 12:45:42 +01:00
Łukasz Magiera	60ea33b1c7	client: cleanup clientRetrieve	2021-11-22 12:45:42 +01:00
Łukasz Magiera	08e297a217	client: Support json selectors in retrieval	2021-11-22 12:45:42 +01:00
Aayush Rajasekaran	32fc03886d	CLI: Add a lotus multisig cancel command	2021-11-18 19:21:17 -05:00
Aarsh Shah	7655e660f3	integrate store-the-hash	2021-11-18 12:07:12 +04:00
Anton Evangelatov	5454aebf13	CLI tools for the DAGStore inverted index (#7361 ) * cli commands for dagstore * address comments from Dirk * rename inverted index to piece index	2021-11-17 15:39:45 +02:00
Dirk McCormick	501308239f	feat: CLI command to announce deal to indexers	2021-11-17 12:16:22 +01:00
Aarsh Shah	8d3017febb	make gen should work	2021-11-17 15:02:11 +04:00
Aarsh Shah	caf3cfa7d8	Merge branch 'feat/cid-to-piece-idx' into feat/indexer-provider	2021-11-17 11:43:49 +04:00
Aarsh Shah	bfee6699d2	update go mod and inject store the hash index in dagstore	2021-11-17 11:42:14 +04:00
Dirk McCormick	627d4fbf74	wip: storage miner index provider	2021-11-16 17:45:42 +01:00
Matija Petrunić	df17ea06f5	Merge branch 'master' into libp2p-pubsub-tracer	2021-11-10 12:47:09 +00:00
Matija Petrunic	59040d4419	Trace reject messages in pubsub	2021-11-10 13:45:40 +01:00
TheMenko	b45924a234	annotated repo_test	2021-11-01 10:49:30 +01:00
TheMenko	fb911a4542	annotated repo_test	2021-11-01 01:24:26 +01:00
TheMenko	9297e1b90d	annotated repo_test	2021-10-31 12:24:07 +01:00
Łukasz Magiera	d2e9d21031	Gather graphsync metrics on provider side as well	2021-10-19 19:45:25 +02:00
Łukasz Magiera	32a855b984	Fix lint	2021-10-19 19:22:32 +02:00
Łukasz Magiera	9a993d25d0	Collect and expose graphsync metrics	2021-10-19 19:20:00 +02:00
Jiaying Wang	afe54ab5f3	Merge pull request #7511 from filecoin-project/frrist/fix-lily fix: support node instantiation in external packages	2021-10-13 20:33:17 -04:00
frrist	6cd6732225	fix: support node instantiation in external packages - implement ChainGetPath on ChainModuleAPI	2021-10-13 15:24:28 -07:00
Anton Evangelatov	6e5ccc87cf	cli: add retry for deals stuck in Publish with no funds	2021-10-11 14:03:55 +02:00
Peter Rabbitson	a40aa9f90b	Normlize selector use within lotus	2021-10-07 11:01:29 +02:00
Peter Rabbitson	f8ac98bf46	Adjust error wording	2021-10-05 18:16:14 +02:00
Peter Rabbitson	5e6aceeb0e	Merge remote-tracking branch 'origin/master' into feat/datamodel-selector-retrieval	2021-10-05 18:15:51 +02:00
Jiaying Wang	cbb147de23	Merge pull request #7405 from filecoin-project/feat/update-graphsync-0.10.0 Update to graphsync to v0.10.0, enable seperate storage and retrieval transfer limits	2021-10-05 11:50:59 -04:00
Rod Vagg	97c2cdb8d6	traversals: limit link load count for SelectiveCar use as well	2021-10-05 10:47:49 +02:00
Rod Vagg	43f7fd5e10	traversals: limit maximum number of DAG links to traverse Impacts CommP and graphsync transfers	2021-10-05 10:47:49 +02:00
Dirk McCormick	a4e2fce9ca	feat: update to go-fil-markets v1.13.1	2021-10-05 10:40:01 +02:00
hannahhoward	368d72ebfe	feat(graphsync): update to v0.10.0-rc1 also add config changes	2021-10-05 14:13:58 +11:00
Peter Rabbitson	af0d9b650d	Error on corner cases with corresponding negative tests	2021-10-04 23:21:24 +02:00
Peter Rabbitson	2a1094fc17	Merge banch feat/go-fil-markets-v1.13.0 into feat/datamodel-selector-retrieval	2021-10-04 21:56:11 +02:00
Dirk McCormick	16973bc2e8	feat: use go-fil-markets@v1.13.0	2021-10-04 15:41:40 +02:00
Jennifer Wang	38e01cba83	Merge branch 'release/v1.12.0' into jen/12tomaster	2021-10-04 02:38:00 -04:00
Aayush Rajasekaran	f004d036dc	Set BatchPreCommitAboveBaseFee correctly	2021-10-01 14:23:41 -04:00
Łukasz Magiera	6fd9d5f28b	Precommit batch balancer support/config	2021-10-01 14:23:41 -04:00
Anton Evangelatov	809289f5ef	add Dealmaking.StartEpochSealingBuffer config	2021-10-01 17:44:15 +02:00
Łukasz Magiera	a8a9818043	Expose storage states on the metrics endpoint	2021-10-01 14:45:01 +02:00
Łukasz Magiera	7c84e22c3e	Add envconfig docs to the config	2021-09-30 18:23:05 +02:00
Aayush Rajasekaran	c3c46e9097	Fix Drand fetching around null tipsets	2021-09-29 11:57:59 -04:00
Mak Muftic	530df4298e	Fix linter errors	2021-09-29 16:49:03 +02:00
Mak Muftic	ea7673618d	Add unit tests for lotus tracer	2021-09-29 16:39:44 +02:00
Matija Petrunic	53c8e68fba	Remove extra tracer dtypes	2021-09-29 16:23:08 +02:00
Matija Petrunic	181b35dc5e	Extend peer score event with score components	2021-09-29 16:20:43 +02:00
Mak Muftic	7b19beaf5d	Fix linter errors	2021-09-29 13:12:42 +02:00
Mak Muftic	3433fa4a6e	Add stringdocs to config props and generate docs	2021-09-29 12:51:43 +02:00
Jennifer Wang	d7690aa719	Merge branch 'releases' into jen/backport	2021-09-29 03:21:19 -04:00
Mak Muftic	36913f4efc	Fix elasticsearch index setup	2021-09-27 13:28:28 +02:00
Matija Petrunic	6c680e306d	Rename json tracer flag	2021-09-24 16:25:41 +02:00
Mak Muftic	30fc2387be	Merge branch 'mpetrun5/lotus-extended-tracer' of github.com:ChainSafe/lotus into mpetrun5/lotus-extended-tracer	2021-09-24 14:16:31 +02:00
Mak Muftic	9332b9f399	Change json config naming	2021-09-24 13:53:31 +02:00
Mak Muftic	4438c4bd87	Move index name to config only and add default value	2021-09-24 13:43:25 +02:00
Mak Muftic	22bbb113e7	Fix schema creation	2021-09-24 12:58:32 +02:00
Mak Muftic	94a1601ab2	Remove hardcoded url schema	2021-09-24 12:40:33 +02:00
Mak Muftic	e46da5a181	Add cli flags	2021-09-24 12:37:16 +02:00
Łukasz Magiera	8e0314b3a3	Merge pull request #7362 from filecoin-project/chore/update-libp2p Update go-libp2p to v0.15.0	2021-09-23 14:24:12 +01:00
Aayush Rajasekaran	b9bfcc4b69	Randomness: Move getters from ChainAPI to StateAPI	2021-09-21 14:20:15 -04:00
Anton Evangelatov	c7e4217693	fix staging area path buildup	2021-09-21 16:45:00 +02:00
Aarsh Shah	89ac8b09b5	update libp2p	2021-09-21 15:10:04 +04:00
Matija Petrunic	0c390d12f7	Append source auth flag to lotus tracer event	2021-09-21 12:54:07 +02:00
Matija Petrunic	d65d9aa6e0	Send peerID peer score as separate events to enable usage of kibana runtime fields	2021-09-21 12:30:50 +02:00
Mak Muftic	082f70757c	Revert additional logging in elastic client	2021-09-20 12:29:12 +02:00
Mak Muftic	a685ccc955	Remove document id	2021-09-17 15:14:39 +02:00
Mak Muftic	0962e73e3e	Add loger to elastic search client	2021-09-17 14:22:57 +02:00
Matija Petrunic	7d0aefacbb	Change peer score event type to avoid exceeding elastisearch field limit	2021-09-17 13:46:26 +02:00
Matija Petrunic	a91ffa60d6	Set peer id event as string in lotus trace event	2021-09-17 12:11:14 +02:00
Mak Muftic	dc79ea363d	Change index name	2021-09-17 11:35:36 +02:00
ldoublewood	eb2bbfebe3	format code by make gen	2021-09-17 00:20:24 +08:00
Mak Muftic	f09f5c2d11	Change elastic document name	2021-09-16 16:49:47 +02:00
Mak Muftic	8e2b474fc7	Use only one document on elastic	2021-09-16 16:44:35 +02:00
Matija Petrunic	9efa495b0e	Write every trace to new line when using json transport	2021-09-16 16:42:18 +02:00
Matija Petrunic	941a0f7153	Update peer score tracker creation to include lotus tracer	2021-09-16 16:31:36 +02:00
Matija Petrunic	9cdc5261c4	Fix lotus tracer being nil when remote tracer is not configured	2021-09-16 15:53:39 +02:00
Matija Petrunic	d067bc9f01	Move creating and opening file to json transport constructor	2021-09-16 15:22:18 +02:00
Matija Petrunic	e2206147cc	Add source auth token to lotus traces	2021-09-16 12:21:57 +02:00
Mak Muftic	c25a5e06cf	Use config string for elasticsearch client	2021-09-16 12:15:42 +02:00
ldoublewood	3a1fc8e7da	fix bug for CommittedCapacitySectorLifetime	2021-09-16 09:25:02 +08:00
Mak Muftic	5cbd2519da	Remove processing elasticsearch result	2021-09-15 16:20:36 +02:00
Mak Muftic	67c1d633c6	Rename jsonEvent to evt	2021-09-15 16:16:12 +02:00
Mak Muftic	add13c2646	Code cleanup	2021-09-15 16:05:44 +02:00
Mak Muftic	71a2ca9e7c	Add new wrapper structure in tracer transport	2021-09-15 16:03:51 +02:00
Mak Muftic	707faf57b5	Code cleanup	2021-09-15 15:11:34 +02:00
Mak Muftic	e20cd0ef27	Propagate error on creating transport	2021-09-15 14:58:36 +02:00
Mak Muftic	cf06096565	Fix tracer initialization	2021-09-15 14:54:20 +02:00
Mak Muftic	bb85ca031d	Add elasticsearch transport	2021-09-15 14:50:27 +02:00
Matija Petrunic	c064ce4852	Add generated documentation	2021-09-15 14:22:48 +02:00
Matija Petrunic	8a657b90a7	Refactor lotus tracer to behave like lp2p tracer	2021-09-15 13:41:56 +02:00
Matija Petrunic	ab86c5419d	Implement json tracer transport	2021-09-15 13:41:03 +02:00
Matija Petrunic	8e6a01de6d	Add configuration in pubsub to trace locally to json file	2021-09-15 13:40:54 +02:00
Aayush Rajasekaran	1d6db306ec	Randomness: Move getters from ChainAPI to StateAPI	2021-09-15 00:03:13 +02:00
Matija Petrunic	d5a4b0637a	Add lotus tracer transport calls	2021-09-10 15:37:38 +02:00
Mak Muftic	c367533a3c	Add transport interface	2021-09-10 12:36:38 +02:00
Peter Rabbitson	0444435589	Expose basic text-based datamodel selector on retrieval Syntaxt of selection is located at https://pkg.go.dev/github.com/ipld/go-ipld-selector-text-lite#SelectorSpecFromPath Example use, assuming that: - The root of the deal is a plain dag-pb unixfs directory - The directory is not sharded - The user wants to retrieve the first entry in that directory lotus client retrieve --miner f0XXXXX --datamodel-path-selector 'Links/0/Hash' bafyROOTCID ~/output For a much more elaborate example see the top of ./itests/deals_partial_retrieval_test.go	2021-09-10 09:44:11 +02:00
Peter Rabbitson	2c5f4386ba	Prepare the retrieval codepath for selectors Slight import-name shuffle, no functional changes	2021-09-09 12:42:58 +02:00
Łukasz Magiera	3a3a4fc413	Merge pull request #7276 from filecoin-project/nonsense/reject-deal-too-many-staging-deals introduce MaxStagingDealsBytes - reject new deals if our staging deals area is full	2021-09-06 10:01:44 -07:00
Anton Evangelatov	9e2129338a	clarify that 0 is unlimited	2021-09-06 17:07:19 +02:00
Anton Evangelatov	0b7793a927	GiB to Bytes	2021-09-06 15:52:25 +02:00
Anton Evangelatov	c69d1db1eb	introduce MaxStagingDealsGiB - reject new deals if our staging deals area is "full"	2021-09-06 15:01:06 +02:00
hannahhoward	b4e9bc50f8	feat(deps): update go-graphsync v0.8.0 Update to go-graphsync v0.8.0 with go-ipld-prime linksystem branch & trusted store.	2021-09-03 16:21:55 +02:00
Łukasz Magiera	31efe8a690	Merge pull request #7255 from filecoin-project/cleanup/consensus chain: Cleanup consensus logic	2021-09-02 19:48:37 +02:00
Łukasz Magiera	30fccaa0bd	fix lint	2021-09-02 18:45:18 +02:00
Łukasz Magiera	95b128b7bc	chain: Cleanup consensus logic	2021-09-02 18:09:37 +02:00
Łukasz Magiera	52e93371bd	builder: Handle chainstore config in ConfigFullNode	2021-08-31 13:25:18 +02:00
Matija Petrunic	b1dafd81b8	Add peer score tracing on pubsub peer score inspect	2021-08-30 14:49:10 +02:00
Matija Petrunic	99fbd7039e	Update pubsub tracking to trace from lotus and lp2p tracers	2021-08-30 14:38:15 +02:00
Matija Petrunic	522858c871	Add lotus tracer interface	2021-08-30 14:35:15 +02:00
Dirk McCormick	7b7a5b0b21	revert: changes to OnDealExpiredOrChanged in #5431 #7201	2021-08-30 10:42:41 +02:00
Łukasz Magiera	3708a7ac30	config: Port some docstrings from go-ipfs	2021-08-27 19:17:32 +02:00
Łukasz Magiera	17b7dcef6f	config for disabling NAT port mapping	2021-08-27 19:14:29 +02:00
dirkmc	77a19774cf	fix events API timeout handling for nil blocks (#7184 )	2021-08-27 09:05:00 +02:00
Łukasz Magiera	1ba427f638	alerting: Address review	2021-08-26 16:09:18 +02:00
Łukasz Magiera	b094e0913d	untangle fsjournal dependencies	2021-08-26 15:44:45 +02:00
Łukasz Magiera	5366821144	api for getting alert states	2021-08-26 15:44:45 +02:00
Łukasz Magiera	81b1dd12f8	Simple alert system; FD limit alerts	2021-08-26 15:44:45 +02:00
Aarsh Shah	928ac2191b	fix throttling bug	2021-08-25 16:19:02 +05:30
Aarsh Shah	e1a056db46	ignore nil throttler	2021-08-24 10:42:00 +05:30
Łukasz Magiera	12875a9664	api/command for encoding actor params	2021-08-20 17:32:57 +02:00
Łukasz Magiera	0beb48f11d	Fix error handling in SectorAddPieceToAny api impl	2021-08-19 12:56:40 +02:00
Jennifer Wang	7ff5844aa2	Merge branch 'releases' into jen/mas	2021-08-17 10:38:36 -04:00
Frrist	518b6f1d41	fix(miner): always create miner deal staging directory (#7098 ) - fixes #7097 Co-authored-by: Raúl Kripalani <raul@protocol.ai> Co-authored-by: Raúl Kripalani <raul@protocol.ai>	2021-08-17 00:16:06 +01:00
Aarsh Shah	d7076778e2	integrate DAG store and CARv2 in deal-making (#6671 ) This commit removes badger from the deal-making processes, and moves to a new architecture with the dagstore as the cental component on the miner-side, and CARv2s on the client-side. Every deal that has been handed off to the sealing subsystem becomes a shard in the dagstore. Shards are mounted via the LotusMount, which teaches the dagstore how to load the related piece when serving retrievals. When the miner starts the Lotus for the first time with this patch, we will perform a one-time migration of all active deals into the dagstore. This is a lightweight process, and it consists simply of registering the shards in the dagstore. Shards are backed by the unsealed copy of the piece. This is currently a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so when it's time to acquire a shard to serve a retrieval, the unsealed CARv1 is joined with its index (safeguarded by the dagstore), to form a read-only blockstore, thus taking the place of the monolithic badger. Data transfers have been adjusted to interface directly with CARv2 files. On inbound transfers (client retrievals, miner storage deals), we stream the received data into a CARv2 ReadWrite blockstore. On outbound transfers (client storage deals, miner retrievals), we serve the data off a CARv2 ReadOnly blockstore. Client-side imports are managed by the refactored imports.Manager component (when not using IPFS integration). Just like it before, we use the go-filestore library to avoid duplicating the data from the original file in the resulting UnixFS DAG (concretely the leaves). However, the target of those imports are what we call "ref-CARv2s": CARv2 files placed under the `$LOTUS_PATH/imports` directory, containing the intermediate nodes in full, and the leaves as positional references to the original file on disk. Client-side retrievals are placed into CARv2 files in the location: `$LOTUS_PATH/retrievals`. A new set of `Dagstore` JSON-RPC operations and `lotus-miner dagstore` subcommands have been introduced on the miner-side to inspect and manage the dagstore. Despite moving to a CARv2-backed system, the IPFS integration has been respected, and it continues to be possible to make storage deals with data held in an IPFS node, and to perform retrievals directly into an IPFS node. NOTE: because the "staging" and "client" Badger blockstores are no longer used, existing imports on the client will be rendered useless. On startup, Lotus will enumerate all imports and print WARN statements on the log for each import that needs to be reimported. These log lines contain these messages: - import lacks carv2 path; import will not work; please reimport - import has missing/broken carv2; please reimport At the end, we will print a "sanity check completed" message indicating the count of imports found, and how many were deemed broken. Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com> Co-authored-by: Raúl Kripalani <raul@protocol.ai> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>	2021-08-16 23:34:32 +01:00
ZenGround0	10615b53ad	Merge pull request #6853 from filecoin-project/frrist/miner-temp-file-path feat(miner deals): create subdir to miner repo for staged deals	2021-08-16 16:19:05 -04:00
ZenGround0	eaff65c00f	Remove network version panic	2021-08-12 23:50:30 -04:00
Raúl Kripalani	f9ea2e834c	address nits.	2021-08-12 22:11:09 -04:00
Raúl Kripalani	85a4f8325f	fix docs and nits.	2021-08-12 22:08:20 -04:00
Anton Evangelatov	45166e760b	fixup	2021-08-12 22:06:11 -04:00
Anton Evangelatov	e90e71456b	add RuntimeSubsystems API method; use it in `lotus-miner info`	2021-08-12 22:05:08 -04:00
ZenGround0	6da26dcefc	Remove network version panic	2021-08-10 13:07:30 -04:00
Steven Allen	57bf5c2143	feat: add ChainGetTipSetAfterHeight This is identical to ChainGetTipSetByHeight, but returns the tipset following any null tipsets. This is what the user usually wants anyways. (and I need it for another PR)	2021-08-05 12:07:14 -07:00
Peter Rabbitson	eefdcb0454	Remove forgotten non-functioning config from the pre-mainnet days	2021-08-02 17:04:23 +02:00
frrist	ca697c9aa2	feat(miner deals): create subdir to miner repo for staged deals	2021-07-29 21:08:38 -07:00
vyzo	1d50dadc18	fix bug in message exclusion	2021-07-29 21:58:59 +03:00
Raúl Kripalani	b3c951c924	add RepoType#String; adjust repo parsing logic.	2021-07-29 12:22:27 -04:00
Raúl Kripalani	8442bac5b2	support MARKETS_API_INFO env var; support markets-repo, markets-api-url flags.	2021-07-29 12:20:05 -04:00
Raúl Kripalani	3144da86f3	add RepoType#String; adjust repo parsing logic.	2021-07-29 13:49:51 +01:00
Raúl Kripalani	4e19d8d562	support MARKETS_API_INFO env var; support markets-repo, markets-api-url flags.	2021-07-29 10:55:37 +01:00
Raúl Kripalani	d8c90b91be	address nits.	2021-07-29 04:41:27 -04:00
Raúl Kripalani	c119ab6ed9	fix docs and nits.	2021-07-29 04:39:28 -04:00
Anton Evangelatov	0dd83c6755	fixup	2021-07-29 04:38:25 -04:00
Anton Evangelatov	9b7a7713a7	add RuntimeSubsystems API method; use it in `lotus-miner info`	2021-07-29 04:38:19 -04:00
Raúl Kripalani	d3742048f9	address nits.	2021-07-28 23:56:18 +01:00
Raúl Kripalani	a61ba7b09c	Merge branch 'master' into nonsense/get-enabled-subsystems-api	2021-07-28 23:55:59 +01:00
Aayush Rajasekaran	21244704fc	Merge branch 'master' into mg/feat/configurable-sector-expirations	2021-07-28 16:45:27 -04:00
raulk	80aedcd276	Merge branch 'master' into nonsense/get-enabled-subsystems-api	2021-07-28 20:09:43 +01:00
Raúl Kripalani	07f40b9488	fix docs and nits.	2021-07-28 20:03:25 +01:00
Łukasz Magiera	389f71251c	Remove townhall	2021-07-28 17:58:46 +02:00
Anton Evangelatov	4cf5c8f656	fixup	2021-07-28 16:23:04 +03:00
Anton Evangelatov	de4a847078	add RuntimeSubsystems API method; use it in `lotus-miner info`	2021-07-28 16:02:05 +03:00
Jiaying Wang	d45b4a7a1f	Merge branch 'master' into chore/cleanup-chain	2021-07-27 17:11:36 -04:00
Łukasz Magiera	9bd312881d	VMSys doesn't belong in chainstore	2021-07-27 15:30:23 +02:00
vyzo	7c195245a1	make cfgdoc-gen	2021-07-27 12:13:26 +03:00
vyzo	9d25464703	use functional options for hotstore gc, rename MovingGC to FullGC	2021-07-27 09:53:22 +03:00
vyzo	aa0bd51b2c	thread GCFrequency option into the splitstore config	2021-07-26 15:05:57 +03:00
vyzo	20f93a520f	make cfgdoc-gen	2021-07-26 15:05:57 +03:00
vyzo	4cdb34e448	add docstrings for splitstore config	2021-07-26 15:05:57 +03:00
vyzo	5acae50e07	add config option for splitstore moving gc frequency	2021-07-26 15:05:57 +03:00
vyzo	a0d6fdba33	add ChainBlockstoreInfo APIv1 endpoint	2021-07-26 08:30:07 +03:00
vyzo	2dc72d5849	satisfy linter who wants to be a spell checker in comments	2021-07-25 11:47:21 +03:00
vyzo	3d2ae433ee	add ChainCheckBlockstore API	2021-07-25 11:14:48 +03:00
Mike Greenberg	e68c59f984	fix: Update node config docs	2021-07-23 21:05:50 -04:00
Mike Greenberg	64d2b7a122	Merge branch 'origin/master' into mg/feat/configurable-sector-expirations * origin/master: (47 commits) Fix links in issue templates Update issue templates and add templates for M1 config: Add newlines consistently config: Add note on codegen to types.go fix lint cli docsgen config: Bring some docs from filecoin-docs add comment about the necessity of FileIO improve README config doc gen: generate formatted code config: Fix doc finding logic Nicer lotus-miner config commands Move doc-comment logic to the config pkg config: doc struct codegen lotus daemon config utils adjust compaction test with badger to hit the db add splitstore compaction test with badger markset update README with configuration options tweak badger options some tweaks in badger markset ... Conflicts: node/config/def.go	2021-07-23 20:36:54 -04:00
Łukasz Magiera	91c6aca7e7	config: Add newlines consistently	2021-07-23 16:59:55 +02:00
Łukasz Magiera	80af6de316	config: Add note on codegen to types.go	2021-07-23 16:02:45 +02:00
Łukasz Magiera	ac2a1bfeab	fix lint	2021-07-23 15:53:31 +02:00
Łukasz Magiera	a26712ebc4	config: Bring some docs from filecoin-docs	2021-07-23 15:40:40 +02:00
Łukasz Magiera	ed7f70e68e	config doc gen: generate formatted code	2021-07-23 15:18:20 +02:00
Łukasz Magiera	3f4973cb41	config: Fix doc finding logic	2021-07-23 15:17:22 +02:00
Łukasz Magiera	8b1f19e94c	Move doc-comment logic to the config pkg	2021-07-23 14:55:19 +02:00
Łukasz Magiera	97f5bb66c7	config: doc struct codegen	2021-07-23 13:55:50 +02:00
Mike Greenberg	adb62a3fff	chore: Move cfg getter into PCPolicy; Clamp values on get	2021-07-22 15:07:03 -04:00
Mike Greenberg	9e34cce1f2	feat(config): support configurable CC sector expiration	2021-07-22 15:07:03 -04:00
Łukasz Magiera	660829703a	Merge remote-tracking branch 'origin/master' into feat/split-net-api	2021-07-22 15:38:06 +02:00
Łukasz Magiera	da5aeda197	Merge branch 'master' into feat/splitstore-hot-messages	2021-07-22 12:58:06 +02:00
Peter Rabbitson	4830a3406b	Introduce the LOTUS_CHAIN_BADGERSTORE_DISABLE_FSYNC envvar Allows the user to control the opts.SyncWrites option of BadgerDs	2021-07-21 22:38:14 +02:00
Anton Evangelatov	fe31956713	fetch peer id from chain	2021-07-21 11:43:19 +02:00
vyzo	839b00ab40	rename messagepool ProtectMessages to ForEachPendingMessage	2021-07-20 09:23:36 +03:00
vyzo	ebbaf23af8	support out-of-chain reference protection	2021-07-20 09:02:40 +03:00
vyzo	1b77361301	add option for hotstore message retention	2021-07-17 08:35:35 +03:00
vyzo	e003203bea	implement exposed splitstore	2021-07-15 13:12:10 +03:00
Łukasz Magiera	812dc266cf	builder: Don't require specific NetAPI impl in StorageMinerAPI	2021-07-15 12:01:13 +02:00
Łukasz Magiera	49e26cce7d	api: Separate the Net interface from Common	2021-07-15 11:41:30 +02:00
hunjixin	bdaa73a413	add ChainGetMessagesInTipset api	2021-07-15 14:52:09 +08:00
Łukasz Magiera	583a8a13d9	Merge pull request #6629 from filecoin-project/feat/pledge-from-miner-balance Config for collateral from miner available balance	2021-07-13 17:22:00 +02:00
Łukasz Magiera	837322ea59	Merge pull request #6356 from filecoin-project/nonsense/split-market-miner-processes Support standalone miner-market process	2021-07-13 17:16:07 +02:00
Łukasz Magiera	7526a074d9	sealing: collateral buffer / falback config	2021-07-13 17:04:58 +02:00
Łukasz Magiera	c37401a1a3	Merge pull request #6474 from filecoin-project/feat/splitstore-redux Splitstore Enhanchements	2021-07-13 12:43:57 +02:00
hannahhoward	92ec239202	fix(node): ClientRetrieve stops on cancel When a deal is cancelled, any in progress calls to ClientRetrieve or ClientRetrieveWithEvents should return	2021-07-12 14:46:18 -07:00
Łukasz Magiera	83f2368507	Add CollateralFromMinerBalance config	2021-07-12 14:11:58 +02:00
Anton Evangelatov	4bc9fa04ba	rename cfg.Subsystems.EnableStorageMarket to EnableMarkets	2021-07-12 12:12:29 +02:00
Anton Evangelatov	056136ef5a	rename argument from url to apiInfo	2021-07-12 11:36:22 +02:00
Anton Evangelatov	a423c46f2a	rename mock.go to net.go	2021-07-12 11:35:03 +02:00
Anton Evangelatov	d89ddb9315	resolve conflicts	2021-07-12 11:34:37 +02:00
Anton Evangelatov	16784aa2cc	remove pieceProvider from DI; small refactors	2021-07-12 11:30:26 +02:00
vyzo	c0a1cfffa1	rename noopstore to discardstore	2021-07-09 19:19:37 +03:00
vyzo	abdf4a161a	explicitly switch marksets for concurrent marking this has very noticeable impact in initial marking time; it also allows us to get rid of the confusing ts monikers.	2021-07-09 04:26:36 +03:00
Łukasz Magiera	ed0cc0b56d	Handle bad peerIDs in MarketListRetrievalDeals	2021-07-08 19:27:03 +02:00
Łukasz Magiera	8f5c0c60f5	Config for deal publishing control addresses	2021-07-07 18:00:54 +02:00
Anton Evangelatov	82efe01c70	fixup	2021-07-07 18:00:28 +02:00
Anton Evangelatov	6b014f57e5	pass Subsystems to StorageMiner option; add enableLibp2p bool in Settings	2021-07-07 13:56:37 +02:00
Anton Evangelatov	42ae876711	remove ApplyIfEnableLibP2P	2021-07-07 13:31:56 +02:00
Anton Evangelatov	7a78527fde	remove not used sa; cleanup	2021-07-06 17:46:21 +02:00
Anton Evangelatov	10dc90f7fe	addressing comments	2021-07-06 17:33:47 +02:00
Anton Evangelatov	898687ea2e	Update node/modules/storageminer.go Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>	2021-07-06 17:28:52 +02:00
Anton Evangelatov	ff2772a58c	resolved conflicts	2021-07-06 16:00:41 +02:00
Łukasz Magiera	97207df705	Merge pull request #6639 from filecoin-project/frrist/context-in-statetree-diff polish(statetree): accept a context in statetree diff for timeouts	2021-07-05 17:30:31 +02:00
Anton Evangelatov	4be0a7a215	resolve conflicts with master	2021-07-05 13:13:32 +02:00
vyzo	190cb18ab0	housekeeping - remove defunct tracking store implementations - update splitstore node config - use mark set type config option (defaulting to mapts); a memory constrained node may want to use an on-disk one	2021-07-04 18:38:28 +03:00
vyzo	5f7ae1f489	update splistore DI constructor	2021-07-04 18:38:28 +03:00
vyzo	68bc5d2291	skip moving cold blocks when running with a noop coldstore it is a noop but it still takes (a lot of) time because it has to read all the cold blocks.	2021-07-04 18:38:28 +03:00
vyzo	649b7dd162	add config option for hot headers	2021-07-04 18:38:28 +03:00
vyzo	997f2c098b	keep headers hot when running with a noop splitstore	2021-07-04 18:38:28 +03:00
vyzo	04f2e102a1	kill full splitstore compaction, simplify splitstore configuration	2021-07-04 18:38:28 +03:00
vyzo	5cca29d1db	hook noop blockstore for splitstore in DI	2021-07-04 18:38:28 +03:00
Łukasz Magiera	8a94ab676e	storage: Fix FinalizeSector with sectors in stoage paths	2021-07-02 19:54:45 +02:00
Anton Evangelatov	9e85492b85	fix test	2021-07-02 12:24:07 +02:00
Anton Evangelatov	96245c3cfa	lint	2021-07-02 12:02:36 +02:00
Anton Evangelatov	4194c7fa24	revert back optional for Miner; remove ActorAddress call in node.MinerHandler	2021-07-02 11:58:31 +02:00
Raúl Kripalani	1869de7662	unwire host from storage.Miner.	2021-07-01 13:53:20 +01:00
Łukasz Magiera	c094aa82ec	commit batch: AggregateAboveBaseFee config	2021-07-01 13:33:54 +02:00
frrist	d02a2a2cfa	polish(statetree): accept a context in statetree diff for timeouts - this operation can take a while, this changes gives users the option to abort if it takes "too long".	2021-06-30 16:40:43 -07:00
Anton Evangelatov	3aa6d03cc1	connect miner to full node , only when running markets	2021-06-30 16:45:11 +02:00
Anton Evangelatov	4f0a96c9c7	resolved conflicts	2021-06-30 13:16:52 +02:00
Anton Evangelatov	9ec7cd7239	enable libp2p options	2021-06-30 12:55:43 +02:00
Łukasz Magiera	e9dd3e8650	Test Miner SimultaneousTransfers	2021-06-28 18:17:29 +02:00
Łukasz Magiera	37c5dd5afc	Miner SimultaneousTransfers config	2021-06-28 14:24:14 +02:00
Łukasz Magiera	3f3336340f	Fix wallet error messages	2021-06-25 10:48:47 +02:00
Łukasz Magiera	c3480dc0e8	Merge remote-tracking branch 'origin/releases' into chore/merge-release	2021-06-24 09:39:11 +02:00
Łukasz Magiera	9521c0b773	Add miner-side MaxDealStartDelay config	2021-06-23 19:45:13 +02:00
Łukasz Magiera	0e4cd3c599	Merge pull request #6563 from filecoin-project/asr/batch-starter Always flush when timer goes off	2021-06-23 19:19:30 +02:00
Aayush Rajasekaran	616e5688fc	Remove MinPreCommitBatch	2021-06-23 12:30:32 -04:00
Łukasz Magiera	fa2be42eb0	Merge remote-tracking branch 'origin/release/v1.10.0' into chore/merge-1.10	2021-06-23 15:51:00 +02:00
Anton Evangelatov	4217ec5308	make auth on remote rpc server optional	2021-06-23 13:27:03 +02:00
Anton Evangelatov	21b51328f9	adding TestDealWithMarketAndMinerNode	2021-06-23 12:44:52 +02:00
Jakub Sztandera	bf82b96fd8	Update default fees for aggregates Resolves #6422 Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-06-22 20:20:01 +02:00
Anton Evangelatov	6720463799	resolve merge conflicts	2021-06-22 11:28:23 +02:00
Raúl Kripalani	b6147fb27f	extern/storage: retype resource filtering strategy to enum.	2021-06-21 20:28:15 +01:00
Anton Evangelatov	577b730415	wip	2021-06-18 18:42:34 +02:00
Łukasz Magiera	097baeb9b0	Make batch deal input test less flaky	2021-06-17 20:52:09 -04:00
Łukasz Magiera	5c8498b603	storagefsm: Fix batch deal packing behavior	2021-06-17 20:50:14 -04:00
Łukasz Magiera	44de67cf7d	Merge pull request #6175 from filecoin-project/feat/dynamic-retreival-pricing Dynamic Retrieval pricing	2021-06-17 10:25:48 +02:00
Anton Evangelatov	d45bb14015	Merge branch 'master' into nonsense/split-market-miner-processes	2021-06-16 18:49:15 +02:00
Aayush Rajasekaran	520a0091f5	Merge branch 'releases' into release/v1.10.0	2021-06-15 17:40:36 -04:00
Łukasz Magiera	0514f38dc4	Merge remote-tracking branch 'origin/master' into chore/merge-1.10	2021-06-14 13:27:53 +02:00
Anton Evangelatov	31e6fb154e	resolve conflicts	2021-06-14 11:27:54 +02:00
aarshkshah1992	4419205b0e	fix compilation	2021-06-14 10:10:29 +05:30
Aarsh Shah	bed2bc73a3	Update node/config/def.go Co-authored-by: raulk <raul@protocol.ai>	2021-06-14 09:42:22 +05:30
Łukasz Magiera	94be3a973a	Don't enable early finalization by default	2021-06-11 11:45:20 +02:00
Łukasz Magiera	ec06f086ef	sealing: Early finalization option	2021-06-11 11:41:28 +02:00
aarshkshah1992	fed5afa704	merge master	2021-06-11 09:35:20 +05:30
Łukasz Magiera	6f80d9f3c2	Merge pull request #6311 from filecoin-project/raulk/itests consolidate integration tests into `itests` package; create test kit; cleanup	2021-06-10 12:33:29 +02:00
Jennifer Wang	8f426b49ec	updated configuration comments for docs	2021-06-09 23:36:44 -04:00
Jennifer	3a1ad50da5	Merge pull request #6440 from filecoin-project/jen/miner-config-doc updated configuration comments for docs	2021-06-09 23:35:04 -04:00
Jennifer Wang	95ad74a156	updated configuration comments for docs	2021-06-09 20:44:20 -04:00
Raúl Kripalani	6d46be53bd	make tests no longer create auth tokens.	2021-06-10 00:15:29 +01:00
Łukasz Magiera	f0a2e97cb5	fee config for sector batching	2021-06-09 14:23:06 -04:00
Raúl Kripalani	81b3c6e2ab	Merge branch 'master' into raulk/itests	2021-06-09 17:34:46 +01:00
Anton Evangelatov	beb5bffaf6	fix log.Error; rename RemoteStore	2021-06-09 13:05:54 +02:00
Łukasz Magiera	78171055e7	fee config for sector batching	2021-06-08 15:45:32 +02:00
Anton Evangelatov	9f3ec82cc1	initial resolution of conflicts	2021-06-08 13:01:22 +02:00
aarshkshah1992	4e9bb16532	changes as per review	2021-06-08 10:09:27 +05:30
aarshkshah1992	16166504f4	merged master	2021-06-08 09:17:40 +05:30
Raúl Kripalani	a274fcc533	Merge branch 'master' into raulk/itests	2021-06-07 23:44:22 +01:00
Raúl Kripalani	db1a61852d	Merge branch 'master' into raulk/itests	2021-06-07 23:17:44 +01:00
Jakub Sztandera	f9acd07987	Update libp2p to 0.14.1 This does not resolve the CI issue but prepares us for it after https://github.com/libp2p/go-libp2p/pull/1116 Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-06-07 18:51:48 +02:00
Łukasz Magiera	fadc79a487	Merge pull request #6280 from filecoin-project/feat/pieceread-outside-scheduler Bypass task scheduler for reading unsealed pieces	2021-06-07 18:11:44 +02:00
Łukasz Magiera	8625da3479	Bump miner/worker api versions	2021-06-07 16:57:24 +02:00
aarshkshah1992	21e6b50294	finished rebasing PR	2021-06-07 16:02:15 +05:30
Anton Evangelatov	de646a5b93	fix node builder	2021-06-07 12:16:10 +02:00
aarshkshah1992	4efaa8d688	unit tests for the remote store Reader	2021-06-07 15:02:52 +05:30
aarshkshah1992	65eb610ec3	docs, logs and green ci	2021-06-07 15:02:49 +05:30
aarshkshah1992	670835fca0	bypass task scheduler for reading unsealed pieces	2021-06-07 15:02:04 +05:30
Lion	e4588aed5c	Fix the doc errors of the sealing config funcs	2021-06-07 12:35:43 +08:00
Dirk McCormick	52b90e4c9d	test: offline deals	2021-06-04 22:42:07 +02:00
Łukasz Magiera	a7746961fb	Merge remote-tracking branch 'origin/master' into feat/list-retrievals	2021-06-04 20:49:01 +02:00
Anton Evangelatov	ded5341da0	fixup test node builder	2021-06-04 17:22:36 +02:00
Anton Evangelatov	ed634bc3a4	rebuild docs	2021-06-04 17:06:55 +02:00
Anton Evangelatov	c4d10bc44e	apply messagepool.NewProviderLite to lite nodes	2021-06-04 16:51:07 +02:00
Anton Evangelatov	6f125cc129	add prover to fullnode and miner builders	2021-06-04 16:26:32 +02:00
Anton Evangelatov	d9a7348ae1	use masters filecoin-ffi	2021-06-04 16:20:40 +02:00
Anton Evangelatov	8bd09e39ca	resolve merge conflicts	2021-06-04 16:17:00 +02:00
Anton Evangelatov	c877166b4a	add messagepool to test node constructor	2021-06-04 15:53:33 +02:00
Anton Evangelatov	88756f3ebf	fix TestAPIDeal tests	2021-06-04 15:41:38 +02:00
aarshkshah1992	303ef15a83	fix ci	2021-06-04 17:33:11 +05:30
Łukasz Magiera	6d9779ecae	Merge remote-tracking branch 'origin/master' into feat/nv13-1.11	2021-06-02 15:07:13 +02:00
Łukasz Magiera	a676da8a7e	Merge remote-tracking branch 'origin/master' into feat/nv13-1.11	2021-06-01 21:07:04 +02:00
Łukasz Magiera	dc642d0b7b	Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11	2021-06-01 21:06:58 +02:00
Dirk McCormick	8997ed508b	feat: update to markets-v1.4.0	2021-06-01 10:16:12 -06:00
Łukasz Magiera	d69032b2f0	Merge pull request #6367 from filecoin-project/fix/robustify-commit-batcher Robustify commit batcher	2021-06-01 17:49:46 +02:00
Łukasz Magiera	5a7f97210f	Merge pull request #6317 from filecoin-project/jen/test-datacap-topup Add test for AddVerifiedClient	2021-06-01 14:41:05 +02:00
Łukasz Magiera	5a9e6c8142	Merge pull request #6364 from filecoin-project/asr/resolve-messages Improve address resolution for messages	2021-06-01 14:38:31 +02:00
Łukasz Magiera	482e1110c2	precommit batcher: Improve error propagation	2021-06-01 14:35:30 +02:00
Łukasz Magiera	cb4eb487f4	commit batcher: Fix min aggregate size check	2021-06-01 12:27:22 +02:00
Łukasz Magiera	9fcb564bef	Make commit batcher more robust	2021-06-01 11:56:19 +02:00
Anton Evangelatov	e4136b0d42	add modules.StorageAuthWithURL to set correct token	2021-06-01 11:42:28 +02:00
Jennifer Wang	fcfc214ed1	Use MockSbBuilder	2021-05-31 20:14:11 -04:00
Aayush Rajasekaran	9ceee6028b	More tweaking of context handling in the messagepool	2021-05-31 18:13:23 -04:00
Aayush Rajasekaran	ed93d0725f	Protect mp.localAddrs and mp.pending behind helper functions	2021-05-31 18:13:23 -04:00
Aayush Rajasekaran	1f03a618f9	Plumb contexts through	2021-05-31 18:13:23 -04:00
Łukasz Magiera	ffa47659a1	Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11	2021-05-31 21:38:34 +02:00
Łukasz Magiera	c3e8eddb9b	Merge remote-tracking branch 'origin/master' into feat/nv13-1.11	2021-05-31 21:24:56 +02:00
Łukasz Magiera	2a1b35e694	Merge pull request #6361 from filecoin-project/asr/fip-0015 Implement FIP-0015	2021-05-31 20:58:21 +02:00
Łukasz Magiera	2d6a159dcc	Merge pull request #6041 from filecoin-project/fix/batch-deal-packing storagefsm: Fix batch deal packing behavior	2021-05-31 20:45:12 +02:00
Aayush Rajasekaran	7fca1c1ee7	Implement FIP-0015	2021-05-31 14:40:54 -04:00
Peter Rabbitson	1e72d8f14c	Reduce noise from 'peer has different genesis' messages After the unification of all networks behind a build-tag, the amount of these warnings has risen sharply. Reduce it to a debug-level, since it isn't actionable to an operator.	2021-05-31 13:43:24 +02:00
Anton Evangelatov	d195a12565	Merge branch 'master' into nonsense/split-market-miner-processes	2021-05-31 11:26:11 +02:00
Łukasz Magiera	9475079b97	Make batch deal input test less flaky	2021-05-30 17:46:22 +02:00
Łukasz Magiera	c8d603557b	storagefsm: Fix batch deal packing behavior	2021-05-30 17:46:22 +02:00
Jennifer Wang	2b08e9f3a6	Add test AddVerifiedClient	2021-05-28 15:41:57 -04:00
Aayush Rajasekaran	fba5c65ffd	Extend the default deal start epoch delay	2021-05-28 12:35:48 -04:00
hannahhoward	19b6dc8d1e	feat(cli): add a list retrievals command Currently, there is no way to inspect retrievals on a client. This adds said command, allow with corresponding APIs	2021-05-27 11:48:25 -07:00
Łukasz Magiera	1e4456138e	Merge master into feat/nv13	2021-05-27 12:28:20 +02:00
Anton Evangelatov	c311520968	add missing messagepool override	2021-05-27 10:28:41 +02:00
Aayush Rajasekaran	d9e86afa24	Tweak client calcDealExpiration to consider genesis miners	2021-05-26 21:46:37 -04:00
Aayush Rajasekaran	21b4741e30	Fix randomness fetching around null blocks	2021-05-26 21:06:06 -04:00
Aayush Rajasekaran	7714537239	Allow starting networks from arbitrary actor versions	2021-05-26 12:48:14 -04:00
Anton Evangelatov	9979de24db	logs and defaults	2021-05-26 12:47:21 +02:00
Łukasz Magiera	43c62f4406	Revert "Allow starting networks from arbitrary actor versions"	2021-05-26 12:33:08 +02:00
Aayush Rajasekaran	cf574ca9a1	Allow starting networks from arbitrary actor versions	2021-05-25 19:30:20 -04:00
Łukasz Magiera	4c6c9a0edb	Fix windowpost/deadline tests	2021-05-25 21:27:32 +02:00
Łukasz Magiera	04658e1cae	fix lint	2021-05-25 16:26:59 +02:00
Łukasz Magiera	d90ab2b81f	config: Document batching fields	2021-05-25 16:20:49 +02:00
aarshkshah1992	78255eac54	test unsealing prices for default pricing strategy	2021-05-24 15:11:50 +05:30
Raúl Kripalani	c3f45cc836	Merge branch 'raulk/refactor-rpc' into raulk/itests	2021-05-23 14:56:54 +01:00
Raúl Kripalani	2337248aa5	rpc: separate handlers/server; refactor miner rpc.	2021-05-23 14:48:42 +01:00
Raúl Kripalani	627248d610	Merge branch 'raulk/refactor-rpc' into raulk/itests	2021-05-23 12:41:25 +01:00
Raúl Kripalani	de454b8e26	add godocs to ServeRPC.	2021-05-23 12:40:33 +01:00
aarshkshah1992	dc6dbc9a11	dpr changes and test based on new unsealing PR	2021-05-22 22:40:21 +05:30
Raúl Kripalani	5b77dbd06f	decouple rpc server and shutdown logic from cmd.	2021-05-21 21:34:17 +01:00
aarshkshah1992	bd9959070e	unit tests for the remote store Reader	2021-05-20 15:25:46 -06:00
aarshkshah1992	73613ee883	docs, logs and green ci	2021-05-20 15:25:46 -06:00
aarshkshah1992	2a40c802ea	bypass task scheduler for reading unsealed pieces	2021-05-20 15:25:46 -06:00
Dirk McCormick	885564fe24	Revert "chore: update go-libp2p" This reverts commit `f7fbaef361`.	2021-05-20 11:22:19 -06:00
Łukasz Magiera	13ff6ed462	Test pledging before nv13 upgrade	2021-05-20 18:38:33 +02:00
Anton Evangelatov	4693c61305	re-arrange NodeBuilder and add storageminer_svc	2021-05-20 13:10:14 +02:00
Anton Evangelatov	76bb424de0	update StorageMinerAPI	2021-05-20 13:01:14 +02:00
Raúl Kripalani	06b8ee5b95	Merge branch 'master' into raulk/itests	2021-05-20 11:56:43 +01:00
Anton Evangelatov	cb603c62d9	update retrievaladapter ; add piece_provider	2021-05-20 12:49:53 +02:00
Anton Evangelatov	2aad7b6979	update sectorstorage.New (Manager) interface	2021-05-20 12:32:29 +02:00
Dirk McCormick	706fdcbb80	feat: increase data transfer timeouts	2021-05-19 12:56:07 -06:00
Łukasz Magiera	3a99f95f8e	Merge pull request #6149 from filecoin-project/feat/markets-1.3 Update to markets 1.3	2021-05-19 20:51:33 +02:00
Raúl Kripalani	c46d4ae529	wip	2021-05-19 17:30:43 +01:00
Łukasz Magiera	2a1b359ede	config: fmt	2021-05-19 15:20:34 +02:00
Łukasz Magiera	a5677d1b7a	ffiwrapper: Separate Prover interface	2021-05-19 15:20:23 +02:00
Łukasz Magiera	2a0c0e379b	Working default batching config	2021-05-19 14:32:41 +02:00
Anton Evangelatov	a989f60e27	add SectorAddPieceToAny and SectorUnsealPiece to StorageMiner iface; model moved to api package - PieceDealInfo, DealSchedule	2021-05-19 13:05:07 +02:00
Łukasz Magiera	5112b9fe2b	Lower default batch slack	2021-05-18 21:04:47 +02:00
Łukasz Magiera	9690bc882c	Test to trigger batching logic	2021-05-18 18:41:42 +02:00
Łukasz Magiera	81b5d8c671	Make things build with both batchers	2021-05-18 16:53:49 +02:00
Łukasz Magiera	c7ba083fa4	Import precommit batcher	2021-05-18 16:51:06 +02:00
vyzo	7512748b56	wire in sealing config values	2021-05-18 15:31:52 +03:00
vyzo	74bad49068	correctly handle commit batch timer	2021-05-18 14:30:47 +03:00
vyzo	357c0868b7	proper config for termination batching and commit wait	2021-05-18 12:20:19 +03:00
vyzo	febf7cf28f	sane config defaults	2021-05-18 12:02:53 +03:00
Aayush Rajasekaran	9a6e601754	Merge pull request #5961 from filecoin-project/feat/stateless-offline-dealflow Introduce stateless offline dealflow, bypassing the FSM/deallists	2021-05-17 12:35:39 -04:00
Łukasz Magiera	506f39b294	WIP: Integrate FIP0013	2021-05-11 22:10:29 -04:00
Peter Rabbitson	d54ed1b0c6	Merge remote-tracking branch 'origin/master' into feat/stateless-offline-dealflow	2021-05-11 04:54:22 +02:00
Steven Allen	f7fbaef361	chore: update go-libp2p From 0.12.0 to 0.14.0. Headline for lotus: faster yamux.	2021-05-10 19:41:57 -07:00
Peter Rabbitson	6f71eab8ce	Adjust lp2p retry params	2021-05-11 04:26:04 +02:00
Jakub Sztandera	18cbdcfc81	Disable checks API on Lotus Lite Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 17:19:17 +02:00
Jakub Sztandera	d777680449	Fix mpool.GetActor for lite node Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:06 +02:00
Jakub Sztandera	8d75da1629	Use MessagePrototype for check API Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:05 +02:00
vyzo	e2d0047a2a	introduce message prototypes This introduces message prototypes to applicable API endpoints, which allows us to invert control of message sending and give the user a chance to intervene with an interactive ui. Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:05 +02:00
vyzo	d782250aba	implement MessagePool.CheckReplaceMessages Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:04 +02:00
vyzo	91e774063e	implement MessagePool.CheckMessages Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:04 +02:00
vyzo	ed61642b3a	implement NodeStatus API Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-05-07 15:30:04 +02:00
Peter Rabbitson	e648b72d8d	Merge remote-tracking branch 'origin/master' into feat/stateless-offline-dealflow	2021-05-06 15:57:10 +02:00
Dirk McCormick	5b1fab8ffa	feat: markets v1.3	2021-05-06 09:49:44 +02:00
Steven Allen	eb10918470	Merge pull request #6107 from filecoin-project/feat/checkpoint-sync feat: allow checkpointing to forks	2021-04-29 16:30:52 -07:00
Łukasz Magiera	9b87566dbf	Merge pull request #6141 from filecoin-project/split-TestBatchDealInput-from-TestAPIDealFlow flaky tests improvement: separate TestBatchDealInput from TestAPIDealFlow	2021-04-29 19:01:35 +02:00
Steven Allen	f983fa8cd5	Merge pull request #6134 from filecoin-project/asr/emptytsk Use EmptyTSK where appropriate	2021-04-29 09:35:43 -07:00
Anton Evangelatov	c7d74ef16e	separate TestBatchDealInput from TestAPIDealFlow	2021-04-29 17:39:00 +02:00
Łukasz Magiera	a456f394ac	Merge pull request #6007 from filecoin-project/asr/state-list-messages Speed up StateListMessages in some cases	2021-04-29 07:47:26 +02:00
Aayush Rajasekaran	d794b49df3	Use EmptyTSK where appropriate	2021-04-29 01:00:03 -04:00
Steven Allen	69da6a2a29	feat: allow checkpointing to forks Previously, `lotus sync checkpoint` would only checkpoint on the current chain. Now, it can switch to a new fork.	2021-04-28 13:54:26 -07:00
Łukasz Magiera	a24a73bb52	Merge tag 'v1.8.0' into release/v1.9.0	2021-04-27 10:30:33 +02:00
Łukasz Magiera	c4137a438d	Merge commit 'f1ded63d608e1b191d79c780b8065f8cff367c98' into feat/nv12-1.11	2021-04-27 08:22:55 +02:00
Aayush Rajasekaran	f1ded63d60	v4 specs-actors integration, nv12 migration	2021-04-27 02:02:44 -04:00
Reskorey	58bee65bcb	go fmt	2021-04-23 16:10:51 +08:00
Reskorey	7738886936	fix full node painc fix: full node painc when calling RPC Filecoin.ClientRetrieve without the order.Tatol pecified.	2021-04-23 15:43:41 +08:00
Łukasz Magiera	c72eb0ccaf	Merge pull request #6048 from filecoin-project/fix/nonce-getting Fix nonce getting on Lotus lite	2021-04-18 16:25:47 +02:00
Peter Rabbitson	dab2a41792	Raise client MaxFee default The original value is too conservative for the current state of mainnet Resolves (temporarily) #5543	2021-04-17 00:35:40 +02:00
Jakub Sztandera	66c408938b	Fix signature in messagepool, wire in context Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-04-16 18:43:28 +02:00
Jakub Sztandera	77d004ec06	Fix nonce getting on Lotus lite Resolves #5593 #5995 Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-04-15 19:57:09 +02:00
Łukasz Magiera	e76935147f	Merge remote-tracking branch 'origin/master' into feat/v1-api	2021-04-13 14:24:22 +02:00
Aayush Rajasekaran	ff48a70ed9	Speed up StateListMessages in some cases	2021-04-10 02:33:34 -04:00
Łukasz Magiera	88bdb591d5	Merge pull request #6003 from filecoin-project/fix/fbs-di Fix fallback chainstore	2021-04-10 01:58:38 +02:00
Łukasz Magiera	7e4cb9da84	Fix fallback chainstore	2021-04-09 16:11:02 +02:00
Dirk McCormick	4a3261a986	feat: go-fil-markets-v1.2.5-rc1	2021-04-08 14:34:35 +02:00
Peter Rabbitson	25a77d905d	Centralize everything on ipfs/go-log/v2 I am not entirely sure this is right, but everything seems to build...	2021-04-06 15:04:32 +02:00
Łukasz Magiera	e8f28d7b9f	Fix tests	2021-04-06 12:24:58 +02:00
Łukasz Magiera	81bd27911f	Propagate StateMsg api changes	2021-04-05 19:56:53 +02:00
Peter Rabbitson	b79be2a2c2	Merge remote-tracking branch 'origin/master' into feat/stateless-offline-dealflow	2021-04-05 16:06:44 +02:00
Łukasz Magiera	deb2b90b6a	Fix lotus/miner build	2021-04-05 13:23:46 +02:00
Peter Rabbitson	7fddbb528d	Introduce stateless offline dealflow, bypassing the FSM/deallists This is aproposal for an additional flag --manual-stateless-deal and a corresponding API endpoint ClientStatelessDeal. This allows firing off an offline-style deal against a miner without keeping further track of it locally. Not keeping any local state introduces the limitation of requiring free storage deals, as there is nothing to tie the payment channel setup to. Rationale/need for this type of flow is the case of incredibly large sets of data nd deals, where the client and providers have prearranged payment ahead of time, and the client has a separate-from-lotus database of deal inventory. This way the client can use their lotus node merely as a network gateway, without running into any limitations currently present in both lotus as a whole and go-fil-markets in particular. Specific context for this work is filecoin-discover, where the requirement is to onboard ~ 12,000,000 individual deals against a pool of miners with whom the client has prearranged a relationship.	2021-04-05 13:11:10 +02:00
Łukasz Magiera	f4e46c9003	implement v1 api with api modules	2021-04-03 13:20:50 +02:00
Łukasz Magiera	1b32d7f52f	cli: fix build with v1 api changes	2021-04-03 12:55:29 +02:00
Łukasz Magiera	0103d2f621	v1 api: Cleanup message wait/search apis	2021-04-02 13:52:24 +02:00
Łukasz Magiera	c9866da688	Merge remote-tracking branch 'origin/master' into feat/v1-api	2021-04-02 13:15:33 +02:00
Łukasz Magiera	4538550999	Merge remote-tracking branch 'origin/master' into feat/local-retrieval	2021-04-02 12:36:28 +02:00
Anton Evangelatov	df003d44d2	include Stages and DealStages only on ClientDealInfo, not on ClientListDeals	2021-04-01 17:08:01 +03:00
Łukasz Magiera	1a5c4d70f4	Merge remote-tracking branch 'origin/master' into feat/v1-api	2021-04-01 15:33:37 +02:00
Anton Evangelatov	967fbf52c6	resolve conflicts	2021-03-31 23:14:29 +03:00
Łukasz Magiera	dfe8a1c1e2	client: Set the client var in the correct scope	2021-03-31 19:38:02 +02:00
Łukasz Magiera	ee04f9b71c	Merge remote-tracking branch 'origin/master' into nonsense/add-cancel-retrieval-deal-cmd	2021-03-31 19:15:21 +02:00
Łukasz Magiera	10ac9039b4	Merge pull request #5853 from filecoin-project/feat/markets-v1.2.0 update to markets v1.2.x	2021-03-31 18:59:20 +02:00
Łukasz Magiera	f2ab316fb1	Local retrieval support	2021-03-30 21:32:06 +02:00
Anton Evangelatov	139ff4ae64	resolved conflicts	2021-03-30 16:15:42 +03:00
Anton Evangelatov	77bc8d6ca3	remove storedCounters	2021-03-29 18:39:07 +03:00
Anton Evangelatov	8ba0bcb375	Merge branch 'master' into nonsense/cli-show-deals-backup	2021-03-29 18:19:28 +03:00
hunjixin	896303c6a2	fix lint	2021-03-29 10:31:31 +08:00
Raúl Kripalani	fd91c095c4	rename command to cancel-retrieval; rename args to follow Lotus style.	2021-03-26 16:37:46 +00:00
Dirk McCormick	b13f6a3209	fix: cancel retrieval deal - disallow negative deal ID	2021-03-26 09:51:07 +01:00
hunjixin	449b33abee	remote calc winningpost proof	2021-03-26 13:32:03 +08:00
Łukasz Magiera	e003977559	apiclient: Version client funcs	2021-03-25 15:39:48 +01:00
Łukasz Magiera	7a23b411ae	Merge apistruct with the api package	2021-03-25 15:10:08 +01:00
Anton Evangelatov	a202f9d19e	add cancel-retrieval-deal cmd	2021-03-24 18:18:49 +02:00
Dirk McCormick	1359fb6cc4	fix: disable pull channel monitoring for now	2021-03-24 13:56:59 +01:00
Łukasz Magiera	c41777dcd2	API proxy struct codegen (#5854 ) * mostly working api proxy gen * api: Consistent api names * fix docsgen * regenerate api struct * api: expand external interfaces * Add missing gen files * apigen: fix perm detection * api: Move perm tags to the interface * gofmt * worker perms * docsgen * docsgen: ignore tag comments * apigen: add codegen warning * gofmt * missing actor type * docsgen * make linter happy * fix lint * apigen: use directives for tags * docsgen * regen openrpc docs	2021-03-23 13:42:56 +01:00
Dirk McCormick	aee9890ee7	feat: update go-data-transfer and go-fil-markets	2021-03-23 13:28:30 +01:00
Anton Evangelatov	22217b7cae	Merge branch 'master' into nonsense/cli-show-deals	2021-03-23 13:58:26 +02:00
frrist	6caafaeee3	polish(node): define settings opts for 3rd party dep injection - motivating use case running a lotus-sentinel node	2021-03-22 15:37:44 -07:00
Jakub Sztandera	5f672c2ed0	perf: add cache for gas permium estimation Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-03-22 18:06:14 +01:00
Dirk McCormick	9fb1dd43a9	feat: update to markets v1.2.0	2021-03-22 15:31:13 +01:00
Łukasz Magiera	43d9cc36a4	OpenRPC Support (#5843 ) * main: init implement rpc.Discover RPC method This implement the basic functionality for the method over HTTP RPC. Signed-off-by: meows <b5c6@protonmail.com> * main,go.mod,go.sum: init example with go-openrpc-reflect lib Signed-off-by: meows <b5c6@protonmail.com> Conflicts: go.mod go.sum * main: make variable name human-friendly Signed-off-by: meows <b5c6@protonmail.com> * main,go.mod,go.sum: init impl of go-openrp-reflect printing document Signed-off-by: meows <b5c6@protonmail.com> Conflicts: go.mod go.sum * go.mod,go.sum: use go-openrpc-reflect and open-rpc/meta-schema hackforks This is for development only. Versions need to be bumped when they're ready for use as canonical remotes. Signed-off-by: meows <b5c6@protonmail.com> * main,openrpc,main: refactor openrpc supporting code to own package This eliminates code duplication. Signed-off-by: meows <b5c6@protonmail.com> * main: add rpc.Discover to openrpc document Signed-off-by: meows <b5c6@protonmail.com> * openrpc: fix rpc.discover method name casing Also fixes casing stuff for the rest of Filecoin. methods. Signed-off-by: meows <b5c6@protonmail.com> * Revert "main: add rpc.Discover to openrpc document" This reverts commit 116898efb10f33e405ac74acb1aa6daefcd46a62. * main: fix document creation method name This fixes an issue caused with the latest reverting commit. Signed-off-by: meows <b5c6@protonmail.com> * main,docgen,openrpc: refactor to share api parsing, etc as docgen exported stuff Signed-off-by: meows <b5c6@protonmail.com> Makefile: fix docgen refactoring for makefile use of command Signed-off-by: meows <b5c6@protonmail.com> * openrpc: add schema.examples to app reflector There are quite of few of these already registered for the docgen command, so it makes sense to use those! Signed-off-by: meows <b5c6@protonmail.com> * openrpc: init method pairing examples Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump go.mod to use latest meta-schema and openrpc-reflect versions Signed-off-by: meows <b5c6@protonmail.com> * openrpc: init SchemaType mapper function This function will handle the manual configurations for app-specific data types w/r/t their json schema representation. This is useful for cases where the reflect library is unable to provide a sufficient representation automatically. Provided in this commit is an initial implementation for the integerD type (assuming number are represented in the API as hexs), and a commonly used cid.Cid type. Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: tame dependencies by bumping etclabscore/go-openrpc-reflect This removes a problematic dependency on github.com/ethereum/go-ethereum, which was imported as a dependency for a couple github.com/etclabscore/go-openrpc-reflect tests. etclabscore/go-openrpc-reflect v0.0.36 has removed this dependency, so this commit is the result of bumping that version and then running 'go mod tidy' This is in response to a review at https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205 Date: 2020-11-21 06:52:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: add 'miner' arg to openrpc gen cmd This allows the command to EITHER generate the doc for Full or Miner APIs. See comment for usage. Date: 2020-11-21 07:48:05-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgen: add missing examples for Miner API Generating the Miner API OpenRPC doc (via 'go run ./api/openrpc/cmd miner') caused the example logic to panic because some types were missing. This commit adds those missing types, although I'm not an expert in the API so I can't suggest that the example values provided are ideal or well representative. Date: 2020-11-21 07:50:21-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build/openrpc/full.json,build/openrpc/miner.json: add build/openrpc/[full/miner].json docs These will be used as static documents provided by the rpc.discover method. Date: 2020-11-21 07:51:39-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: init go-rice openrpc static assets Date: 2020-11-21 08:23:06-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: remove rpc.discover implementation from runtime plugin Instead of generating the doc on the fly, we're going to serve a static asset. Rel https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205 This removes the runtime implementation from the RPC server construction. Date: 2020-11-21 08:41:20-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,common: add Discover(ctx) method to CommonAPI interface and structs Date: 2020-11-21 08:41:56-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: use rpc server method aliasing for rpc.discover This depends on a currently-forked change at filecoin-project/go-jsonrpc 8350f9463ee451b187d35c492e32f1b999e80210 which establishes this new method RPCServer.AliasMethod. This solves the problem that the OpenRPC spec says that the document should be served at the system extension-prefixed endpoing rpc.discover (not Filecoin.Discover). In fact, the document will be available at BOTH endpoints, but that duplicity is harmless. Date: 2020-11-21 09:18:26-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,build,common: rpc.discover: return json object instead of string Instead of casting the JSON asset from bytes to string, unmarshal it to a map[string]interface{} so the server will provide it as a JSON object. Date: 2020-11-21 09:27:11-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile: merge resolve: docsgen command path Date: 2020-11-22 07:19:36-06:00 Signed-off-by: meows <b5c6@protonmail.com> * apistruct,main,docgen,openrpc: merge resolve: fix func exporteds, signatures Date: 2020-11-22 07:31:03-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: 'get get' auto-bumps version Date: 2020-11-22 07:31:44-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile,docgen,main,build/openrpc: refactor openrpc documentation generation This creates Makefile command docsgen-openrpc-json, and refactors the docsgen command to generate both the markdown and openrpc json documents, redirecting the output of the openrpc json documentation to the build/openrpc/ directory, where those json files will be compiled as static assets via go-rice boxes. The api/openrpc/cmd now uses usage argumentation congruent to that of the docgen command (switching on API context). Date: 2020-11-22 08:01:18-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main,docgen_openrpc: rename api/openrpc -> api/docgen-openrpc Renames the package as well. This is intended to parallel the existing docgen package and command namespacing. Date: 2020-11-22 10:34:46-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,docgen,build,build/openrpc: use typed Discover response Instead of using a map[string]interface{}, use a typed response for the Discover method implementation. This avoids having to set a docgen Example for the generic map[string]interface{} (as an openrpc document) which both pollutes the generic type and lacks useful information for the Discover method example. Date: 2020-11-22 08:31:16-06:00 Signed-off-by: meows <b5c6@protonmail.com> * apistruct,build,main,impl: implement Discover method for Worker and StorageMiner APIs Methods return static compiled assets respective to the APIs. Date: 2020-11-22 08:57:18-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgen_openrpc,build/openrpc: remove timestamping from openrpc doc info This should allow openrpc docs generated at different times to be equal. This is important because the CI (Circle) runs the docgen command and tests that the output and the source are unchanged (via git diff). Date: 2020-11-22 10:47:07-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main,docgen_openrpc,main,build: fix lint issues Fixes goimports, staticcheck, golint issues. Date: 2020-11-22 11:06:46-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc: fix: don't use an underscore in package name (golint) Date: 2020-11-22 11:07:53-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.sum: fix: mod-tidy-check (run 'go mod tidy') Date: 2020-11-22 11:09:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump filecoin-project/go-jsonrpc dep to latest This version includes the necessary RPCServer.AliasMethod method. Date: 2020-11-23 12:16:15-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile,main,build,build/openrpc: init gzipped openrpc static docs Date: 2020-11-24 06:15:06-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: refactor gzip reading Date: 2020-11-24 06:18:34-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: add basic test for openrpc doc from static assets Date: 2020-11-24 06:30:23-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: handle reader Close error This keeps the errcheck linter happy. Date: 2020-11-24 06:33:14-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.sum: run 'go mod tidy' Date: 2020-11-24 06:36:07-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: go mod tidy Tidying up after resolving the merge conflicts with master at go.mod Date: 2020-11-24 06:40:45-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump filecoin-project/go-jsonrpc to latest This is a repeat of 76e6fd2, since the latest merge to master seems to have reverted this. Date: 2020-11-24 06:42:30-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc,build/openrpc: remove method example pairings, improve schema examples Removing method example pairings since they were redundant to schema examples and were not implemented well. Improved schema examples by using the ExampleValue method instead of the map lookup. Made a note in the comment here that this is not ideal, since we have to make a shortcut assumption /workaround by using 'unknown' as the method name and the typea as its own parent. Luckily these values aren't heavily used by the method logic. Date: 2020-11-27 12:57:36-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc: use generic number jsonschema for number types Previously used an integer schema assuming hex encoding. It appears, based on review some of the examples, that this may not be the case. Obvioussly this schema could be more descriptive, but just shooting for mostly likely to be not wrong at this point. Date: 2020-12-15 14:44:37-06:00 Signed-off-by: meows <b5c6@protonmail.com> * cmd/lotus,go.mod,go.sum: maybe fix straggling merge resolution conflicts Date: 2021-01-19 12:30:42-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build/openrpc/full.json.gz,build/openrpc/miner.json.gz,build/openrpc/worker.json.gz: run 'make docsgen' Date: 2021-01-19 12:33:55-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/apistruct,node/impl: (lint) gofmt Date: 2021-01-19 12:39:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen: maybe fix parse error: open ./api: no such file or directory Date: 2021-01-19 12:52:04-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen,build/openrpc: maybe fix no such file error and run 'make docsgen' Date: 2021-01-19 12:55:52-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen: return if AST comment/groupdoc parsing encounters any error This will returns empty comments/docs maps. This should fix issues like: https://app.circleci.com/pipelines/github/filecoin-project/lotus/12445/workflows/4ebadce9-a298-4ad1-939b-f19ef4c0a5bf/jobs/107218 where the environment makes file lookups hard or impossible. Date: 2021-01-19 13:04:58-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api: Don't depend on build/ * make: support parallel docsgen * openrpc gen: Use simple build version * methodgen * goimports Co-authored-by: meows <b5c6@protonmail.com>	2021-03-19 19:22:46 +01:00
Anton Evangelatov	ac576c75d4	amend restart config	2021-03-19 14:08:50 +02:00
Anton Evangelatov	86772a09c7	use new channelmonitor.Config	2021-03-19 13:53:03 +02:00
Anton Evangelatov	9d6c77d93f	remove interactive UX; add `inspect-deal` cmd	2021-03-18 14:53:03 +02:00
Aayush Rajasekaran	a54c6bfb05	Merge pull request #4653 from filecoin-project/4621-delete-wallet Aftering importing a previously deleted key, be able to delete it again.	2021-03-14 20:36:56 -04:00
Aayush Rajasekaran	c3d1534044	Merge pull request #5743 from filecoin-project/feat/keep-unsealed-by-default Default the AlwaysKeepUnsealedCopy flag to true	2021-03-14 15:47:07 -04:00
Łukasz Magiera	1aea1ebb9d	Merge pull request #5801 from filecoin-project/fix/fsm-input-stuck storagefsm: Trigger input processing when below limits	2021-03-12 19:02:15 +01:00
Łukasz Magiera	08472f4fac	batch deal input processing test	2021-03-12 15:41:46 +01:00
Łukasz Magiera	e5226412fb	builder: set modules.PaychAPI correctly	2021-03-12 11:27:00 +01:00
whyrusleeping	cbc7f1c244	fix paychmgr constructor to take an easier to implement interface	2021-03-12 00:39:20 -08:00
whyrusleeping	e0b650d4ec	make rpcstatemanager its own package, for easy reuse	2021-03-11 18:21:18 -08:00
whyrusleeping	bf42a3be5e	move DI stuff for paychmgr into modules	2021-03-11 17:35:31 -08:00
Łukasz Magiera	bd8864ae43	Merge pull request #5755 from filecoin-project/feat/kvlog Metadata datastore log	2021-03-10 14:13:01 +01:00
Łukasz Magiera	05274943c9	Merge pull request #5620 from filecoin-project/feat/disable-owner-worker-fallback miner: Config to disable owner/worker address fallback	2021-03-10 11:21:39 +01:00
Łukasz Magiera	d11f2def6e	Merge branch 'feat/max-deal-collateral-multiplier' into next	2021-03-10 11:13:15 +01:00
Łukasz Magiera	0a513952c8	better DisableWorkerFallback doc	2021-03-09 23:42:08 +01:00
Łukasz Magiera	d4a981f327	fix lint	2021-03-09 23:38:28 +01:00
Łukasz Magiera	ca7e70bf3a	Metadata datastore log	2021-03-09 22:33:01 +01:00
Łukasz Magiera	260ddacc47	Fix wallet import cycle, better trash key backup	2021-03-09 16:52:51 +01:00
jennijuju	49c44b70c2	Aftering importing a previously deleted key, be able to delete it again. Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>	2021-03-09 16:43:01 +01:00
Łukasz Magiera	1c62d7a40f	Merge pull request #5729 from filecoin-project/feat/api-no-dep-build Feat/api no dep build	2021-03-09 16:01:53 +01:00
Łukasz Magiera	5388f3e6f8	Add connmgr metadata to NetPeerInfo	2021-03-09 00:22:48 +01:00
Łukasz Magiera	6591af960f	Merge pull request #4992 from filecoin-project/feat/splitstore hot/cold blockstore segregation (aka. splitstore)	2021-03-08 22:01:06 +01:00
Łukasz Magiera	305c2ec77d	miner: Config to disable owner/worker addcess fallback	2021-03-08 21:33:46 +01:00
Łukasz Magiera	b13046293f	Merge pull request #5735 from filecoin-project/feat/pubsub-ip-colocatio-whitelist Add configuration option for pubsub IPColocationWhitelist subnets	2021-03-08 21:01:39 +01:00
Dirk McCormick	a6c8b56ab6	feat: Default the AlwaysKeepUnsealedCopy flag to true	2021-03-08 16:35:46 +01:00
vyzo	64646de761	add configuration option for pubsub IPColocationWhitelist subnets	2021-03-06 20:05:32 +02:00
vyzo	4a74f752c0	implement extended peer info in net peers cli	2021-03-06 19:14:13 +02:00
whyrusleeping	e7a1d72ba8	extract build/version to api/version, remove api package dep on build	2021-03-05 14:28:13 -08:00
vyzo	6b8c60a659	don't ID wrap the hotstore	2021-03-05 14:46:18 +02:00
vyzo	dd0c308427	move Blockstore config to FullNode, rename to Chainstore and add default for HotStoreType	2021-03-05 14:46:18 +02:00
vyzo	6014273e69	storage miner doesn't need a splitstore	2021-03-05 14:46:18 +02:00
Raúl Kripalani	4b1e1f4b52	rename liveset => markset; rename snoop => tracking store; docs.	2021-03-05 14:46:18 +02:00
Raúl Kripalani	8cfba5b092	renames and polish.	2021-03-05 14:46:18 +02:00
Raúl Kripalani	b1b452bc0f	remove dependency from blockstore/splitstore => chain/store.	2021-03-05 14:46:18 +02:00
Raúl Kripalani	1a804fbdec	move splitstore into blockstore package.	2021-03-05 14:46:18 +02:00
Raúl Kripalani	1b51c10d78	split off lmdb support to a different branch.	2021-03-05 14:46:18 +02:00
vyzo	0fc2f3a26f	fix post-rebase compilation errors	2021-03-05 14:46:18 +02:00
vyzo	97abbe1eca	add (salted) bloom filter liveset	2021-03-05 14:46:17 +02:00
vyzo	73259aa350	add configuration for splitstore and default to a simple compaction algorithm	2021-03-05 14:46:17 +02:00
vyzo	b83994797a	separate LMDB options for hotstore and tracking stores	2021-03-05 14:46:17 +02:00
vyzo	68b6f913c7	propagate useLMDB option to splitstore through DI	2021-03-05 14:46:17 +02:00
vyzo	99c7d8e3eb	more informative names for the hotstore directories	2021-03-05 14:46:17 +02:00
vyzo	a586d42c3b	make hot store DI injectable in the split store, default to badger.	2021-03-05 14:46:17 +02:00
vyzo	622b4f7d9d	hook splitstore into DI	2021-03-05 14:46:17 +02:00
Łukasz Magiera	448813d2fe	Merge pull request #5695 from filecoin-project/feat/segregate-blockstores segregate chain and state blockstores	2021-03-03 10:53:46 +01:00
Dirk McCormick	ced455bba9	feat: add configurable max provider collateral	2021-03-02 10:24:57 +01:00
Łukasz Magiera	8bd5173a54	Merge branch 'next' into refactor/lib/blockstore	2021-03-01 19:04:40 +01:00
Łukasz Magiera	3f5b17dcfc	Merge pull request #5694 from filecoin-project/asr/nits Correct some logs	2021-03-01 18:22:40 +01:00
Raúl Kripalani	3795cc2bd2	segregate chain and state blockstores. This paves the way for better object lifetime management. Concretely, it makes it possible to: - have different stores backing chain and state data. - having the same datastore library, but using different parameters. - attach different caching layers/policies to each class of data, e.g. sizing caches differently. - specifying different retention policies for chain and state data. This separation is important because: - access patterns/frequency of chain and state data are different. - state is derivable from chain, so one could never expunge the chain store, and only retain state objects reachable from the last finality in the state store.	2021-02-28 22:49:44 +00:00
Raúl Kripalani	9f0c68bb70	fix unused import post-merge.	2021-02-28 19:57:37 +00:00
Raúl Kripalani	7f0f7d0b36	Merge branch 'master' into refactor/lib/blockstore	2021-02-28 19:55:23 +00:00
Aayush Rajasekaran	48bce386cd	Correct some logs	2021-02-28 01:18:29 -05:00
Łukasz Magiera	29c9fa3137	Merge pull request #5612 from filecoin-project/feat/cc-fsm-cleanup storagefsm: Cleanup CC sector creation	2021-02-26 11:39:28 +01:00
Łukasz Magiera	e49a412f6d	Merge pull request #5375 from filecoin-project/feat/refactor-fsm-input storagefsm: Rewrite input handling	2021-02-25 14:28:04 +01:00
Łukasz Magiera	26399dba70	Update markets, cbor-gen with soft map decoding	2021-02-19 20:11:43 +01:00
Łukasz Magiera	90b8612cf4	Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input	2021-02-16 22:39:06 +01:00
Łukasz Magiera	39ad3d3b27	Check format in client commP util	2021-02-16 20:48:31 +01:00
Łukasz Magiera	fd90c03018	Roturn SectorID from PledgeSector	2021-02-16 19:16:35 +01:00
Łukasz Magiera	f719765069	storageminer: exit PledgeSector after sectors enter sealing pipeline	2021-02-16 17:41:58 +01:00
Łukasz Magiera	46b880a6b9	storageminer: Skip uninitialized sectors in SectorsList	2021-02-16 17:34:06 +01:00
Łukasz Magiera	fc5e243c92	storagefsm: Cleanup CC sector creation	2021-02-16 17:14:59 +01:00
Łukasz Magiera	dd82729f60	Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input	2021-02-16 13:45:39 +01:00
Dirk McCormick	a6e3856776	Revert "feat: deals - show data transfer %" This reverts commit `b6c9ddccff`.	2021-02-16 12:32:45 +01:00
Łukasz Magiera	7111e924cc	Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input	2021-02-11 13:30:22 +01:00
Łukasz Magiera	95e47cf998	Merge pull request #5553 from filecoin-project/feat/data-xfer-percent show data transfer % for storage deals	2021-02-11 13:29:43 +01:00
Łukasz Magiera	0c6aef221d	Fix error logging format strings	2021-02-11 12:00:26 +01:00
Dirk McCormick	b6c9ddccff	feat: deals - show data transfer %	2021-02-10 09:56:23 +01:00
whyrusleeping	e387f3810e	add code cid to stateReadState output	2021-02-09 14:18:19 -08:00
Łukasz Magiera	e92b8b24b1	Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input	2021-02-09 14:04:21 +01:00
Łukasz Magiera	adeb4cefee	node builder: Cleanup and add some docstrings	2021-02-09 13:52:03 +01:00
Łukasz Magiera	6de62411c6	Merge pull request #5538 from filecoin-project/feat/deal-publisher-force add commands to list pending deals and force publish	2021-02-05 23:00:52 +01:00
Łukasz Magiera	b3f4e50c58	market: APIs to manage pending deals	2021-02-05 18:58:55 +01:00
Łukasz Magiera	2c772d6fda	skip TestDealMining in short mode	2021-02-05 13:18:41 +01:00
Łukasz Magiera	8f603717a6	Merge remote-tracking branch 'origin/master' into steb/refactor-consistent-tipset-methods	2021-02-05 13:08:49 +01:00
whyrusleeping	0efb4ca3a7	add an api endpoint to get pending transactions for multisigs	2021-02-03 20:46:10 -08:00
Łukasz Magiera	ba571794dc	Merge pull request #5309 from filecoin-project/feat/deal-batch-publish batch publish deal messages	2021-02-04 01:19:56 +01:00
Łukasz Magiera	6e7fcb7ee6	Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input	2021-02-03 14:01:56 +01:00
Łukasz Magiera	9b65793509	Merge pull request #5404 from filecoin-project/feat/test-window-post Retry proofs if we generate an incorrect one	2021-02-03 13:04:33 +01:00
Łukasz Magiera	1e7a107484	Merge pull request #5433 from filecoin-project/feat/keep-unsealed-cfg Add miner config to always keep ensealed deal copies	2021-02-03 12:40:02 +01:00
Łukasz Magiera	332ea8a126	Merge remote-tracking branch 'origin/master' into feat/deal-batch-publish	2021-02-02 18:21:14 +01:00
Dirk McCormick	a19d6ce8a3	fix: zero price-per-byte retrieval	2021-02-01 16:55:18 +01:00
Dirk McCormick	987f41011a	refactor: move publish msg config inside deals config	2021-02-01 10:23:05 +01:00
Steven Allen	feb24c9684	choose the correct nonce Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>	2021-01-30 10:18:51 -08:00
Łukasz Magiera	56a9d05491	Merge remote-tracking branch 'origin/master' into steb/refactor-consistent-tipset-methods	2021-01-30 12:05:21 +01:00
Raúl Kripalani	d1104fec4c	rename blockstores for consistency.	2021-01-29 23:17:25 +00:00
Raúl Kripalani	af429dba0f	Merge branch 'master' into chore/snake_context_through_blockstore_init	2021-01-29 21:21:46 +00:00
Raúl Kripalani	b0cbc932bd	consolidate all blockstores in blockstore package.	2021-01-29 20:01:00 +00:00
Steven Allen	8d3cc632ac	add support for running pre-migrations and caching their results This can significantly speedup state migrations.	2021-01-27 12:55:53 -08:00
Łukasz Magiera	d4c7b63aac	Add miner config to always keep ensealed deal copies	2021-01-26 17:50:31 +01:00
Raúl Kripalani	a1da1dab85	add context to LockedRepo#Datastore().	2021-01-26 11:01:43 +00:00
Łukasz Magiera	fabcbb621d	Merge remote-tracking branch 'origin/master' into next	2021-01-26 00:47:22 +01:00
Łukasz Magiera	422f0991e9	Merge pull request #5379 from filecoin-project/asr/disputer Build a WindowPoSt disputer	2021-01-26 00:46:26 +01:00
Aayush Rajasekaran	29d3d746a3	Build a WindowPoSt disputer	2021-01-25 17:26:09 -05:00
Raúl Kripalani	ce3af308ed	Merge branch 'master' into chore/snake_context_through_blockstore_init	2021-01-25 19:31:41 +00:00
Dirk McCormick	adac340f3f	feat: batch publish deal messages	2021-01-25 14:25:57 +01:00
Steven Allen	0a294f146f	ensure we can't dispute a correct window post	2021-01-22 14:30:54 -08:00
Steven Allen	8de4e5e005	retry proofs if we generate an incorrect one	2021-01-22 13:31:47 -08:00
Steven Allen	0131b077be	re-enable watchdog	2021-01-22 11:19:37 -08:00
Jakub Sztandera	8c7fda4441	fix(gas): when estimating GasLimit only apply priors up to the nonce The bug is applying all messages from given From address are priors before appling the message that we are estimating. If user tries replacing message in the middle with gas limit estimation then message sequence is off and user will either get an execution error or gas mis-esimation. Resolves #5402 Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-01-22 18:50:02 +01:00
Steven Allen	1b67eed82c	make the dispute test run by itself It's kind of slow.	2021-01-21 20:16:03 -08:00
Steven Allen	0ba1306969	test prove then dispute	2021-01-21 19:48:34 -08:00
Steven Allen	d412ccdb3f	disable the watchdog It's causing the tests to flake.	2021-01-21 18:28:37 -08:00
Steven Allen	026d236252	more reliably close in tests	2021-01-21 15:21:20 -08:00
Steven Allen	03cd3760bb	correctly pick the seal type based on the network version Of course, we should really just run all of our tests post actors v2.	2021-01-21 15:21:20 -08:00
Aayush Rajasekaran	ab90a3b2bc	Actors update: MinerInfo.SealProofType has been removed	2021-01-21 15:21:19 -08:00
Aayush Rajasekaran	82635f2515	Rename Deadline.PoStSubmissions to PartitionsPoSted	2021-01-21 15:21:19 -08:00
Łukasz Magiera	1070ad2289	storagefsm: Drop unused TargetWaitDealsSectors	2021-01-21 22:20:16 +01:00
Łukasz Magiera	542357a1df	storagefsm: Start packing correctly	2021-01-21 17:40:50 +01:00
Łukasz Magiera	239d6f8f4d	storagefsm: Rewrite input handling	2021-01-21 17:40:50 +01:00
Raúl Kripalani	b9adf95010	watchdog: increase heapprof capture threshold to 90%.	2021-01-21 14:06:18 +00:00
Raúl Kripalani	931cfe1ed1	upgrade to raulk/go-watchdog@v1.0.1 This pulls in the improvements introduced in: - https://github.com/raulk/go-watchdog/releases/tag/v1.0.0 - https://github.com/raulk/go-watchdog/releases/tag/v1.0.1 Lotus tries to initialize the watchdog in the following order of precedence: 1. If a max heap limit has been provided, initialize a heap-driven watchdog. 2. Else, try to initialize a cgroup-driven watchdog. 3. Else, try to initialize a system-driven watchdog. 4. Else, log a warning that the system is flying solo, and return. This PR also enabled automatic heap profile capture when memory usage surpasses 90% of the limit. Profiles are written to <LOTUS_HOME>/heapprof. A single heap profile is captured per episode, with a max of 10 episodes captured during the lifetime of the process. Episode = instance of usage climbing above the 90% threshold.	2021-01-20 18:09:19 +00:00
Łukasz Magiera	7a042ab21f	Merge pull request #5210 from filecoin-project/feat/mkts-miner-dont-dial-client Miner should not dial client on restart	2021-01-20 13:37:09 +01:00
Dirk McCormick	1206154718	feat: update to go-fil-markets v1.1.0 for better retry config	2021-01-20 10:00:02 +01:00
Łukasz Magiera	0a85dc6232	Merge pull request #5369 from filecoin-project/feat/gas-premium-55 Use 55th percentile instead of median for gas-price	2021-01-19 19:10:47 +01:00
Dirk McCormick	827a473391	feat: expose StateSearchMessage on gateway	2021-01-19 11:27:43 +01:00
Jakub Sztandera	eeadfedbe6	Use 55th percentile instead of median for gas-price The aim is to put some negative pressure on gas-premium instead of maintining status quo. 55th percentile instead of median should not make much difference for block inclusion timing. Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2021-01-18 15:44:20 +01:00
Dirk McCormick	835fd4b23c	feat: markets - miner should not dial client on restart	2021-01-14 16:38:56 +01:00
Łukasz Magiera	144b5a1350	perning termination API	2021-01-14 12:37:23 +01:00
Łukasz Magiera	1564db1fce	Sector termination test	2021-01-14 00:11:41 +01:00
Łukasz Magiera	3522c8d45a	SectorTerminateFlush API	2021-01-13 23:32:04 +01:00
Łukasz Magiera	52cc2cd3eb	Initial sector termination support	2021-01-13 00:42:01 +01:00
Peter Rabbitson	47141042b6	Add comment Co-authored-by: raulk <raul@protocol.ai>	2021-01-12 18:11:18 +01:00
Dirk McCormick	c58086ee27	feat: better CLI for wallet market withdraw and client info	2021-01-07 10:18:29 +01:00
Dirk McCormick	c6820ec059	feat: lotus wallet market add (adds funds to storage market actor)	2021-01-06 14:32:46 +01:00
Peter Rabbitson	9334e73396	Snake a context through the Chain-blockstore creation	2021-01-04 15:27:52 +01:00
Łukasz Magiera	2e154ef6d0	Merge pull request #5176 from filecoin-project/shaodan-miner-sectors-info Shaodan miner sectors info	2020-12-10 20:09:15 +01:00
Łukasz Magiera	0cb2f51549	Merge pull request #5094 from filecoin-project/asr/verified-only Allow miners to filter (un)verified deals	2020-12-10 19:48:22 +01:00
Łukasz Magiera	564b88af52	docsgen, lint	2020-12-10 19:36:02 +01:00
Łukasz Magiera	d347e7ab8f	Merge branch 'miner-sectors-info' of github.com:shaodan/lotus into shaodan-miner-sectors-info	2020-12-10 19:32:26 +01:00
Steven Allen	e164dbb953	Compute the next nonce from the tipset, instead of actually executing the tipset We could also do this in the message pool itself, but I'm not sure if it's worth it?	2020-12-09 11:29:41 -08:00
Steven Allen	b8e3808c4f	Make state tipset usage consistent in the API _Always_ (almost) use the tipset's parent state, instead of computing. Exceptions: * MinerGetBaseInfo. Fixing this would break things so we need to be careful (although we could bump the API version, fix it, then fix the call sites). * StateReplay. This is replaying a message on top of the given tipset. * GasEstimateGasLimit. This executes the message on-top-of the tipset's computed state (unlike call which executes it on the tipset's parent state). * Having this method and Call apply the message at different heights is really weird.	2020-12-09 11:29:40 -08:00
Łukasz Magiera	2ce5a29004	Merge pull request #5150 from filecoin-project/feat/list-deals-xfer-id show data transfer ID in list-deals	2020-12-09 19:39:46 +01:00
Jakub Sztandera	6ca5caef31	Refactor DefaultMessageSendSpec Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2020-12-08 20:51:27 +01:00
Dirk McCormick	72cb130e17	feat: show data transfer ID in list-deals	2020-12-08 15:23:26 +01:00
Dan Shao	d56170d70e	Optimize sectors info loading	2020-12-07 13:42:37 +08:00
Ingar Shu	13c8a235b6	Use FundManager to withdraw funds, add MarketWithdraw to API	2020-12-03 13:08:18 -08:00
Łukasz Magiera	dc06d30f52	Merge pull request #5101 from filecoin-project/raulk/memory-watchdog introduce memory watchdog; LOTUS_MAX_HEAP	2020-12-03 15:46:09 +01:00
Raúl Kripalani	f9cbf6d526	watchdog: add LOTUS_DISABLE_WATCHDOG escape hatch.	2020-12-03 13:16:10 +00:00
Raúl Kripalani	d033f8aab1	watchdog: remove silence period, reduce tick interval to 5 secs.	2020-12-02 22:31:57 +00:00
Raúl Kripalani	e34a759889	avoid global ResourceConstraints.	2020-12-02 22:26:30 +00:00
Raúl Kripalani	2ef8acdfc8	back out index cache usage in badger blockstore.	2020-12-02 21:35:13 +00:00
Łukasz Magiera	c3a0feef96	miner: Improve default address config	2020-12-02 21:54:38 +01:00
Łukasz Magiera	2fd93a55ac	plumb AddressSelector to stoage fsm	2020-12-02 21:47:54 +01:00
Łukasz Magiera	783971d2c8	docsgen, gofmt	2020-12-02 20:46:40 +01:00
Łukasz Magiera	f74a1b70ab	miner: Fix actor control list cli	2020-12-02 20:46:07 +01:00
Łukasz Magiera	7246ef273f	miner: Control address config for (pre)commits	2020-12-02 19:58:00 +01:00
Raúl Kripalani	e8a5e0d2fe	integrate memory watchdog; impose limits on badger caches.	2020-12-02 16:40:28 +00:00
Raúl Kripalani	655aea1018	Merge branch 'master' into raulk/badger-index-cache	2020-12-02 11:46:57 +00:00
Aayush Rajasekaran	370817eb60	Allow miners to filter (un)verified deals	2020-12-02 01:32:34 -05:00
Łukasz Magiera	95eaf13b5a	sectorstorage: Fix tests	2020-12-02 00:36:32 +01:00
Łukasz Magiera	c6f85886bf	miner: Add slow mode to proving check	2020-12-02 00:32:14 +01:00
Łukasz Magiera	82b5cb89cd	Merge pull request #5061 from filecoin-project/fix/err-late-deals Error out deals that are not activated by proposed deal start epoch	2020-12-01 17:02:06 +01:00
Łukasz Magiera	614f45dcfb	Merge pull request #5041 from filecoin-project/feat/5013 Sector check command	2020-12-01 14:59:01 +01:00
Dirk McCormick	cb044f83a7	fix: ensure deal start is far enough in future for tests	2020-12-01 14:25:48 +01:00
Jakub Sztandera	65d50c76d6	bonus: fix WalletSign using t0 addresses in WalletSignMessage Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>	2020-11-27 17:19:41 +01:00
Łukasz Magiera	fcec665267	Merge remote-tracking branch 'origin/master' into feat/deal-await-precommit	2020-11-26 11:51:36 +01:00
s1m0n21	663b3f4f21	add a sector check command	2020-11-26 15:02:43 +08:00
Łukasz Magiera	2e544e3e6a	configure SimultaneousTransfers in node/builder	2020-11-25 19:57:38 +01:00
hannahhoward	694834e8d5	feat(graphsync): configure simultaneous requests allow configuration of the number of simultaneous requests graphsync will process at once	2020-11-24 14:32:30 -08:00
Dirk McCormick	239e180683	feat: markets - separate watching for pre-commit from prove-commit	2020-11-24 13:24:41 -08:00
hannahhoward	afec58f780	feat(markets): add logging on data-transfer to miner	2020-11-23 12:43:30 -08:00
Łukasz Magiera	be2a5942fb	Merge pull request #4082 from leviok/fix-nil-pointer fix nil pointer in StateSectorPreCommitInfo	2020-11-23 18:58:32 +01:00
Łukasz Magiera	0e39e760e5	Merge pull request #4929 from filecoin-project/feat/update-markets-commp-extraction Remove all temp file generation for deals	2020-11-23 18:43:00 +01:00
Łukasz Magiera	7d540905ac	Don't return nil in StateSectorPreCommitInfo	2020-11-23 12:14:49 +01:00
Dirk McCormick	4a8df0c3fa	feat: add data transfer logging	2020-11-23 10:31:39 +01:00
leviok	9fe3649dd2	correct it	2020-11-23 11:08:21 +08:00
leviok	9b199f0476	fix nil pointer	2020-11-23 11:08:21 +08:00
hannahhoward	218f7c2c5d	feat(markets): update markets, extract commp	2020-11-20 13:29:14 -08:00
Łukasz Magiera	4483d94db3	Merge pull request #4902 from GFZRZK/GFZRZK/patch/remove_unuseful_code_in_newstorageask just return storedask.NewStoredAsk to reduce unuseful code	2020-11-20 16:28:03 +01:00
Łukasz Magiera	94763c2aaa	Merge pull request #4849 from filecoin-project/feat/conngater Connection Gater support	2020-11-20 16:25:02 +01:00
Raúl Kripalani	44040f0f3f	badger options: enable IndexCache and CompactL0OnClose.	2020-11-20 00:33:43 +00:00
Raúl Kripalani	55061ae6fb	fix blockstore directory not created automatically.	2020-11-19 15:55:02 +00:00
wangchao	49f8590578	just return storedask.NewStoredAsk to reduce unuseful code	2020-11-18 13:53:48 +08:00
Aayush Rajasekaran	68a16afeda	Merge branch 'master' into release/v1.2.0	2020-11-17 19:40:05 -05:00
Łukasz Magiera	50146fb9b9	Merge pull request #4872 from filecoin-project/raulk/fix-blockstore-import fix badger double open on daemon --import-snapshot; chainstore lifecycle	2020-11-18 01:36:16 +01:00
Łukasz Magiera	3536ec9f85	Merge pull request #4879 from filecoin-project/fix/4833 Fix MaxSealingSectrosForDeals getting reset to 0	2020-11-17 23:41:22 +01:00
hannahhoward	a2abeef727	feat(markets): upgrade to fix cid recording issue	2020-11-16 17:57:57 -08:00
Łukasz Magiera	dd9682f2be	Fix MaxSealingSectrosForDeals getting reset to 0	2020-11-17 00:28:49 +01:00
Raúl Kripalani	0c6072a1a0	chainstore lifecycle: close via Close() instead of context.	2020-11-16 22:28:59 +00:00
Steven Allen	bead8ac38e	use the miner's proof type when proposing storage deals	2020-11-16 19:04:17 +01:00
Steven Allen	638900f97f	Remove static proof type configuration from market subsystem Instead, use proof type from miner actor. This will, in turn, use the upgraded proof type if/when it's switched at runtime. TODO: Consider making this some form of config option instead?	2020-11-16 19:04:17 +01:00
Steven Allen	d8afd71f4c	add a test to demonstrate SDR upgrade while committing sectors	2020-11-16 19:04:17 +01:00
Steven Allen	869867e92e	fix compile	2020-11-16 19:04:17 +01:00
Steven Allen	508375a55c	Upgrade seal proof version in miner info after the upgrade. This is terrible, but we don't (currently) update this in the miner info. Q: Maybe we should delay this by a few epochs? Some pre-commits could fail if we get a reorg.	2020-11-16 19:04:17 +01:00
Łukasz Magiera	915c5e0fdd	Get most tests to pass	2020-11-16 19:03:30 +01:00
Steven Allen	931979809f	get lotus building	2020-11-16 19:03:30 +01:00
Steven Allen	219c695967	fix storage adapter miner	2020-11-16 19:03:30 +01:00
Łukasz Magiera	6bea9dd178	Making sealing logic work with multiple seal proof types	2020-11-16 19:03:30 +01:00
Raúl Kripalani	d7b4f92f1f	add context to ChainStore.	2020-11-16 15:52:19 +00:00
Łukasz Magiera	49d3bb9497	Merge pull request #4851 from filecoin-project/fix/daemon-test-mode-restart Truncate genesis file before generating	2020-11-16 16:13:33 +01:00
Jakub Sztandera	afe6c51016	Merge pull request #4813 from filecoin-project/chore/blockstore_nits Chore/blockstore nits	2020-11-15 19:52:10 +01:00
Aayush Rajasekaran	1af1ceb1e0	Merge pull request #3646 from filecoin-project/feat/delayed-pubsub-subscribe delayed pubsub subscribe for messages topic	2020-11-13 17:44:32 -05:00
vyzo	91e2530e11	remove dtypes dependency from api; move NetBlockList type to api/types.go	2020-11-13 21:11:17 +02:00
Dirk McCormick	f9fc601567	fix: truncate genesis file before generating	2020-11-13 17:30:12 +01:00
vyzo	4b187b85eb	close existing connections that match the filters when adding rules to the connection gater	2020-11-13 14:38:13 +02:00
vyzo	b71f069854	implement connection gater API	2020-11-13 13:59:38 +02:00
vyzo	5a0b9f4197	connectiong gater API scaffolding	2020-11-13 12:17:48 +02:00
vyzo	4b1d1b6540	connection gater for libp2p host	2020-11-13 11:44:29 +02:00
Łukasz Magiera	99031bae17	Merge pull request #4709 from filecoin-project/feat/mute-logs Make some logs quieter	2020-11-12 20:40:10 +01:00
Dirk McCormick	26e12e6f6a	fix: dont crash on startup if funds migration fails	2020-11-12 17:15:05 +01:00
Peter Rabbitson	29817b6617	Remove all Chain GC wrappings/types There is no practical way to do GC against the chainstore given the current state of affairs: remove all GC-related types for the time being.	2020-11-12 06:49:53 +01:00
Peter Rabbitson	7be7a9b5e3	Stop referring to github.com/ipfs/go-ipfs-blockstore outside of lib	2020-11-12 06:01:35 +01:00
Łukasz Magiera	09f9f871a3	Create a command to abort sealing calls	2020-11-11 17:39:12 +01:00
hannahhoward	9f7204ee26	feat(builder): finish new FundManager setup Finish setup of new FundManager and provide a migration for previously reserved funds	2020-11-10 21:11:05 -08:00
Dirk McCormick	0d243bb824	refactor: integrate new FundManager	2020-11-10 21:11:05 -08:00
Aayush Rajasekaran	735c04f999	Merge pull request #4681 from filecoin-project/badger-viewable move to native badger blockstore; leverage zero-copy View() to deserialize in-place	2020-11-10 18:08:12 -05:00

... 12 13 14 15 16 ...

3079 Commits