vyzo
a9ec408844
collect resource manager metrics
2022-02-14 19:46:05 +02:00
Masih H. Derkani
7e7844f169
Fix log message formatting when instantiating index provider
2022-02-11 12:15:21 +00:00
Will
296eab3045
Merge pull request #8045 from gammazero/feat/cid-to-piece-idx
...
Add indexer pubsub message authentication and rate limiting
2022-02-10 13:49:58 -08:00
gammazero
eedcf910eb
change scoring based on review
2022-02-10 10:53:07 -08:00
gammazero
b72cd90c19
Add indexer message scoring
2022-02-10 09:21:21 -08:00
gammazero
3f3d61b043
Service creation takes interface, not implementation
2022-02-10 08:41:18 -08:00
Jennifer Wang
de1f4792cd
Merge remote-tracking branch 'origin/master' into jen/v15
2022-02-09 19:27:34 -05:00
Łukasz Magiera
44fd0e3349
Merge pull request #8008 from filecoin-project/feat/splitstore-sortless-compaction
...
splitstore sortless compaction
2022-02-09 17:17:57 +00:00
Jennifer Wang
6ed97f0d04
Merge branch 'release/v1.14.0' into jen/v15
2022-02-09 03:17:38 -05:00
zenground0
e32704e7f6
Update node doc gen
2022-02-08 12:56:32 -05:00
zenground0
1e524b7210
Review Response
2022-02-08 12:39:35 -05:00
zenground0
5a09b6496d
Actually read config value
2022-02-08 12:39:35 -05:00
zenground0
2aa5911fb9
config gen
2022-02-08 12:39:35 -05:00
zenground0
4d8b912acc
New config for turning off auto deal creation
2022-02-08 12:39:35 -05:00
Masih H. Derkani
cc61650f86
Upgrade to latest index-provider and set miner ID as extra gossip data
...
Upgrade to the latest `index-provider` which upgrades the go-legs
protocol to allow the inclusion of extra gossip data that may be used
for gossip validation purposes. In the case of lotus gossip message
validators the miner ID is used to verify the sender's peer ID on chain.
Relates to:
- https://github.com/filecoin-project/lotus/pull/8045
2022-02-08 14:32:03 +00:00
gammazero
b2805823ce
Pass to validator the interfaces needed to get miner info
2022-02-08 04:55:59 -08:00
gammazero
1dc6a2fea6
Add indexer pubsub message authentication and rate limiting
2022-02-08 02:53:25 -08:00
Aarsh Shah
399a1afd94
Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx
2022-02-08 12:37:57 +04:00
vyzo
1221c0ba3f
make gen
2022-02-06 12:28:21 +02:00
vyzo
03352ea863
make badger the default splitstore markset type
2022-02-06 11:21:16 +02:00
Aarsh Shah
e155eb9fa3
Merge pull request #8016 from filecoin-project/nonsense/add-fullnode-api
...
add indexProvider host to markets
2022-02-04 12:56:36 +04:00
Aarsh Shah
4691b2b809
changes to the indexer message relay PR
2022-02-04 12:15:01 +04:00
Aarsh Shah
3ecf478ff0
Merge pull request #8026 from gammazero/feat/cid-to-piece-idx
...
Lotus chain nodes relay indexer pubsub messages
2022-02-04 12:05:46 +04:00
gammazero
c084130d3e
Lotus chain nodes relay indexer pubsub messages
...
Content providers announce the availability of indexer data using gossip pubsub. The content providers are not connected directly to indexers, so the pubsub messages are relayed to indexers via chain nodes. This PR makes chain nodes relay gossip pubsub messages, on the /indexer/ingest/<netname> topic.
2022-02-03 14:56:21 -08:00
Will Scott
7247f8e41f
add net protect api methods
...
fix #8015
2022-02-03 17:24:49 +01:00
Anton Evangelatov
dde2ab59ae
fixup
2022-02-03 15:53:46 +01:00
Anton Evangelatov
595ad44ee7
refactor indexprovider libp2p host connection to fullnode with meshcreator
2022-02-03 15:44:18 +01:00
Masih H. Derkani
575e6add43
Rename IndexerProvider to IndexProvider and add more docs
...
Rename the config section corresponding to indexing to `IndexProvider`
for better readability.
Update existing docs for better clarity and add docs for config
parameters embedded from `index-provider` `Ingest` config library.
2022-02-03 13:10:47 +00:00
Masih H. Derkani
235377585d
Clarify the purpose of MaxSimultaneousTransfers config
...
Clarify the documentation of `MaxSimultaneousTransfers` to state that
the "transfer" refers to multihash syncing only and not the data itself.
2022-02-03 11:51:01 +00:00
Anton Evangelatov
faa3a79a5f
use NetAddrListener iface
2022-02-02 16:01:19 +01:00
Anton Evangelatov
2790cf35af
change fullnodeApi to listen addrs
2022-02-02 13:06:53 +01:00
Anton Evangelatov
f9b63c5f8c
add idxProvHost to StorageProvider
2022-02-02 13:03:41 +01:00
Anton Evangelatov
db5e4da268
add fullnodeApi to StorageProvider
2022-02-02 13:03:41 +01:00
Łukasz Magiera
98f4d37f1a
Merge remote-tracking branch 'origin/master' into feat/post-worker
2022-01-31 19:50:52 +00:00
Darko Brdareski
e51ce5c508
Merge remote-tracking branch 'upstream/master' into bloxico/system-test-matrix
2022-01-27 10:57:56 +01:00
Darko Brdareski
8c41e17c93
Fix typo in client_test annotations
2022-01-27 10:56:33 +01:00
Masih H. Derkani
5d72ebe0cc
Upgrade index-provider and update topic name
...
Update the topic name to be consistent with default
configured in storetheindex.
Upgrade to the latest `index-provider`.
2022-01-27 09:48:10 +00:00
Aayush
1b18236f91
feat: sealer: allow users to abort in-flight snap upgrades
2022-01-25 13:01:51 -05:00
Łukasz Magiera
4235a97cf4
retrieval: OffChainRetrieval config
2022-01-20 18:19:27 +01:00
Łukasz Magiera
8b19b84140
paych: option to force off-chain get
2022-01-20 18:19:26 +01:00
Łukasz Magiera
8e46b9ea5d
paych: API to pre-fund channels
2022-01-20 18:15:46 +01:00
Aarsh Shah
b8bf2af3e3
upgrade on master
2022-01-20 16:34:13 +04:00
Aarsh Shah
9ee940098d
Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx
2022-01-20 16:21:48 +04:00
Łukasz Magiera
e476cf7968
Merge remote-tracking branch 'origin/master' into feat/post-worker
2022-01-20 13:15:48 +01:00
vyzo
73ec10a49e
close the rcmgr on shutdown
2022-01-20 11:47:39 +02:00
vyzo
325a4c0e5b
fix lint
2022-01-20 11:46:39 +02:00
vyzo
554848fe6b
add resource manager Net(Set)Limit api
2022-01-20 11:36:11 +02:00
vyzo
d05d5bcb70
add resource manager NetStat api
2022-01-20 11:36:11 +02:00
vyzo
8d3f98fe38
instantiate resource manager in DI
2022-01-20 11:36:11 +02:00
vyzo
826cdb2186
use the libp2p connmgr
2022-01-20 11:36:11 +02:00
Łukasz Magiera
b38141601c
Untangle ffi from api
2022-01-18 11:57:04 +01:00
Łukasz Magiera
03fa955e9e
Cleanup imports
2022-01-18 11:37:24 +01:00
Łukasz Magiera
f2496d87c9
post workers: Snapdeals updates
2022-01-18 11:25:04 +01:00
Łukasz Magiera
0c062f9c08
Merge remote-tracking branch 'origin/master' into feat/post-worker
2022-01-18 10:49:42 +01:00
hannahhoward
92d56d82f4
feat(deps): update markets stack
...
update go-fil-markets, go-data-transfer 1.13.0, go-graphsync 0.12.0
2022-01-14 17:21:04 -08:00
vyzo
7e43bfc109
hook the upgrade schedule to splitstore start
2022-01-14 17:14:32 -05:00
zenground0
d6aa17e21f
Snap Deals Integration
...
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
2022-01-14 17:14:32 -05:00
Łukasz Magiera
4a874eff70
post workers: Cleanup, tests
2022-01-14 14:17:52 +01:00
c r
da6752eccb
feat: #7747 sealing: Adding conf variable for capping number of concurrent unsealing jobs ( #7884 )
...
* adding the new variables- now time for logic
* putting parameters into right placeS
* adding unsealing throttle
* fixing linter issues
* removing one last thing...
2022-01-13 12:26:13 -06:00
Aarsh Shah
8459d75597
update index provider config
2022-01-13 16:15:43 +04:00
Aarsh Shah
b780a64ed9
send markets multiaddrs to the engine
2022-01-13 15:31:15 +04:00
Aarsh Shah
8c8652e7bb
add a new cli
2022-01-12 18:06:48 +04:00
Aarsh Shah
dec79c70a1
fix compilation
2022-01-12 15:03:46 +04:00
Aarsh Shah
21a8f71859
compiles
2022-01-12 14:59:09 +04:00
Aarsh Shah
edfc2c215c
Merge remote-tracking branch 'origin/main' into feat/cid-to-piece-idx
2022-01-12 12:48:54 +04:00
zenground0
33f2d24f54
Snap Deals Integration
...
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
2022-01-10 15:39:38 +05:30
hannahhoward
1078dff3a9
feat(storageminer): also list data transfers with no id
2021-12-22 13:41:56 -08:00
hannahhoward
8a923fb420
fix(storageminer): fix conditional
2021-12-22 13:41:56 -08:00
hannahhoward
177282d9f9
fix(storageminer): lint/mod-tidy/extra checks
2021-12-22 13:41:55 -08:00
hannahhoward
cddf63efe9
feat(storageminer): add api for transfer diagnostics
...
Add API + CLI for inspecting in depth diagnostics on graphsync transfers with a given peer
2021-12-22 13:41:29 -08:00
Aayush Rajasekaran
1bc987772c
Merge pull request #7818 from filecoin-project/asr/vm-nvfactor
...
Refactor: VM: Remove the NetworkVersionGetter
2021-12-21 12:19:53 -05:00
Darko Brdareski
a54ac2ef79
Remove FS, MEM, GAS leftover test anotations.
2021-12-20 17:00:01 +01:00
Darko Brdareski
86cf17c1f4
Remove bad annotation from gas_test.go
2021-12-20 16:51:09 +01:00
Darko Brdareski
dda1a42a2a
Merge branch 'bloxico/system-test-matrix' of https://github.com/filecoin-project/lotus into merge_lotus
2021-12-20 15:48:16 +01:00
Aayush Rajasekaran
3f4eaa99d5
Refactor: State: Rename stmgr::GetNtwkVersion to GetNetworkVersion
2021-12-17 18:43:39 -05:00
Dirk McCormick
b1734f84b3
feat: retrieval ask CLI command
2021-12-17 15:55:12 +01:00
Łukasz Magiera
a4728d3c72
Merge pull request #7578 from filecoin-project/rvagg/SimultaneousTransfersForStoragePerClient
...
feat(graphsync): allow setting of per-peer incoming requests for miners
2021-12-17 14:27:15 +01:00
vyzo
dd327f0b22
plumb more contexts
2021-12-17 11:42:09 +02:00
Rod Vagg
9e7d9affbe
feat(graphsync): allow setting of per-peer incoming requests for miners
2021-12-17 15:04:16 +11:00
Darko Brdareski
3e32aa896c
Annotate client storage deals feature
2021-12-16 14:36:02 +01:00
vyzo
22ce395130
fix some unit tests
2021-12-14 17:05:10 +02:00
vyzo
716b4a3b09
hook a great context in the sky to satisfy DI; sigh...
2021-12-14 16:57:09 +02:00
vyzo
eb48dc9b68
fix issues with new peerstore constructor signature in DI
2021-12-14 16:21:55 +02:00
vyzo
f157ac9b68
update for new ConnManager constructor
2021-12-14 15:30:29 +02:00
Dirk McCormick
a51b08e386
feat: add IndexerProvider AnnounceAddress config
2021-12-13 18:24:05 +01:00
Masih H. Derkani
6b7b9793ab
Set default index provider ingest config values
...
Set default values for index-provider ingest config.
Leave a TODO to use provider config APIs once defaults exposed.
Run `go mod tidy`
2021-12-13 14:44:56 +00:00
Darko Brdareski
0addca1070
Fix bad annotations
2021-12-13 13:41:04 +01:00
vyzo
84710cf27e
plumb more contexts in lotus
2021-12-13 14:26:59 +02:00
Aayush Rajasekaran
dfb65ed89f
Plumb contexts through
2021-12-11 17:04:00 -05:00
vyzo
5d6398f20e
hook the upgrade schedule to splitstore start
2021-12-03 12:11:54 +02:00
Łukasz Magiera
f39942283b
Merge pull request #7706 from filecoin-project/feat/ret-matchding-cars
...
retrieval: Only output matching nodes, MatchPath dagspec
2021-11-29 22:14:25 +01:00
Łukasz Magiera
d019853687
review: Cleanup some comments
...
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
2021-11-29 21:29:00 +01:00
Łukasz Magiera
5b5e6b9e44
retrieval: DagSpec.MatchPath -> ExportMerkleProof
2021-11-29 21:14:00 +01:00
Łukasz Magiera
61791b90ea
retrieval: Only output matching nodes, MatchPath dagspec
2021-11-29 20:40:55 +01:00
Steven Allen
797147097c
Merge pull request #7689 from filecoin-project/disable-mplex
...
disable mplex stream muxer
2021-11-29 09:46:24 -08:00
Łukasz Magiera
77d75b7739
Merge pull request #7688 from filecoin-project/chore/partret_cleanup_comment_and_flow
...
Cleanup partial retrieval codepaths ( zero functional changes )
2021-11-29 16:39:55 +01:00
Łukasz Magiera
8d955d5f30
dagstore mount: Add random access support
2021-11-26 17:40:53 +01:00
Marten Seemann
cf0faf58dc
remove muxer config via LIBP2P_MUX_PREFS env
2021-11-26 11:01:54 +04:00
Marten Seemann
12dc3baff9
disable mplex stream muxer
2021-11-25 09:38:57 -08:00
Peter Rabbitson
d5b4d93575
Cleanup partial retrieval codepaths ( zero functional changes )
...
- Adjust the ExportRef comment to reflect change from #7673
- Simplify control flow in parseDagSpec() ( read diff under -w )
2021-11-25 10:35:05 +01:00
Łukasz Magiera
0c884734a3
Merge pull request #7673 from filecoin-project/chore/DM-level-tests_plus_merkle-proof-cars
...
Chore/dm level tests plus merkle proof cars
2021-11-24 20:43:39 +01:00
Łukasz Magiera
bd4927b494
retrieval: Cleanup some comments
2021-11-24 20:13:49 +01:00
Peter Rabbitson
46937a1b9f
Add tests cementing the DM-level behavior on simple paths
2021-11-24 15:31:15 +01:00
Łukasz Magiera
78949d6c05
retrieval: Fix deadlock in ClientGetRetrievalUpdates
2021-11-23 18:32:56 +01:00
Łukasz Magiera
407c2ed114
retrieval: Drop the RootSelector hack
2021-11-23 17:42:43 +01:00
Łukasz Magiera
25e89d3a7a
retrieval: require unixfs exports to be aligned on block boundary
2021-11-22 13:29:09 +01:00
Łukasz Magiera
9ea229ed5a
retrieval: fix defult ask
2021-11-22 13:04:12 +01:00
Łukasz Magiera
2c583b03ff
retrieval: Support DM-paths in ls
2021-11-22 12:46:29 +01:00
Łukasz Magiera
74f645a098
retrieval: Fix parseDagSpec root check
2021-11-22 12:46:29 +01:00
Łukasz Magiera
597b72e286
retrieval: Fix lint, cli docsgen
2021-11-22 12:46:29 +01:00
Łukasz Magiera
9c119bfdad
retrieval: Make the ls command work
2021-11-22 12:46:17 +01:00
Łukasz Magiera
450d0687da
retrieval: REST export endpoint
2021-11-22 12:46:17 +01:00
Łukasz Magiera
b83a9b902a
gofmt
2021-11-22 12:46:17 +01:00
Łukasz Magiera
b26906963b
retrieval: Support multi-root export
2021-11-22 12:46:17 +01:00
Łukasz Magiera
b9bd061bdd
fix unixfs selector root node selection
2021-11-22 12:46:04 +01:00
Łukasz Magiera
b868769ec8
more retrieval api work
2021-11-22 12:46:02 +01:00
Łukasz Magiera
89138bab4d
Simplify retrieval APIs
2021-11-22 12:45:42 +01:00
Łukasz Magiera
60ea33b1c7
client: cleanup clientRetrieve
2021-11-22 12:45:42 +01:00
Łukasz Magiera
08e297a217
client: Support json selectors in retrieval
2021-11-22 12:45:42 +01:00
Aayush Rajasekaran
32fc03886d
CLI: Add a lotus multisig cancel command
2021-11-18 19:21:17 -05:00
Aarsh Shah
7655e660f3
integrate store-the-hash
2021-11-18 12:07:12 +04:00
Anton Evangelatov
5454aebf13
CLI tools for the DAGStore inverted index ( #7361 )
...
* cli commands for dagstore
* address comments from Dirk
* rename inverted index to piece index
2021-11-17 15:39:45 +02:00
Dirk McCormick
501308239f
feat: CLI command to announce deal to indexers
2021-11-17 12:16:22 +01:00
Aarsh Shah
8d3017febb
make gen should work
2021-11-17 15:02:11 +04:00
Aarsh Shah
caf3cfa7d8
Merge branch 'feat/cid-to-piece-idx' into feat/indexer-provider
2021-11-17 11:43:49 +04:00
Aarsh Shah
bfee6699d2
update go mod and inject store the hash index in dagstore
2021-11-17 11:42:14 +04:00
Dirk McCormick
627d4fbf74
wip: storage miner index provider
2021-11-16 17:45:42 +01:00
TheMenko
b45924a234
annotated repo_test
2021-11-01 10:49:30 +01:00
TheMenko
fb911a4542
annotated repo_test
2021-11-01 01:24:26 +01:00
TheMenko
9297e1b90d
annotated repo_test
2021-10-31 12:24:07 +01:00
Łukasz Magiera
d2e9d21031
Gather graphsync metrics on provider side as well
2021-10-19 19:45:25 +02:00
Łukasz Magiera
32a855b984
Fix lint
2021-10-19 19:22:32 +02:00
Łukasz Magiera
9a993d25d0
Collect and expose graphsync metrics
2021-10-19 19:20:00 +02:00
Jiaying Wang
afe54ab5f3
Merge pull request #7511 from filecoin-project/frrist/fix-lily
...
fix: support node instantiation in external packages
2021-10-13 20:33:17 -04:00
frrist
6cd6732225
fix: support node instantiation in external packages
...
- implement ChainGetPath on ChainModuleAPI
2021-10-13 15:24:28 -07:00
Anton Evangelatov
6e5ccc87cf
cli: add retry for deals stuck in Publish with no funds
2021-10-11 14:03:55 +02:00
Peter Rabbitson
a40aa9f90b
Normlize selector use within lotus
2021-10-07 11:01:29 +02:00
Peter Rabbitson
f8ac98bf46
Adjust error wording
2021-10-05 18:16:14 +02:00
Peter Rabbitson
5e6aceeb0e
Merge remote-tracking branch 'origin/master' into feat/datamodel-selector-retrieval
2021-10-05 18:15:51 +02:00
Jiaying Wang
cbb147de23
Merge pull request #7405 from filecoin-project/feat/update-graphsync-0.10.0
...
Update to graphsync to v0.10.0, enable seperate storage and retrieval transfer limits
2021-10-05 11:50:59 -04:00
Rod Vagg
97c2cdb8d6
traversals: limit link load count for SelectiveCar use as well
2021-10-05 10:47:49 +02:00
Rod Vagg
43f7fd5e10
traversals: limit maximum number of DAG links to traverse
...
Impacts CommP and graphsync transfers
2021-10-05 10:47:49 +02:00
Dirk McCormick
a4e2fce9ca
feat: update to go-fil-markets v1.13.1
2021-10-05 10:40:01 +02:00
hannahhoward
368d72ebfe
feat(graphsync): update to v0.10.0-rc1
...
also add config changes
2021-10-05 14:13:58 +11:00
Peter Rabbitson
af0d9b650d
Error on corner cases with corresponding negative tests
2021-10-04 23:21:24 +02:00
Peter Rabbitson
2a1094fc17
Merge banch feat/go-fil-markets-v1.13.0 into feat/datamodel-selector-retrieval
2021-10-04 21:56:11 +02:00
Dirk McCormick
16973bc2e8
feat: use go-fil-markets@v1.13.0
2021-10-04 15:41:40 +02:00
Jennifer Wang
38e01cba83
Merge branch 'release/v1.12.0' into jen/12tomaster
2021-10-04 02:38:00 -04:00
Aayush Rajasekaran
f004d036dc
Set BatchPreCommitAboveBaseFee correctly
2021-10-01 14:23:41 -04:00
Łukasz Magiera
6fd9d5f28b
Precommit batch balancer support/config
2021-10-01 14:23:41 -04:00
