cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
15,369 Commits 24 Branches 342 Tags 314 MiB
e767981aaf
Commit Graph

2483 Commits

Author SHA1 Message Date
Aarsh Shah
e3f5b7c59f update dagstore top level index db 2022-02-18 14:00:01 +04:00
Masih H. Derkani
a09f386537 Remove redundant IndexProvider config now that host is shared 
Now that host is shared having separate config for listen and announce
addresses along with graphsync limit makes no sense since all of that is
shared with the markets host and datatransfer manager.
 2022-02-16 13:46:44 +00:00
Masih H. Derkani
2ebc111b70 Use the same host and datatransfer as markets for index provider 
Remove the bespoke instantiation of libp2p host and datatransfer manager
for index provider and reuse the existing instances used by markets.

The rationale for reuse is the following:
 1. Separation of host introduces a discovery problem, where without
    gossipsub the index provider endpoint will not be discoverable.
    Using the same host as markets would mean the chain can be used to
    discover addresses, putting less empassis on criticality of
    gossipsub considering its set-up cost and lack of message delivery
    guarantees.

 2. Only a single instance of graphsync/datatransfer can be instantiated
    per libp2p host; therefore, if the host is shared, so should
    datatransfer manager.

 3. it is not clear if the assumptions under which separation was
    decided still hold.
 2022-02-16 13:46:44 +00:00
Łukasz Magiera
681b90768a Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse 2022-02-16 10:06:36 +01:00
vyzo
9c00af1b86 don't track peer ids in rcmgr metrics 2022-02-15 16:12:17 +02:00
Łukasz Magiera
48dea93c7e
Merge pull request #8089 from filecoin-project/feat/rcmgr-metrics 
Resource Manager Metrics
 2022-02-15 14:40:44 +01:00
Aayush
75b8e4c981 fix: sealer: allow enable/disabling ReplicaUpdate tasks 2022-02-14 15:25:52 -05:00
Aayush
56df886b55 fix: sealer: allow enable/disabling ReplicaUpdate tasks 2022-02-14 15:24:13 -05:00
Łukasz Magiera
60b9acc09d gen 2022-02-14 21:16:41 +01:00
Aayush
eebe784191 fix: sealer: allow enable/disabling ReplicaUpdate tasks 2022-02-14 15:08:28 -05:00
Łukasz Magiera
36a1934845 paychmgr: Fix tests after api changes 2022-02-14 20:16:30 +01:00
Łukasz Magiera
df5bd147fa
paychmgr: Review comments 
Co-authored-by: Hannah Howard <hannah@hannahhoward.net>
Co-authored-by: dirkmc <dirkmdev@gmail.com>
 2022-02-14 20:07:13 +01:00
Łukasz Magiera
f61eb23f8f api: separate method for paych funding 2022-02-14 19:56:02 +01:00
Łukasz Magiera
10af768c60 Merge commit 'origin/release/v1.15.0~2' into feat/paych-avail-reuse 2022-02-14 19:27:12 +01:00
vyzo
a9ec408844 collect resource manager metrics 2022-02-14 19:46:05 +02:00
Masih H. Derkani
7e7844f169
Fix log message formatting when instantiating index provider 2022-02-11 12:15:21 +00:00
Will
296eab3045
Merge pull request #8045 from gammazero/feat/cid-to-piece-idx 
Add indexer pubsub message authentication and rate limiting
 2022-02-10 13:49:58 -08:00
gammazero
eedcf910eb change scoring based on review 2022-02-10 10:53:07 -08:00
gammazero
b72cd90c19 Add indexer message scoring 2022-02-10 09:21:21 -08:00
gammazero
3f3d61b043 Service creation takes interface, not implementation 2022-02-10 08:41:18 -08:00
Jennifer Wang
de1f4792cd Merge remote-tracking branch 'origin/master' into jen/v15 2022-02-09 19:27:34 -05:00
Łukasz Magiera
44fd0e3349
Merge pull request #8008 from filecoin-project/feat/splitstore-sortless-compaction 
splitstore sortless compaction
 2022-02-09 17:17:57 +00:00
Jennifer Wang
6ed97f0d04 Merge branch 'release/v1.14.0' into jen/v15 2022-02-09 03:17:38 -05:00
zenground0
e32704e7f6 Update node doc gen 2022-02-08 12:56:32 -05:00
zenground0
1e524b7210 Review Response 2022-02-08 12:39:35 -05:00
zenground0
5a09b6496d Actually read config value 2022-02-08 12:39:35 -05:00
zenground0
2aa5911fb9 config gen 2022-02-08 12:39:35 -05:00
zenground0
4d8b912acc New config for turning off auto deal creation 2022-02-08 12:39:35 -05:00
Masih H. Derkani
cc61650f86 Upgrade to latest index-provider and set miner ID as extra gossip data 
Upgrade to the latest `index-provider` which upgrades the go-legs
protocol to allow the inclusion of extra gossip data that may be used
for gossip validation purposes. In the case of lotus gossip message
validators the miner ID is used to verify the sender's peer ID on chain.

Relates to:
- https://github.com/filecoin-project/lotus/pull/8045
 2022-02-08 14:32:03 +00:00
gammazero
b2805823ce Pass to validator the interfaces needed to get miner info 2022-02-08 04:55:59 -08:00
gammazero
1dc6a2fea6 Add indexer pubsub message authentication and rate limiting 2022-02-08 02:53:25 -08:00
Aarsh Shah
399a1afd94 Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx 2022-02-08 12:37:57 +04:00
vyzo
1221c0ba3f make gen 2022-02-06 12:28:21 +02:00
vyzo
03352ea863 make badger the default splitstore markset type 2022-02-06 11:21:16 +02:00
Aarsh Shah
e155eb9fa3
Merge pull request #8016 from filecoin-project/nonsense/add-fullnode-api 
add indexProvider host to markets
 2022-02-04 12:56:36 +04:00
Aarsh Shah
4691b2b809 changes to the indexer message relay PR 2022-02-04 12:15:01 +04:00
Aarsh Shah
3ecf478ff0
Merge pull request #8026 from gammazero/feat/cid-to-piece-idx 
Lotus chain nodes relay indexer pubsub messages
 2022-02-04 12:05:46 +04:00
gammazero
c084130d3e Lotus chain nodes relay indexer pubsub messages 
Content providers announce the availability of indexer data using gossip pubsub.  The content providers are not connected directly to indexers, so the pubsub messages are relayed to indexers via chain nodes. This PR makes chain nodes relay gossip pubsub messages, on the /indexer/ingest/<netname> topic.
 2022-02-03 14:56:21 -08:00
Will Scott
7247f8e41f
add net protect api methods 
fix #8015
 2022-02-03 17:24:49 +01:00
Anton Evangelatov
dde2ab59ae fixup 2022-02-03 15:53:46 +01:00
Anton Evangelatov
595ad44ee7 refactor indexprovider libp2p host connection to fullnode with meshcreator 2022-02-03 15:44:18 +01:00
Masih H. Derkani
575e6add43
Rename IndexerProvider to IndexProvider and add more docs 
Rename the config section corresponding to indexing to `IndexProvider`
for better readability.

Update existing docs for better clarity and add docs for config
parameters embedded from `index-provider` `Ingest` config library.
 2022-02-03 13:10:47 +00:00
Masih H. Derkani
235377585d
Clarify the purpose of MaxSimultaneousTransfers config 
Clarify the documentation of `MaxSimultaneousTransfers` to state that
the "transfer" refers to multihash syncing only and not the data itself.
 2022-02-03 11:51:01 +00:00
Anton Evangelatov
faa3a79a5f use NetAddrListener iface 2022-02-02 16:01:19 +01:00
Anton Evangelatov
2790cf35af change fullnodeApi to listen addrs 2022-02-02 13:06:53 +01:00
Anton Evangelatov
f9b63c5f8c add idxProvHost to StorageProvider 2022-02-02 13:03:41 +01:00
Anton Evangelatov
db5e4da268 add fullnodeApi to StorageProvider 2022-02-02 13:03:41 +01:00
Darko Brdareski
e51ce5c508 Merge remote-tracking branch 'upstream/master' into bloxico/system-test-matrix 2022-01-27 10:57:56 +01:00
Darko Brdareski
8c41e17c93 Fix typo in client_test annotations 2022-01-27 10:56:33 +01:00
Masih H. Derkani
5d72ebe0cc
Upgrade index-provider and update topic name 
Update the topic name to be consistent with default
configured in storetheindex.

Upgrade to the latest `index-provider`.
 2022-01-27 09:48:10 +00:00
Aayush
1b18236f91 feat: sealer: allow users to abort in-flight snap upgrades 2022-01-25 13:01:51 -05:00
Łukasz Magiera
4235a97cf4 retrieval: OffChainRetrieval config 2022-01-20 18:19:27 +01:00
Łukasz Magiera
8b19b84140 paych: option to force off-chain get 2022-01-20 18:19:26 +01:00
Łukasz Magiera
8e46b9ea5d paych: API to pre-fund channels 2022-01-20 18:15:46 +01:00
Aarsh Shah
b8bf2af3e3 upgrade on master 2022-01-20 16:34:13 +04:00
Aarsh Shah
9ee940098d Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx 2022-01-20 16:21:48 +04:00
vyzo
73ec10a49e close the rcmgr on shutdown 2022-01-20 11:47:39 +02:00
vyzo
325a4c0e5b fix lint 2022-01-20 11:46:39 +02:00
vyzo
554848fe6b add resource manager Net(Set)Limit api 2022-01-20 11:36:11 +02:00
vyzo
d05d5bcb70 add resource manager NetStat api 2022-01-20 11:36:11 +02:00
vyzo
8d3f98fe38 instantiate resource manager in DI 2022-01-20 11:36:11 +02:00
vyzo
826cdb2186 use the libp2p connmgr 2022-01-20 11:36:11 +02:00
hannahhoward
92d56d82f4 feat(deps): update markets stack 
update go-fil-markets, go-data-transfer 1.13.0, go-graphsync 0.12.0
 2022-01-14 17:21:04 -08:00
vyzo
7e43bfc109 hook the upgrade schedule to splitstore start 2022-01-14 17:14:32 -05:00
zenground0
d6aa17e21f Snap Deals Integration 
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
 2022-01-14 17:14:32 -05:00
c r
da6752eccb
feat: #7747 sealing: Adding conf variable for capping number of concurrent unsealing jobs (#7884) 
* adding the new variables- now time for logic

* putting parameters into right placeS

* adding unsealing throttle

* fixing linter issues

* removing one last thing...
 2022-01-13 12:26:13 -06:00
Aarsh Shah
8459d75597 update index provider config 2022-01-13 16:15:43 +04:00
Aarsh Shah
b780a64ed9 send markets multiaddrs to the engine 2022-01-13 15:31:15 +04:00
Aarsh Shah
8c8652e7bb add a new cli 2022-01-12 18:06:48 +04:00
Aarsh Shah
dec79c70a1 fix compilation 2022-01-12 15:03:46 +04:00
Aarsh Shah
21a8f71859 compiles 2022-01-12 14:59:09 +04:00
Aarsh Shah
edfc2c215c Merge remote-tracking branch 'origin/main' into feat/cid-to-piece-idx 2022-01-12 12:48:54 +04:00
zenground0
33f2d24f54 Snap Deals Integration 
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
 2022-01-10 15:39:38 +05:30
hannahhoward
1078dff3a9 feat(storageminer): also list data transfers with no id 2021-12-22 13:41:56 -08:00
hannahhoward
8a923fb420 fix(storageminer): fix conditional 2021-12-22 13:41:56 -08:00
hannahhoward
177282d9f9 fix(storageminer): lint/mod-tidy/extra checks 2021-12-22 13:41:55 -08:00
hannahhoward
cddf63efe9 feat(storageminer): add api for transfer diagnostics 
Add API + CLI for inspecting in depth diagnostics on graphsync transfers with a given peer
 2021-12-22 13:41:29 -08:00
Aayush Rajasekaran
1bc987772c
Merge pull request #7818 from filecoin-project/asr/vm-nvfactor 
Refactor: VM: Remove the NetworkVersionGetter
 2021-12-21 12:19:53 -05:00
Darko Brdareski
a54ac2ef79 Remove FS, MEM, GAS leftover test anotations. 2021-12-20 17:00:01 +01:00
Darko Brdareski
86cf17c1f4 Remove bad annotation from gas_test.go 2021-12-20 16:51:09 +01:00
Darko Brdareski
dda1a42a2a Merge branch 'bloxico/system-test-matrix' of https://github.com/filecoin-project/lotus into merge_lotus 2021-12-20 15:48:16 +01:00
Aayush Rajasekaran
3f4eaa99d5 Refactor: State: Rename stmgr::GetNtwkVersion to GetNetworkVersion 2021-12-17 18:43:39 -05:00
Dirk McCormick
b1734f84b3 feat: retrieval ask CLI command 2021-12-17 15:55:12 +01:00
Łukasz Magiera
a4728d3c72
Merge pull request #7578 from filecoin-project/rvagg/SimultaneousTransfersForStoragePerClient 
feat(graphsync): allow setting of per-peer incoming requests for miners
 2021-12-17 14:27:15 +01:00
vyzo
dd327f0b22 plumb more contexts 2021-12-17 11:42:09 +02:00
Rod Vagg
9e7d9affbe
feat(graphsync): allow setting of per-peer incoming requests for miners 2021-12-17 15:04:16 +11:00
Darko Brdareski
3e32aa896c Annotate client storage deals feature 2021-12-16 14:36:02 +01:00
vyzo
22ce395130 fix some unit tests 2021-12-14 17:05:10 +02:00
vyzo
716b4a3b09 hook a great context in the sky to satisfy DI; sigh... 2021-12-14 16:57:09 +02:00
vyzo
eb48dc9b68 fix issues with new peerstore constructor signature in DI 2021-12-14 16:21:55 +02:00
vyzo
f157ac9b68 update for new ConnManager constructor 2021-12-14 15:30:29 +02:00
Dirk McCormick
a51b08e386 feat: add IndexerProvider AnnounceAddress config 2021-12-13 18:24:05 +01:00
Masih H. Derkani
6b7b9793ab
Set default index provider ingest config values 
Set default values for index-provider ingest config.

Leave a TODO to use provider config APIs once defaults exposed.

Run `go mod tidy`
 2021-12-13 14:44:56 +00:00
Darko Brdareski
0addca1070 Fix bad annotations 2021-12-13 13:41:04 +01:00
vyzo
84710cf27e plumb more contexts in lotus 2021-12-13 14:26:59 +02:00
Aayush Rajasekaran
dfb65ed89f Plumb contexts through 2021-12-11 17:04:00 -05:00
vyzo
5d6398f20e hook the upgrade schedule to splitstore start 2021-12-03 12:11:54 +02:00
Łukasz Magiera
f39942283b
Merge pull request #7706 from filecoin-project/feat/ret-matchding-cars 
retrieval: Only output matching nodes, MatchPath dagspec
 2021-11-29 22:14:25 +01:00
Łukasz Magiera
d019853687
review: Cleanup some comments 
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
 2021-11-29 21:29:00 +01:00
Łukasz Magiera
5b5e6b9e44 retrieval: DagSpec.MatchPath -> ExportMerkleProof 2021-11-29 21:14:00 +01:00