Commit Graph

14619 Commits

Author SHA1 Message Date
Masih H. Derkani
230b8554f6 Override testnetnet to mainnet in indexer ingest topic
Override `testnetnet` to `mainnet` when constructing indexer ingestion
topic.
2022-02-11 19:52:19 +00:00
Masih H. Derkani
b47cde70fa Decode gossip extra data as []bytes
The type of extra data in go-legs gossip is bytes. But when it is parsed
as miner ID, it is cast to string then parsed. Instead, it should be
decoded from bytes.
2022-02-11 18:59:00 +00:00
Masih H. Derkani
7e7844f169
Fix log message formatting when instantiating index provider 2022-02-11 12:15:21 +00:00
Will
296eab3045
Merge pull request #8045 from gammazero/feat/cid-to-piece-idx
Add indexer pubsub message authentication and rate limiting
2022-02-10 13:49:58 -08:00
gammazero
c1b2080f4e spelling in comment 2022-02-10 11:25:30 -08:00
gammazero
eedcf910eb change scoring based on review 2022-02-10 10:53:07 -08:00
gammazero
b72cd90c19 Add indexer message scoring 2022-02-10 09:21:21 -08:00
gammazero
1e37185d74 Ignore, not regect, indexer messages from self. 2022-02-10 08:44:40 -08:00
gammazero
3f3d61b043 Service creation takes interface, not implementation 2022-02-10 08:41:18 -08:00
Masih H. Derkani
f0accce286 Upgrade index-provider and go-fil-markets to latest
See: https://github.com/filecoin-project/go-fil-markets/pull/680
2022-02-10 14:04:11 +00:00
gammazero
9481fa0a4b Use new indexer pubsub message encoding 2022-02-09 16:40:27 -08:00
gammazero
681ce94a34 Correctly handle seqno check 2022-02-09 16:21:05 -08:00
gammazero
3ff209d95d Add replay rejection 2022-02-09 11:06:56 -08:00
gammazero
a62e027002 review changes 2022-02-09 10:29:49 -08:00
Masih H. Derkani
cc61650f86 Upgrade to latest index-provider and set miner ID as extra gossip data
Upgrade to the latest `index-provider` which upgrades the go-legs
protocol to allow the inclusion of extra gossip data that may be used
for gossip validation purposes. In the case of lotus gossip message
validators the miner ID is used to verify the sender's peer ID on chain.

Relates to:
- https://github.com/filecoin-project/lotus/pull/8045
2022-02-08 14:32:03 +00:00
gammazero
b2805823ce Pass to validator the interfaces needed to get miner info 2022-02-08 04:55:59 -08:00
gammazero
1dc6a2fea6 Add indexer pubsub message authentication and rate limiting 2022-02-08 02:53:25 -08:00
Aarsh Shah
399a1afd94 Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx 2022-02-08 12:37:57 +04:00
Aarsh Shah
eb8296120b changes to mesh creation code 2022-02-04 13:58:47 +04:00
Aarsh Shah
daac7f2988 fix lint 2022-02-04 13:55:26 +04:00
Aarsh Shah
ca46a79144 protect full node connection 2022-02-04 13:04:07 +04:00
Aarsh Shah
e155eb9fa3
Merge pull request #8016 from filecoin-project/nonsense/add-fullnode-api
add indexProvider host to markets
2022-02-04 12:56:36 +04:00
Aarsh Shah
4691b2b809 changes to the indexer message relay PR 2022-02-04 12:15:01 +04:00
Aarsh Shah
3ecf478ff0
Merge pull request #8026 from gammazero/feat/cid-to-piece-idx
Lotus chain nodes relay indexer pubsub messages
2022-02-04 12:05:46 +04:00
gammazero
c084130d3e Lotus chain nodes relay indexer pubsub messages
Content providers announce the availability of indexer data using gossip pubsub.  The content providers are not connected directly to indexers, so the pubsub messages are relayed to indexers via chain nodes. This PR makes chain nodes relay gossip pubsub messages, on the /indexer/ingest/<netname> topic.
2022-02-03 14:56:21 -08:00
Jiaying Wang
f4af6b670c
Merge pull request #8021 from filecoin-project/willscott/net-protect-api
add net protect api methods
2022-02-03 17:54:33 -05:00
Łukasz Magiera
d9c763437e
Merge pull request #8022 from filecoin-project/deps/update-libp2p
update go-libp2p to v0.18.0-rc3
2022-02-03 18:42:04 +00:00
vyzo
e78c4ab9b3 update go-libp2p to v0.18.0-rc3 2022-02-03 19:32:30 +02:00
Will Scott
7247f8e41f
add net protect api methods
fix #8015
2022-02-03 17:24:49 +01:00
Anton Evangelatov
b6b5046cce upgrade go-fil-markets 2022-02-03 16:26:04 +01:00
Anton Evangelatov
dde2ab59ae fixup 2022-02-03 15:53:46 +01:00
Anton Evangelatov
bb6235733d go mod tidy 2022-02-03 15:48:05 +01:00
Anton Evangelatov
595ad44ee7 refactor indexprovider libp2p host connection to fullnode with meshcreator 2022-02-03 15:44:18 +01:00
Anton Evangelatov
20fc5ffa38 fixing tests 2022-02-03 14:43:20 +01:00
Masih H. Derkani
6ef0c382c3
Fix typos in default miner config 2022-02-03 13:24:46 +00:00
Masih H. Derkani
575e6add43
Rename IndexerProvider to IndexProvider and add more docs
Rename the config section corresponding to indexing to `IndexProvider`
for better readability.

Update existing docs for better clarity and add docs for config
parameters embedded from `index-provider` `Ingest` config library.
2022-02-03 13:10:47 +00:00
Masih H. Derkani
235377585d
Clarify the purpose of MaxSimultaneousTransfers config
Clarify the documentation of `MaxSimultaneousTransfers` to state that
the "transfer" refers to multihash syncing only and not the data itself.
2022-02-03 11:51:01 +00:00
Anton Evangelatov
c3d41d4e17 upgrade go-fil-markets 2022-02-02 16:14:44 +01:00
Anton Evangelatov
faa3a79a5f use NetAddrListener iface 2022-02-02 16:01:19 +01:00
Anton Evangelatov
03dd33dbbe upgrade go-fil-markets 2022-02-02 14:07:41 +01:00
Anton Evangelatov
79403f169e fix autogenerated docs 2022-02-02 13:18:53 +01:00
Anton Evangelatov
2790cf35af change fullnodeApi to listen addrs 2022-02-02 13:06:53 +01:00
Anton Evangelatov
f9b63c5f8c add idxProvHost to StorageProvider 2022-02-02 13:03:41 +01:00
Anton Evangelatov
db5e4da268 add fullnodeApi to StorageProvider 2022-02-02 13:03:41 +01:00
Masih H. Derkani
ad348c8e2f
Upgrade to the latest index provider
Upgrade to the latest index provider which rolls out a more compact ad
signature mechanism along with dependency updates.
2022-02-01 09:53:57 +00:00
Łukasz Magiera
51643caf60
Merge pull request #7835 from filecoin-project/bloxico/system-test-matrix
misc: api: Annotate lotus tests according to listed behaviors
2022-01-31 12:23:09 +01:00
vyzo
ff10e0eaf1
Merge pull request #7999 from filecoin-project/feat/splistore-refactors
some basic splitstore refactors
2022-01-28 13:55:16 +02:00
Darko Brdareski
6b8f526df3 Fix merge 2022-01-27 11:06:04 +01:00
Darko Brdareski
e51ce5c508 Merge remote-tracking branch 'upstream/master' into bloxico/system-test-matrix 2022-01-27 10:57:56 +01:00
Darko Brdareski
8c41e17c93 Fix typo in client_test annotations 2022-01-27 10:56:33 +01:00