Commit Graph

105 Commits

Author SHA1 Message Date
gammazero
1dc6a2fea6 Add indexer pubsub message authentication and rate limiting 2022-02-08 02:53:25 -08:00
gammazero
c084130d3e Lotus chain nodes relay indexer pubsub messages
Content providers announce the availability of indexer data using gossip pubsub.  The content providers are not connected directly to indexers, so the pubsub messages are relayed to indexers via chain nodes. This PR makes chain nodes relay gossip pubsub messages, on the /indexer/ingest/<netname> topic.
2022-02-03 14:56:21 -08:00
Łukasz Magiera
95b128b7bc chain: Cleanup consensus logic 2021-09-02 18:09:37 +02:00
Aayush Rajasekaran
bdd1c36802 Incoming: improve a log message 2021-08-25 12:26:10 -04:00
Łukasz Magiera
0514f38dc4 Merge remote-tracking branch 'origin/master' into chore/merge-1.10 2021-06-14 13:27:53 +02:00
Łukasz Magiera
534badad2a mpool: Add more metrics 2021-06-11 13:19:26 +02:00
wangchao
626d482990 correct the change of message size limit 2021-06-09 18:05:16 -04:00
wangchao
4c87818de1 correct the change of message size limit 2021-06-09 15:19:35 +08:00
Łukasz Magiera
a676da8a7e Merge remote-tracking branch 'origin/master' into feat/nv13-1.11 2021-06-01 21:07:04 +02:00
Mimir
ee508120d9
Typo fix in error message: "pubusb" -> "pubsub" 2021-05-31 15:30:10 -07:00
Aayush Rajasekaran
1f03a618f9 Plumb contexts through 2021-05-31 18:13:23 -04:00
Łukasz Magiera
d53c700d18
Merge pull request #5704 from filecoin-project/asr/log-large-delay
Log block CID in the large delay warning
2021-03-05 13:19:03 +01:00
Aayush Rajasekaran
374bd9b578
Update chain/sub/incoming.go
Co-authored-by: raulk <raul@protocol.ai>
2021-03-05 01:09:07 -05:00
Aayush Rajasekaran
22e465ba9c log block cid in the large delay warning 2021-03-02 16:46:10 -05:00
Raúl Kripalani
b0cbc932bd consolidate all blockstores in blockstore package. 2021-01-29 20:01:00 +00:00
Raúl Kripalani
0570306bb6 add metrics for delayed blocks. 2020-12-10 14:52:05 +00:00
zgfzgf
de9cef62f7 replace code with func 2020-11-19 20:46:40 +08:00
vyzo
8191654089 track oversize block errors in metrics 2020-11-17 17:18:49 -05:00
vyzo
e23170d194 add sanitcy check for maximum block size 2020-11-17 17:18:49 -05:00
Łukasz Magiera
9270ac6358 Make some logs quieter 2020-11-03 13:28:41 +01:00
Steven Allen
06af2fb52b Remove the worker key cache and use the correct worker key
We need to use the worker key from the lookback tipset state. We can't
reasonably maintain this cache, unfortunately (unless we kept some form of
"per-height" cache).
2020-10-23 19:40:05 -07:00
Steven Allen
8bb69f1e9c Use pre-computed parent state root for lookback info
There's no need to call TipSetState and recompute it.
2020-10-23 19:40:01 -07:00
Steven Allen
4e730b5ec8 port to v2 imports 2020-10-21 12:16:23 -07:00
Steven Allen
e2fbbdcb15 reject messages with invalid CIDs
These can't possibly be valid messages, so we should just drop the block.
2020-10-14 14:43:58 -07:00
Steven Allen
105aa40007 simplify message syncing logic
1. Allow duplicate blocks from bitswap. This shouldn't happen, but there's no
reason to bail (just log loudly).
2. Simplify logic to very explicitly check to make sure we're fetching every
block.
2020-10-09 11:06:49 -07:00
Jakub Sztandera
973f61bc10
Optimize chain and message sync
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-10-08 23:10:07 +02:00
Aayush Rajasekaran
4a9155ba85 Use new MinerEligibleToMine method post-v2 actors 2020-10-06 04:43:55 -04:00
Steven Allen
b6500beaab Merge branch 'master' into asr/spec-v1 2020-10-05 10:29:09 -07:00
Steven Allen
53cc1f18d0 use bitswap sessions when fetching messages, and cancel them
1. Explicitly use a session when fetching messages for a block. Technically,
GetBlocks would use an internal session so we'd only end up with two, but one
per block is even better.
2. Actually cancel sessions after a timeout (threading through the context).

NOTE: We should seriously consider having a single session for all blocks, but
we can do that in a followup as it may have unintended consequences (e.g., leaks).
2020-10-02 10:49:35 -07:00
Steven Allen
233d8a9b72 update even more imports 2020-09-28 14:25:58 -07:00
Aayush Rajasekaran
31ff5230bb Get State API almost working 2020-09-17 05:05:32 -04:00
Steven Allen
cc4d5306eb Progress 2020-09-14 15:43:12 -07:00
Steven Allen
d3594835c4 [WIP] Network upgrade support
This patch starts adding support for network upgrades.

* It adds an actors abstraction layer for loading abstract (cross-version) actors.
* It starts switching over to a shared deadline type.
* It adds an abstraction for ADTs (hamt/amt).
* It removes the callback-based API in the StateManager (difficult to abstract
across actor versions).
* It _does not_ actually add support for actors v2. We can do that in a followup
patch but that should be relatively easy.

This patch is heavily WIP and does not compile. Feel free to push changes
directly to this branch.

Notes:

* State tree access now needs a network version, because the HAMT type will change.
* I haven't figured out a nice way to abstract over changes to the _message_
types. However, many of them will be type aliased to actors v0 in actors v2 so
we can likely continue using the v0 versions (or use the v2 versions
everywhere). I've been renaming imports to `v0*` to make it clear that we're
importing types from a _specific_ actors version.

TODO:

* Consider merging incremental improvements? We'd have to get this compiling
again first but we could merge in the new abstractions, and slowly switch over.
* Finish migrating to the new abstractions.
* Remove all actor state types from the public API. See `miner.State.Info()` for
the planned approach here.
* Fix the tests. This is likely going to be a massive pain.
2020-09-11 20:16:29 -07:00
vyzo
37ec1a978e nicer check using Time objects 2020-09-08 10:47:41 +03:00
vyzo
27fa9969f3 fix isChainNearSync check in block validator 2020-09-08 10:26:27 +03:00
vyzo
d76a3b87c5 ignore messages with large nonce gaps 2020-09-03 15:28:44 +03:00
Łukasz Magiera
84a632787c
Merge pull request #3313 from filecoin-project/feat/mpool-balance-tracking
Message Pool Rudimentary Spam Protection Measures
2020-08-28 22:35:30 +02:00
lanzafame
64768f093d rename local recordFailure func 2020-08-28 17:15:11 +10:00
lanzafame
a24db6c584 refactor block validate recordFailure func 2020-08-28 17:11:59 +10:00
lanzafame
050a0ec5e4 further reduce code duplication 2020-08-28 17:01:45 +10:00
lanzafame
0254e4b365 reduce code duplication 2020-08-28 16:53:59 +10:00
lanzafame
094baf8aa9 distinguish local message validation failures from remote 2020-08-28 16:11:24 +10:00
vyzo
6c3d8ec1de ignore, don't reject block if we can't determine power 2020-08-27 16:53:53 +03:00
vyzo
82ef0529d8 named error for too many pending messages 2020-08-26 22:48:03 +03:00
vyzo
c473d3c682 rename ErrValidationFailure to ErrSoftValidationFailure
to make it clear to readers that this failure is soft; we might be out of sync.
2020-08-26 15:16:04 +03:00
vyzo
78876947a4 ErrBroadcastAnyway is now ErrValidationFailure 2020-08-26 15:03:13 +03:00
Steven Allen
5733c71c50 Lint everything
We were ignoring quite a few error cases, and had one case where we weren't
actually updating state where we wanted to. Unfortunately, if the linter doesn't
pass, nobody has any reason to actually check lint failures in CI.

There are three remaining XXXs marked in the code for lint.
2020-08-20 20:46:36 -07:00
vyzo
c7a6e32c96 ignore, don't reject messages with nonce that is too low
we seem to have a problem with this leading to many rejected messages and negative scores for
potentially innocent peers.
2020-08-19 13:20:16 +03:00
vyzo
ca552cf83b deduplicate some block validation logic 2020-08-17 21:03:08 +03:00
vyzo
ae88a99c84 add some lightweight validation of local messages 2020-08-17 09:04:22 +03:00