cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
19,775 Commits 24 Branches 342 Tags 314 MiB
c4214e23bf
Commit Graph

158 Commits

Author SHA1 Message Date
gammazero
b2805823ce Pass to validator the interfaces needed to get miner info 2022-02-08 04:55:59 -08:00
gammazero
1dc6a2fea6 Add indexer pubsub message authentication and rate limiting 2022-02-08 02:53:25 -08:00
Aarsh Shah
399a1afd94 Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx 2022-02-08 12:37:57 +04:00
gammazero
c084130d3e Lotus chain nodes relay indexer pubsub messages 
Content providers announce the availability of indexer data using gossip pubsub.  The content providers are not connected directly to indexers, so the pubsub messages are relayed to indexers via chain nodes. This PR makes chain nodes relay gossip pubsub messages, on the /indexer/ingest/<netname> topic.
 2022-02-03 14:56:21 -08:00
Darko Brdareski
a64f2421d2 Annotate 'incoming' subsystem 2021-12-14 11:33:33 +01:00
Łukasz Magiera
95b128b7bc chain: Cleanup consensus logic 2021-09-02 18:09:37 +02:00
Aayush Rajasekaran
bdd1c36802 Incoming: improve a log message 2021-08-25 12:26:10 -04:00
Łukasz Magiera
0514f38dc4 Merge remote-tracking branch 'origin/master' into chore/merge-1.10 2021-06-14 13:27:53 +02:00
Łukasz Magiera
534badad2a mpool: Add more metrics 2021-06-11 13:19:26 +02:00
wangchao
626d482990 correct the change of message size limit 2021-06-09 18:05:16 -04:00
wangchao
4c87818de1 correct the change of message size limit 2021-06-09 15:19:35 +08:00
Łukasz Magiera
a676da8a7e Merge remote-tracking branch 'origin/master' into feat/nv13-1.11 2021-06-01 21:07:04 +02:00
Mimir
ee508120d9
Typo fix in error message: "pubusb" -> "pubsub" 2021-05-31 15:30:10 -07:00
Aayush Rajasekaran
1f03a618f9 Plumb contexts through 2021-05-31 18:13:23 -04:00
Łukasz Magiera
d53c700d18
Merge pull request #5704 from filecoin-project/asr/log-large-delay 
Log block CID in the large delay warning
 2021-03-05 13:19:03 +01:00
Aayush Rajasekaran
374bd9b578
Update chain/sub/incoming.go 
Co-authored-by: raulk <raul@protocol.ai>
 2021-03-05 01:09:07 -05:00
Aayush Rajasekaran
22e465ba9c log block cid in the large delay warning 2021-03-02 16:46:10 -05:00
Raúl Kripalani
b0cbc932bd consolidate all blockstores in blockstore package. 2021-01-29 20:01:00 +00:00
Raúl Kripalani
0570306bb6 add metrics for delayed blocks. 2020-12-10 14:52:05 +00:00
zgfzgf
de9cef62f7 replace code with func 2020-11-19 20:46:40 +08:00
vyzo
8191654089 track oversize block errors in metrics 2020-11-17 17:18:49 -05:00
vyzo
e23170d194 add sanitcy check for maximum block size 2020-11-17 17:18:49 -05:00
Łukasz Magiera
9270ac6358 Make some logs quieter 2020-11-03 13:28:41 +01:00
Steven Allen
06af2fb52b Remove the worker key cache and use the correct worker key 
We need to use the worker key from the lookback tipset state. We can't
reasonably maintain this cache, unfortunately (unless we kept some form of
"per-height" cache).
 2020-10-23 19:40:05 -07:00
Steven Allen
8bb69f1e9c Use pre-computed parent state root for lookback info 
There's no need to call TipSetState and recompute it.
 2020-10-23 19:40:01 -07:00
Steven Allen
4e730b5ec8 port to v2 imports 2020-10-21 12:16:23 -07:00
Steven Allen
e2fbbdcb15 reject messages with invalid CIDs 
These can't possibly be valid messages, so we should just drop the block.
 2020-10-14 14:43:58 -07:00
Steven Allen
105aa40007 simplify message syncing logic 
1. Allow duplicate blocks from bitswap. This shouldn't happen, but there's no
reason to bail (just log loudly).
2. Simplify logic to very explicitly check to make sure we're fetching every
block.
 2020-10-09 11:06:49 -07:00
Jakub Sztandera
973f61bc10
Optimize chain and message sync 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-10-08 23:10:07 +02:00
Aayush Rajasekaran
4a9155ba85 Use new MinerEligibleToMine method post-v2 actors 2020-10-06 04:43:55 -04:00
Steven Allen
b6500beaab Merge branch 'master' into asr/spec-v1 2020-10-05 10:29:09 -07:00
Steven Allen
53cc1f18d0 use bitswap sessions when fetching messages, and cancel them 
1. Explicitly use a session when fetching messages for a block. Technically,
GetBlocks would use an internal session so we'd only end up with two, but one
per block is even better.
2. Actually cancel sessions after a timeout (threading through the context).

NOTE: We should seriously consider having a single session for all blocks, but
we can do that in a followup as it may have unintended consequences (e.g., leaks).
 2020-10-02 10:49:35 -07:00
Steven Allen
233d8a9b72 update even more imports 2020-09-28 14:25:58 -07:00
Aayush Rajasekaran
31ff5230bb Get State API almost working 2020-09-17 05:05:32 -04:00
Steven Allen
cc4d5306eb Progress 2020-09-14 15:43:12 -07:00
Steven Allen
d3594835c4 [WIP] Network upgrade support 
This patch starts adding support for network upgrades.

* It adds an actors abstraction layer for loading abstract (cross-version) actors.
* It starts switching over to a shared deadline type.
* It adds an abstraction for ADTs (hamt/amt).
* It removes the callback-based API in the StateManager (difficult to abstract
across actor versions).
* It _does not_ actually add support for actors v2. We can do that in a followup
patch but that should be relatively easy.

This patch is heavily WIP and does not compile. Feel free to push changes
directly to this branch.

Notes:

* State tree access now needs a network version, because the HAMT type will change.
* I haven't figured out a nice way to abstract over changes to the _message_
types. However, many of them will be type aliased to actors v0 in actors v2 so
we can likely continue using the v0 versions (or use the v2 versions
everywhere). I've been renaming imports to `v0*` to make it clear that we're
importing types from a _specific_ actors version.

TODO:

* Consider merging incremental improvements? We'd have to get this compiling
again first but we could merge in the new abstractions, and slowly switch over.
* Finish migrating to the new abstractions.
* Remove all actor state types from the public API. See `miner.State.Info()` for
the planned approach here.
* Fix the tests. This is likely going to be a massive pain.
 2020-09-11 20:16:29 -07:00
vyzo
37ec1a978e nicer check using Time objects 2020-09-08 10:47:41 +03:00
vyzo
27fa9969f3 fix isChainNearSync check in block validator 2020-09-08 10:26:27 +03:00
vyzo
d76a3b87c5 ignore messages with large nonce gaps 2020-09-03 15:28:44 +03:00
Łukasz Magiera
84a632787c
Merge pull request #3313 from filecoin-project/feat/mpool-balance-tracking 
Message Pool Rudimentary Spam Protection Measures
 2020-08-28 22:35:30 +02:00
lanzafame
64768f093d rename local recordFailure func 2020-08-28 17:15:11 +10:00
lanzafame
a24db6c584 refactor block validate recordFailure func 2020-08-28 17:11:59 +10:00
lanzafame
050a0ec5e4 further reduce code duplication 2020-08-28 17:01:45 +10:00
lanzafame
0254e4b365 reduce code duplication 2020-08-28 16:53:59 +10:00
lanzafame
094baf8aa9 distinguish local message validation failures from remote 2020-08-28 16:11:24 +10:00
vyzo
6c3d8ec1de ignore, don't reject block if we can't determine power 2020-08-27 16:53:53 +03:00
vyzo
82ef0529d8 named error for too many pending messages 2020-08-26 22:48:03 +03:00
vyzo
c473d3c682 rename ErrValidationFailure to ErrSoftValidationFailure 
to make it clear to readers that this failure is soft; we might be out of sync.
 2020-08-26 15:16:04 +03:00
vyzo
78876947a4 ErrBroadcastAnyway is now ErrValidationFailure 2020-08-26 15:03:13 +03:00
Steven Allen
5733c71c50 Lint everything 
We were ignoring quite a few error cases, and had one case where we weren't
actually updating state where we wanted to. Unfortunately, if the linter doesn't
pass, nobody has any reason to actually check lint failures in CI.

There are three remaining XXXs marked in the code for lint.
 2020-08-20 20:46:36 -07:00
vyzo
c7a6e32c96 ignore, don't reject messages with nonce that is too low 
we seem to have a problem with this leading to many rejected messages and negative scores for
potentially innocent peers.
 2020-08-19 13:20:16 +03:00
vyzo
ca552cf83b deduplicate some block validation logic 2020-08-17 21:03:08 +03:00
vyzo
ae88a99c84 add some lightweight validation of local messages 2020-08-17 09:04:22 +03:00
vyzo
884d4ad9df fix clown shoes pubsub validation: we always accept our own self-published messages 2020-08-16 20:46:19 +03:00
Jakub Sztandera
3710f33e14
Don't reject messages that don't pass RBF, ignore them 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-08-09 03:17:40 +02:00
Łukasz Magiera
051be61452 gofmt 2020-08-07 15:04:10 +02:00
Lucas Molas
76d40ec408 first pass, ready for review 2020-08-03 12:20:04 -03:00
Lucas Molas
595de3fccb fix: block validator: check against heaviest tipset 2020-07-30 18:05:28 -03:00
Aayush Rajasekaran
b51e2a868b
Merge branch 'next' into feat/cid-builder 2020-07-28 21:02:30 -04:00
Aayush Rajasekaran
859168015a Use specs actor's MinerNominalPowerMeetsConsensusMinimum 2020-07-28 17:21:05 -04:00
Steven Allen
b7a4dbb07f Support inline CIDs 
And use the new CidBuilder from the spec actors.

This patch does not switch over to inline CIDs by default, but paves the way.
 2020-07-23 23:12:32 -07:00
Steven Allen
5fc83f4d05 Refactor to use actor adt types instead of directly using HAMTs and AMTs 
This way there's a single source of truth. Preparation for fixing
https://github.com/filecoin-project/specs-actors/issues/517 (requires changing
HAMT parameters).
 2020-07-23 09:34:57 -07:00
Raúl Kripalani
13de81b3b2 introduce the ability to mock time. 2020-07-10 15:51:45 +01:00
Łukasz Magiera
4c422c2d50 specs-actors v0.7.1 2020-07-01 13:47:40 +02:00
Jakub Sztandera
156a14eeeb
Integrate Poisson Sortition into chain sync 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-06-23 16:59:44 +02:00
Jeromy
512270593b clean up some of the more spammy logs 2020-06-11 12:31:29 +02:00
Jakub Sztandera
5605aae269
Fix even more lint warnings 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-06-02 16:29:39 +02:00
vyzo
cc98117df2 log block validation time 2020-05-22 17:21:37 +03:00
vyzo
50e05ae1ba use sigs.CheckBlockSignature 2020-05-14 20:48:40 +03:00
vyzo
432c69c616 update message validator to provide rejection signal 2020-05-14 20:48:40 +03:00
vyzo
e714294273 initialize map 2020-05-14 20:48:19 +03:00
vyzo
c3a144bcb6 flesh out signature validation logic 2020-05-14 20:48:19 +03:00
vyzo
dd86a289ce cache miner worker keys 2020-05-14 20:48:19 +03:00
vyzo
6e0dac06f4 utility to retrieve miner worker key from the chain 2020-05-14 20:48:19 +03:00
vyzo
5bd0e92068 remove source flagging from bogus CID blocks 
it's ineffective.
 2020-05-14 20:48:19 +03:00
vyzo
eb24baa70c use go-libp2p-core/peer instead of the deprecated alias, try to pacify the silly linter 2020-05-05 17:26:59 +03:00
vyzo
4e5179557d flag block source when messages cannot be retrieved 2020-05-05 16:49:43 +03:00
vyzo
5824714cdc use extended validator interface 2020-05-05 16:35:03 +03:00
Jeromy
417d434973 several fixes and improvements while debugging interop 2020-03-31 18:35:23 -07:00
Aayush Rajasekaran
6fda3c877e Re: #1302: Refine invalid message filtering 
- This commit slightly weakens the current invalid message check
- The behaviour is that if you can't add a message to your pool, you *probably* won't broadcast it to your peers
- The exceptions are that you will broadcast a message if you fail to validate it because nonce / balance lookup fails
- This commit also lowers the invalid message log to debug (to lessen the annoyance of several invalid messages coming in, and hopefully to prevent confusion among node operators)
 2020-03-18 03:14:18 -04:00
Adrian Lanzafame
d0c748460e remove high cardinality tags from metrics 2020-03-05 19:47:20 +10:00
Nate Walck
8283bb994a Add block metrics to incoming pubsub validate funcs 2020-03-01 19:57:16 -05:00
Nate Walck
33af2409e8 Use head notif func for current node height, add pubsub metrics 2020-03-01 19:26:09 -05:00
whyrusleeping
00efd097c7 implement basic message filtering 2020-02-27 17:40:16 -08:00
whyrusleeping
1dafcf24f1 basic gossip spam protection logic 2020-02-16 21:51:18 -08:00
Jakub Sztandera
1ed62628a7
Update go-log to v2 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-01-08 20:22:31 +01:00
whyrusleeping
a109ae1490 introduce a limit on the number of messages a block can have 2020-01-07 12:41:26 -08:00
whyrusleeping
e5899f614a Tag peers who send us good blocks over pubsub 2019-12-19 12:47:55 -08:00
whyrusleeping
6cb6167393 bump up filecoin related weights 2019-12-17 10:16:10 -08:00
whyrusleeping
7dcc528cd1 tag peers who propagate good blocks to us 2019-12-16 23:06:48 -08:00
whyrusleeping
f141849df5 fix incoming tipset bucketing 2019-12-11 21:41:24 +01:00
whyrusleeping
d3f08c5243 remove more spammy logs 2019-12-07 23:32:34 +01:00
whyrusleeping
bc343dbb67 flag blocks that are received too late 2019-12-07 11:49:05 +01:00
whyrusleeping
b58e7344e8 pull messagepool into separate package 2019-12-02 14:33:11 -08:00
whyrusleeping
c9a5d974c1 drop message nonce message down to a warning 2019-11-14 19:26:13 -08:00
Łukasz Magiera
8f0e3ede24 Bunch of small fixes / improvements, gofmt 2019-11-13 15:48:57 +01:00
Łukasz Magiera
29778cd653 Mostly fix TestDealFlow 2019-11-07 01:18:06 +01:00
Jakub Sztandera
1bf713cb0a
Cleanup imports after rename 
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2019-10-18 13:47:41 +09:00
Jakub Sztandera
49e3fe39f8
Structure logs 
License: MIT
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2019-10-11 02:00:04 +02:00
whyrusleeping
85684ba7e2 don't add bad blocks to tipsetcache 2019-10-10 12:04:10 +09:00