Commit Graph

4238 Commits

Author SHA1 Message Date
Aayush
596b01fba2 feat: chainstore: exit early in MaybeTakeHeavierTipset 2023-05-05 13:56:48 -04:00
Mike Seiler
9c05a1f734 chain errors using xerrors.Errorf 2023-05-04 23:50:07 -10:00
Friðrik Ásmundsson
e6c8072f50
Merge pull request #10827 from filecoin-project/10585-address-review-comments
10585 address review comments
2023-05-04 18:32:04 +00:00
Fridrik Asmundsson
4e8ee737fc Address review comments 2023-05-04 17:10:26 +00:00
Aayush
b45409262f feat: deflake msgindex_test.go 2023-05-04 12:04:28 -04:00
Friðrik Ásmundsson
dec3af7d7f
Merge pull request #10585 from filecoin-project/10584-make-execution-cache-size-configurable
feat: Make execution trace configurable via env variable
2023-05-04 14:07:35 +00:00
Mikers
c1d22d8b32
refactor: streamline error handling in CheckPendingMessages (#10818) 2023-05-03 11:12:47 -10:00
Mikers
742062f84c
perf: mempool: lower priority optimizations (#10693)
* release the read lock earlier as it is not needed for chaincomputebasefee

* chain/messagepool/selection.go change to read lock in SelectMessages

* tighten up locks in chain/messagepool/repub.go and two questions on whether curTsLks are needed as comments

* include suggestion from @Jorropo to preallocate our msgs array so that we only need to make a single allocation

* mp.pending should not be accessed directly but through the getter

* from @arajasek: just check whether the sender is a robust address (anything except an ID address is robust) here, and return if so. That will:

be faster
reduce the size of this cache by half, because we can drop mp.keyCache.Add(ka, ka) on line 491.

* do not need curTslk and clean up code comments
2023-05-03 16:31:39 -04:00
Aayush
49f5825296 feat: chainstore: batch writes of tipsets 2023-05-02 17:26:07 -04:00
Aayush Rajasekaran
ff0fa3471f
chore: refactor: drop unused IsTicketWinner (#10801) 2023-05-02 11:22:02 -07:00
Steven Allen
bb5ba64cca Revert "Merge pull request #9858 from adlrocha/adlrocha/consistent-bcast"
This reverts commit 8b2208fd9a, reversing
changes made to 2db6b12b78.

Unfortunately, this is rather tricky code. We've found several issues so
far and, while we've fixed a few, there are outstanding issues that
would require complex fixes we don't have time to tackle right now.

Luckily, this code isn't actually needed by the main Filecoin chain
which relies on consensus fault reporting to handle equivocation. So we
can just try again later.
2023-04-27 12:25:30 -07:00
Steven Allen
d2c3e84d54
feat: sync: harden chain sync (#10756)
* fix: sync: fail sync instead of logging if we sync the wrong chain

* fix: sync: write headers in the correct order

Just in case. This shouldn't be necessary, but we might as well.

* fix: minus minus

* fix: do put the tipset

Put != Persist
2023-04-26 14:12:45 -07:00
Aayush Rajasekaran
eb0b1c986b
Merge pull request #10713 from filecoin-project/steb/fix-event-subscription
fix: events: don't set GC confidence to 1
2023-04-26 16:14:20 -04:00
Steven Allen
a97dae8f46 fix: sync: reduce log from error to info
And fix the message to account for the fact that we now reject _old_
blocks along with new ones.

We frequently receive "out of date" blocks in hello messages from
syncing and/or out of sync nodes. This isn't an error.
2023-04-25 14:34:13 -07:00
Steven Allen
7162c656cc
fix: chain: record heaviest tipset before notifying (#10694)
Clearly this hasn't caused any issues, but I'm pretty sure we should be
updating the current head _before_ notifying about it.
2023-04-25 11:56:26 -07:00
ychiao
9ae48022ff
fix: Eth JSON-RPC api: handle messages with gasFeeCap less than baseFee (#10614) 2023-04-24 16:03:04 +01:00
Aayush
5a061e4677 Merge branch 'releases' into asr/merge-release-into-master 2023-04-23 14:58:50 -04:00
Steven Allen
784214ae05
feat: sync: validate (early) that blocks fall within range (#10691)
This will reject blocks in pubsub validation if they're either:

1. Too far into the future (5 blocks beyond the expected head).
2. Too far into the past (before finality with respect to our current
   head).

Specifically:

1. We were previously rejecting future blocks in the sync logic, but not
   in pubsub itself.
2. We never used to check if a block was too _old_.

Motivation: Blocks that are too new/too old can cause us to perform
quite a bit of unnecessary work.
2023-04-22 10:15:31 -07:00
Hector Sanjuan
875c09840b
chainstore: Fix raw blocks getting scanned for links during snapshots (#10684)
We have to save raw blocks to the snapshot, but we should not be scanning them
for additional links as if they were CBOR blocks.

This cleans the logic a bit (we were checking that the parent was a CBOR block
before queueing up the children, but then scanning the children... it was weird).

Additionally, more verbose logging is added for the next time ScanForLinks
fails (currently very little info was given).

Our ScanForLinks callback should only enqueue CBOR for further processing.
2023-04-21 15:16:26 -07:00
Friðrik Ásmundsson
9d5d6a5f3d
Merge pull request #10700 from filecoin-project/10538-opt-ethgettransactioncount
perf: Address performance of EthGetTransactionCount
2023-04-21 20:13:56 +00:00
Fridrik Asmundsson
4028c05fea address review comment 2023-04-21 20:03:13 +00:00
Fridrik Asmundsson
553da395e4 perf: Increase noncecache in MessagePool
Bumped from 256 to 32k entries which should be about 6MB of cached
entries given average nonceCacheKey of 200 bytes
2023-04-21 11:56:05 +00:00
Fridrik Asmundsson
953d56e216 perf: Address performance of EthGetTransactionCount
We have observed that EthGetTransactionCount is one of the hotspots
on Glif production notes, and we are seeing regular 10-20 second
latencies when calling this rpc method.

I tracked the high latency spikes and they were correlated when
we were running ExecuteTipSet while following the chain.

To address this, we should not rely on tipset computation to get
nounce and instead look at the parent tipset and then count the
messages sent from the 'addr'.
2023-04-21 11:55:46 +00:00
Łukasz Magiera
a503a0edaa
Merge pull request #10690 from filecoin-project/fix/remove-pointless-panic
fix: remove pointless panic
2023-04-21 13:47:12 +02:00
Steven Allen
c84c07eb74 fix: events: don't set GC confidence to 1
The function/parameter were poorly named and should never have been
exposed. "GC" confidence should always be the same, this parameter
doesn't let us actually set the _confidence_, just the point before
which we no longer support reverts.

fixes #10706
2023-04-20 14:45:55 -07:00
Łukasz Magiera
0a46fc7e73
Merge pull request #10562 from Jorropo/boxo
chore: boxo: migrate from go-libipfs to boxo
2023-04-20 16:24:22 +02:00
ZenGround0
327760acff
chore: build: Merge/v22 into 21 for 23 (#10702)
* chore: update ffi to increase execution parallelism

* Don't enforce walking receipt tree during compaction

* fix: build: drop drand incentinet servers

* chore: release lotus v1.20.4

* Apply suggestions from code review

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

* feat: Introduce nv19 skeleton

Update to go-state-types v0.11.0-alpha-1

Introduce dummy v11 actor bundles

Make new actors adapters

Add upgrade to Upgrade Schedules

make jen

Update to go-state-types v0.11.0-alpha-2

* feat: vm: switch to the new exec trace format (#10372)

This is now "FVM" native. Changes include:

1. Don't treat "trace" messages like off-chain messages. E.g., don't
include CIDs, versions, etc.
2. Include IPLD codecs where applicable.
3. Remove fields that aren't filled by the FVM (timing, some errors,
code locations, etc.).

* feat: implement FIP-0061

* Address review

* Add and test the FIP-0061 migration

* Update actors bundles to fip/20230406

* Update to go-state-types master

* Update to actors v11.0.0-rc1

* - Update go state types
- Keep current expiration defaults on creation, extension some tests
- Update ffi

* ffi experiment

* Integration nv19 migration

- Open splitstore in migration shed tool
- Update state root version

* Post rebase fixup

* Fix

* gen

* nv19 invariant checking

* Try fixig blockstore so bundle is loaded

* Debug

* Fix

* Make butterfly upgrades happen

* Another ffi experiment

* Fix copy paste error

* Actually schedule migration (#10656)

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Butterfly artifacts

* Set calibration net upgrade height

* Review Response

* Fix state tree version assert

* Quick butterfly upgrade to sanity check (#10660)

* Quick butterfly upgrade to sanity check

* Update butterfly artifacts

* Revert fake fix

* Give butterfly net correct genesis

* Butterfly artifacts

* Give time before upgrade

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* chore:releasepolish v1.22 release (#10666)

* Update butterfly artifacts

* register actors v11

* Update calibration upgrade time

* State inspection shed cmds

* Fix

* make gen

* Fix swallowed errors

* Lint fixup

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* v1.22.0-rc3

* bundle fix

* Feat/expedite nv19 (#10681)

* Update go-state-types

* Modify upgrade schedule and params

* Revert fip 0052

* Update gst

* docsgen

* fast butterfly migration to validate migration

* Correct epoch to match specified date

* Update actors v11

* Update changelog build version

* Update butterfly artifacts

* Fix lotus-miner init to work after upgrade

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* fix:deps:stable ffi for stable release  (#10698)

* Point to stable ffi for stable lotus release

* go mod tidy

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Update CHANGELOG.md

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

---------

Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: jennijuju <jiayingw703@gmail.com>
2023-04-19 18:40:18 -04:00
Steven Allen
141c020b4e fix: remove pointless panic
Technically, the block validator caught this panic. But it's pointless
because we have a _real_ mechanism to return the validation reason,
which we should have been using.

In general, panicing like this is a very bad idea because it's
non-obvious and, in this case, completely undocumented.
2023-04-18 15:44:56 -07:00
Jorropo
fe42d974a2 chore: all: migrate from go-libipfs to boxo
github.com/ipfs/libipfs/blocks was unmigrated to github.com/ipfs/go-block-format due to compatibility issues with the rest of the IPLD stack.
2023-04-18 17:22:18 +02:00
Steven Allen
e945c0d6f2
fix: check for nil bcastDict (#10646)
Also hold the lock when checking the length of the blocks in the
bcastDict.
2023-04-17 20:17:10 -07:00
Steven Allen
3f74840b67
test: events: fix race when recording tipsets (#10665)
fixes #10664
2023-04-13 17:20:34 -07:00
vyzo
b4e2e871dc prune excess messages before selection 2023-04-11 13:12:05 +03:00
Jiaying Wang
522388820a
Merge pull request #10552 from filecoin-project/asr/optimize-blockmsgsfortipset
feat: chainstore: optimize BlockMsgsForTipset
2023-04-04 13:53:23 -04:00
vyzo
bf666a3f7e
Merge pull request #10551 from filecoin-project/vyzo/feat/exec-lanes
feat: VM Execution Lanes
2023-03-30 22:19:11 +03:00
Aayush Rajasekaran
139bde3771
Merge pull request #10561 from filecoin-project/mikers/messagepoolRWMutexes
perf: message pool: change locks to RWMutexes for performance
2023-03-30 12:47:44 -04:00
vyzo
d71b528253 reorg initialization code for better readability, remove unused import 2023-03-30 18:15:13 +03:00
vyzo
7b4e68249a add comment about Signal unsoundness 2023-03-30 18:13:08 +03:00
vyzo
54a80a8a97 revert dead code 2023-03-30 18:11:44 +03:00
Aayush Rajasekaran
8b2208fd9a
Merge pull request #9858 from adlrocha/adlrocha/consistent-bcast
feat:networking:  (Synchronous) Consistent Broadcast for Filecoin EC
2023-03-30 09:32:09 -04:00
adlrocha
682ddf6ffa
Update chain/sub/bcast/consistent.go
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
2023-03-30 12:38:00 +02:00
Jiaying Wang
caa6dd9c41
Merge pull request #10594 from filecoin-project/phi/v1210-rc2
build: release: v1.21.0-rc2
2023-03-29 15:59:56 -04:00
Aayush
66fc6dc3e5 refactor: stop using deprecated io/ioutil 2023-03-29 15:40:49 -04:00
Aayush
a724a713da refactor: stop using deprecated io/ioutil 2023-03-29 15:38:46 -04:00
Aayush
3120852eb9 feat: supply: only grab genesis msig locks for writes 2023-03-29 14:26:15 -04:00
Aayush
24381411f2 feat: supply: drop genesis market locked funds 2023-03-29 14:26:08 -04:00
Mikers
06294351fa perf: eth: gas estimate set applyTsMessages false (#10546)
* have gas estimate call callInternal with applyTsMessages = false and other calls with applyTsMessages=true for gas caclulation optimization

* set applyTsMessages = true in CallWithGas call in shed

* update test with new callwithgas api optimization for eth call

* Update chain/stmgr/call.go

Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* make sure that even if we arent apply ts messages we grab ts messages from the particular user who is requesting gas estimation

---------

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-10-0-4-29.us-east-2.compute.internal>
2023-03-29 14:25:40 -04:00
Friðrik Ásmundsson
36913db0cd
Merge pull request #10587 from filecoin-project/10583-enable-index-for-statewaitformessage
feat: Use MessageIndex in WaitForMessage
2023-03-29 18:38:39 +02:00
Mikers
41ea5a6f63
Update chain/messagepool/messagepool.go
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
2023-03-29 05:44:25 -10:00
Fridrik Asmundsson
2c98a187dd Use defaultExecTraceCacheSize in case env cannot be parsed 2023-03-29 17:30:00 +02:00
vyzo
52d70d563a fix tests 2023-03-29 16:46:37 +03:00