cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
20,699 Commits 24 Branches 342 Tags 314 MiB
881a8e4569
Commit Graph

4422 Commits

Author SHA1 Message Date
mx
48912e8f09 fix: chain error 2023-06-13 09:46:36 +08:00
mx
36b5f485cd Merge branch 'slashfilter' of https://github.com/storswiftlabs/lotus into slashfilter 2023-06-09 10:18:45 +08:00
mx
2fca475dd4 fix: supply extra params for ReportConsensusFaultParams 2023-06-09 10:15:14 +08:00
swift-mx
e3c796c848
Merge branch 'master' into slashfilter 2023-06-08 18:56:49 +08:00
Fridrik Asmundsson
9240776237 fix: improve perf of msgindex backfill 2023-06-01 14:36:05 +00:00
swift-mx
2fd8602c5a
Merge branch 'master' into slashfilter 2023-05-31 10:02:01 +08:00
Łukasz Magiera
6cc506f5cf
Merge pull request #10923 from filecoin-project/feat/sectors-upgradebounds-cli 
feat: miner cli: sectors list upgrade-bounds tool
 2023-05-30 20:34:38 +02:00
swift-mx
196cd0b739
fix: add more log for error 2023-05-31 00:06:50 +08:00
Łukasz Magiera
87ae60a2bc
Merge pull request #10650 from filecoin-project/vyzo/test/mpool-select 
Add timing test for mpool select with a large mpool dump
 2023-05-30 17:00:06 +02:00
Łukasz Magiera
183e35f7b7
Merge pull request #10925 from filecoin-project/asr/drop-orange-check 
feat: slashfilter: drop outdated near-upgrade check
 2023-05-30 16:56:43 +02:00
Aayush Rajasekaran
3d061ba0ae
Merge pull request #10649 from filecoin-project/asr/test-gas-reward 
test: messagepool: gas rewards are negative if GasFeeCap too low
 2023-05-30 10:25:35 -04:00
mx
425661772d style:Refactoring slashfilter 2023-05-30 14:33:35 +08:00
Aayush Rajasekaran
b76b5ef4b7
Merge pull request #10863 from AnomalRoil/fix/fuzz/48208 
fix: types: error out on decoding BlockMsg with extraneous data
 2023-05-29 16:58:41 -04:00
Aayush Rajasekaran
867a7b1c01
Merge pull request #10628 from filecoin-project/asr/exec-trace-compute 
fix: include extra messages in ComputeState InvocResult output
 2023-05-29 12:56:43 -04:00
Aayush
10ed18ba12 test: messagepool: gas rewards are negative if GasFeeCap too low 2023-05-29 12:54:04 -04:00
Aayush Rajasekaran
0ec2acbf20
Merge pull request #10652 from filecoin-project/asr/mpool-fee-too-low 
feat: pubsub: treat ErrGasFeeCapTooLow as ignore, not reject
 2023-05-29 12:53:38 -04:00
Aayush
228b306370 feat: slashfilter: drop outdated near-upgrade check 2023-05-29 11:12:19 -04:00
Łukasz Magiera
14f9770859 feat: miner cli: sectors list upgrade-bounds tool 2023-05-26 12:29:36 +02:00
Łukasz Magiera
8c8e287be0
Merge pull request #10896 from filecoin-project/feat/chainindex-sharded-mutex 
feat: chainstore: sharded mutex for filling chain height index
 2023-05-24 12:49:16 +02:00
simlecode
6ec413b7ad fix: not call RUnlock 2023-05-23 17:43:39 +08:00
Yolan Romailler
00e34436b3
Addressing code review comments 
Signed-off-by: Yolan Romailler <AnomalRoil@users.noreply.github.com>
 2023-05-22 21:47:55 +02:00
Jakub Sztandera
cad743ec54
Appease the linter 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2023-05-19 20:02:47 +02:00
Jakub Sztandera
dfa7fc7723
feat: chainstore: sharded mutex for filling chain height index 
This PR introduces as sharded mutex within the ChainIndex#GetTipsetByHeight.
It also replaces a go map with xsync.Map which doesn't require locking.

The lock is taken when it appears that ChainIndex filling work should be
started. After claiming the lock, the status of the cache is rechecked,
if the entry is still missing, the fillCache is started.

Thanks to @snissn and @arajasek for debugging and taking initial stabs at this.

Supersedes #10866 and 10885

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2023-05-19 19:49:22 +02:00
gammazero
f35fa5757a Replace use of storetheindex with go-libipni 2023-05-17 11:41:00 -07:00
Shrenuj Bansal
b0d1385273 Check if epoch is negative in GetTipsetByHeight 2023-05-16 21:19:03 -04:00
Yolan Romailler
0d7d906abf
Adding extra data check on DecodeBlockMsg 
fix: types: error out on decoding BlockMsg with extraneous data
Fixes OSS-fuzz issue 48208: lotus:fuzz_block_msg

Signed-off-by: Yolan Romailler <anomalroil@users.noreply.github.com>
 2023-05-12 21:32:44 +02:00
Friðrik Ásmundsson
b4ea0db04f
Merge pull request #10719 from filecoin-project/10622-add-eth-syncing 
feat: Add eth_syncing RPC method
 2023-05-10 16:21:11 -04:00
Fridrik Asmundsson
2bc205ed62 feat: Add eth_syncing RPC method 
This commit adds eth_syncing RPC method which returns an object
with data about the sync status or false.
 2023-05-10 16:08:18 -04:00
Aayush
fb228ebbff feat: mempool: reintroduce ForEachPendingMessage 2023-05-10 14:19:33 -04:00
Friðrik Ásmundsson
ceb3f1e41e
Validate that FromBlock/ToBlock epoch is indeed a hex value (#10780) 
* Validate that FromBlock/ToBlock epoch is indeed a hex value

* Adding tests
 2023-05-09 17:17:23 -04:00
Aayush
596b01fba2 feat: chainstore: exit early in MaybeTakeHeavierTipset 2023-05-05 13:56:48 -04:00
Mike Seiler
9c05a1f734 chain errors using xerrors.Errorf 2023-05-04 23:50:07 -10:00
Friðrik Ásmundsson
e6c8072f50
Merge pull request #10827 from filecoin-project/10585-address-review-comments 
10585 address review comments
 2023-05-04 18:32:04 +00:00
Fridrik Asmundsson
4e8ee737fc Address review comments 2023-05-04 17:10:26 +00:00
Aayush
b45409262f feat: deflake msgindex_test.go 2023-05-04 12:04:28 -04:00
Friðrik Ásmundsson
dec3af7d7f
Merge pull request #10585 from filecoin-project/10584-make-execution-cache-size-configurable 
feat: Make execution trace configurable via env variable
 2023-05-04 14:07:35 +00:00
Mikers
c1d22d8b32
refactor: streamline error handling in CheckPendingMessages (#10818) 2023-05-03 11:12:47 -10:00
Mikers
742062f84c
perf: mempool: lower priority optimizations (#10693) 
* release the read lock earlier as it is not needed for chaincomputebasefee

* chain/messagepool/selection.go change to read lock in SelectMessages

* tighten up locks in chain/messagepool/repub.go and two questions on whether curTsLks are needed as comments

* include suggestion from @Jorropo to preallocate our msgs array so that we only need to make a single allocation

* mp.pending should not be accessed directly but through the getter

* from @arajasek: just check whether the sender is a robust address (anything except an ID address is robust) here, and return if so. That will:

be faster
reduce the size of this cache by half, because we can drop mp.keyCache.Add(ka, ka) on line 491.

* do not need curTslk and clean up code comments
 2023-05-03 16:31:39 -04:00
Aayush
49f5825296 feat: chainstore: batch writes of tipsets 2023-05-02 17:26:07 -04:00
Aayush Rajasekaran
ff0fa3471f
chore: refactor: drop unused IsTicketWinner (#10801) 2023-05-02 11:22:02 -07:00
Steven Allen
bb5ba64cca Revert "Merge pull request #9858 from adlrocha/adlrocha/consistent-bcast" 
This reverts commit 8b2208fd9a, reversing
changes made to 2db6b12b78.

Unfortunately, this is rather tricky code. We've found several issues so
far and, while we've fixed a few, there are outstanding issues that
would require complex fixes we don't have time to tackle right now.

Luckily, this code isn't actually needed by the main Filecoin chain
which relies on consensus fault reporting to handle equivocation. So we
can just try again later.
 2023-04-27 12:25:30 -07:00
Steven Allen
d2c3e84d54
feat: sync: harden chain sync (#10756) 
* fix: sync: fail sync instead of logging if we sync the wrong chain

* fix: sync: write headers in the correct order

Just in case. This shouldn't be necessary, but we might as well.

* fix: minus minus

* fix: do put the tipset

Put != Persist
 2023-04-26 14:12:45 -07:00
Aayush Rajasekaran
eb0b1c986b
Merge pull request #10713 from filecoin-project/steb/fix-event-subscription 
fix: events: don't set GC confidence to 1
 2023-04-26 16:14:20 -04:00
Steven Allen
a97dae8f46 fix: sync: reduce log from error to info 
And fix the message to account for the fact that we now reject _old_
blocks along with new ones.

We frequently receive "out of date" blocks in hello messages from
syncing and/or out of sync nodes. This isn't an error.
 2023-04-25 14:34:13 -07:00
Steven Allen
7162c656cc
fix: chain: record heaviest tipset before notifying (#10694) 
Clearly this hasn't caused any issues, but I'm pretty sure we should be
updating the current head _before_ notifying about it.
 2023-04-25 11:56:26 -07:00
ychiao
9ae48022ff
fix: Eth JSON-RPC api: handle messages with gasFeeCap less than baseFee (#10614) 2023-04-24 16:03:04 +01:00
Aayush
5a061e4677 Merge branch 'releases' into asr/merge-release-into-master 2023-04-23 14:58:50 -04:00
Steven Allen
784214ae05
feat: sync: validate (early) that blocks fall within range (#10691) 
This will reject blocks in pubsub validation if they're either:

1. Too far into the future (5 blocks beyond the expected head).
2. Too far into the past (before finality with respect to our current
   head).

Specifically:

1. We were previously rejecting future blocks in the sync logic, but not
   in pubsub itself.
2. We never used to check if a block was too _old_.

Motivation: Blocks that are too new/too old can cause us to perform
quite a bit of unnecessary work.
 2023-04-22 10:15:31 -07:00
Hector Sanjuan
875c09840b
chainstore: Fix raw blocks getting scanned for links during snapshots (#10684) 
We have to save raw blocks to the snapshot, but we should not be scanning them
for additional links as if they were CBOR blocks.

This cleans the logic a bit (we were checking that the parent was a CBOR block
before queueing up the children, but then scanning the children... it was weird).

Additionally, more verbose logging is added for the next time ScanForLinks
fails (currently very little info was given).

Our ScanForLinks callback should only enqueue CBOR for further processing.
 2023-04-21 15:16:26 -07:00
Friðrik Ásmundsson
9d5d6a5f3d
Merge pull request #10700 from filecoin-project/10538-opt-ethgettransactioncount 
perf: Address performance of EthGetTransactionCount
 2023-04-21 20:13:56 +00:00
Fridrik Asmundsson
4028c05fea address review comment 2023-04-21 20:03:13 +00:00
Fridrik Asmundsson
553da395e4 perf: Increase noncecache in MessagePool 
Bumped from 256 to 32k entries which should be about 6MB of cached
entries given average nonceCacheKey of 200 bytes
 2023-04-21 11:56:05 +00:00
Fridrik Asmundsson
953d56e216 perf: Address performance of EthGetTransactionCount 
We have observed that EthGetTransactionCount is one of the hotspots
on Glif production notes, and we are seeing regular 10-20 second
latencies when calling this rpc method.

I tracked the high latency spikes and they were correlated when
we were running ExecuteTipSet while following the chain.

To address this, we should not rely on tipset computation to get
nounce and instead look at the parent tipset and then count the
messages sent from the 'addr'.
 2023-04-21 11:55:46 +00:00
Łukasz Magiera
a503a0edaa
Merge pull request #10690 from filecoin-project/fix/remove-pointless-panic 
fix: remove pointless panic
 2023-04-21 13:47:12 +02:00
Steven Allen
c84c07eb74 fix: events: don't set GC confidence to 1 
The function/parameter were poorly named and should never have been
exposed. "GC" confidence should always be the same, this parameter
doesn't let us actually set the _confidence_, just the point before
which we no longer support reverts.

fixes #10706
 2023-04-20 14:45:55 -07:00
Łukasz Magiera
0a46fc7e73
Merge pull request #10562 from Jorropo/boxo 
chore: boxo: migrate from go-libipfs to boxo
 2023-04-20 16:24:22 +02:00
ZenGround0
327760acff
chore: build: Merge/v22 into 21 for 23 (#10702) 
* chore: update ffi to increase execution parallelism

* Don't enforce walking receipt tree during compaction

* fix: build: drop drand incentinet servers

* chore: release lotus v1.20.4

* Apply suggestions from code review

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

* feat: Introduce nv19 skeleton

Update to go-state-types v0.11.0-alpha-1

Introduce dummy v11 actor bundles

Make new actors adapters

Add upgrade to Upgrade Schedules

make jen

Update to go-state-types v0.11.0-alpha-2

* feat: vm: switch to the new exec trace format (#10372)

This is now "FVM" native. Changes include:

1. Don't treat "trace" messages like off-chain messages. E.g., don't
include CIDs, versions, etc.
2. Include IPLD codecs where applicable.
3. Remove fields that aren't filled by the FVM (timing, some errors,
code locations, etc.).

* feat: implement FIP-0061

* Address review

* Add and test the FIP-0061 migration

* Update actors bundles to fip/20230406

* Update to go-state-types master

* Update to actors v11.0.0-rc1

* - Update go state types
- Keep current expiration defaults on creation, extension some tests
- Update ffi

* ffi experiment

* Integration nv19 migration

- Open splitstore in migration shed tool
- Update state root version

* Post rebase fixup

* Fix

* gen

* nv19 invariant checking

* Try fixig blockstore so bundle is loaded

* Debug

* Fix

* Make butterfly upgrades happen

* Another ffi experiment

* Fix copy paste error

* Actually schedule migration (#10656)

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Butterfly artifacts

* Set calibration net upgrade height

* Review Response

* Fix state tree version assert

* Quick butterfly upgrade to sanity check (#10660)

* Quick butterfly upgrade to sanity check

* Update butterfly artifacts

* Revert fake fix

* Give butterfly net correct genesis

* Butterfly artifacts

* Give time before upgrade

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* chore:releasepolish v1.22 release (#10666)

* Update butterfly artifacts

* register actors v11

* Update calibration upgrade time

* State inspection shed cmds

* Fix

* make gen

* Fix swallowed errors

* Lint fixup

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* v1.22.0-rc3

* bundle fix

* Feat/expedite nv19 (#10681)

* Update go-state-types

* Modify upgrade schedule and params

* Revert fip 0052

* Update gst

* docsgen

* fast butterfly migration to validate migration

* Correct epoch to match specified date

* Update actors v11

* Update changelog build version

* Update butterfly artifacts

* Fix lotus-miner init to work after upgrade

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* fix:deps:stable ffi for stable release  (#10698)

* Point to stable ffi for stable lotus release

* go mod tidy

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Update CHANGELOG.md

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

---------

Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: jennijuju <jiayingw703@gmail.com>
 2023-04-19 18:40:18 -04:00
Steven Allen
141c020b4e fix: remove pointless panic 
Technically, the block validator caught this panic. But it's pointless
because we have a _real_ mechanism to return the validation reason,
which we should have been using.

In general, panicing like this is a very bad idea because it's
non-obvious and, in this case, completely undocumented.
 2023-04-18 15:44:56 -07:00
Jorropo
fe42d974a2 chore: all: migrate from go-libipfs to boxo 
github.com/ipfs/libipfs/blocks was unmigrated to github.com/ipfs/go-block-format due to compatibility issues with the rest of the IPLD stack.
 2023-04-18 17:22:18 +02:00
Steven Allen
e945c0d6f2
fix: check for nil bcastDict (#10646) 
Also hold the lock when checking the length of the blocks in the
bcastDict.
 2023-04-17 20:17:10 -07:00
Steven Allen
3f74840b67
test: events: fix race when recording tipsets (#10665) 
fixes #10664
 2023-04-13 17:20:34 -07:00
Aayush
ae84f335cc feat: pubsub: treat ErrGasFeeCapTooLow as ignore, not reject 2023-04-11 10:26:43 -04:00
vyzo
dd6720c61b message data 2023-04-11 16:19:59 +03:00
vyzo
e358b3b358 add real world timing test for selection 2023-04-11 16:19:50 +03:00
vyzo
b4e2e871dc prune excess messages before selection 2023-04-11 13:12:05 +03:00
Aayush
7710a57fa3 fix: include extra messages in ComputeState InvocResult output 2023-04-05 15:25:57 -04:00
Jiaying Wang
522388820a
Merge pull request #10552 from filecoin-project/asr/optimize-blockmsgsfortipset 
feat: chainstore: optimize BlockMsgsForTipset
 2023-04-04 13:53:23 -04:00
vyzo
bf666a3f7e
Merge pull request #10551 from filecoin-project/vyzo/feat/exec-lanes 
feat: VM Execution Lanes
 2023-03-30 22:19:11 +03:00
Aayush Rajasekaran
139bde3771
Merge pull request #10561 from filecoin-project/mikers/messagepoolRWMutexes 
perf: message pool: change locks to RWMutexes for performance
 2023-03-30 12:47:44 -04:00
vyzo
d71b528253 reorg initialization code for better readability, remove unused import 2023-03-30 18:15:13 +03:00
vyzo
7b4e68249a add comment about Signal unsoundness 2023-03-30 18:13:08 +03:00
vyzo
54a80a8a97 revert dead code 2023-03-30 18:11:44 +03:00
Aayush Rajasekaran
8b2208fd9a
Merge pull request #9858 from adlrocha/adlrocha/consistent-bcast 
feat:networking:  (Synchronous) Consistent Broadcast for Filecoin EC
 2023-03-30 09:32:09 -04:00
adlrocha
682ddf6ffa
Update chain/sub/bcast/consistent.go 
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
 2023-03-30 12:38:00 +02:00
Jiaying Wang
caa6dd9c41
Merge pull request #10594 from filecoin-project/phi/v1210-rc2 
build: release: v1.21.0-rc2
 2023-03-29 15:59:56 -04:00
Aayush
66fc6dc3e5 refactor: stop using deprecated io/ioutil 2023-03-29 15:40:49 -04:00
Aayush
a724a713da refactor: stop using deprecated io/ioutil 2023-03-29 15:38:46 -04:00
Aayush
3120852eb9 feat: supply: only grab genesis msig locks for writes 2023-03-29 14:26:15 -04:00
Aayush
24381411f2 feat: supply: drop genesis market locked funds 2023-03-29 14:26:08 -04:00
Mikers
06294351fa perf: eth: gas estimate set applyTsMessages false (#10546) 
* have gas estimate call callInternal with applyTsMessages = false and other calls with applyTsMessages=true for gas caclulation optimization

* set applyTsMessages = true in CallWithGas call in shed

* update test with new callwithgas api optimization for eth call

* Update chain/stmgr/call.go

Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* make sure that even if we arent apply ts messages we grab ts messages from the particular user who is requesting gas estimation

---------

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-10-0-4-29.us-east-2.compute.internal>
 2023-03-29 14:25:40 -04:00
Friðrik Ásmundsson
36913db0cd
Merge pull request #10587 from filecoin-project/10583-enable-index-for-statewaitformessage 
feat: Use MessageIndex in WaitForMessage
 2023-03-29 18:38:39 +02:00
Mikers
41ea5a6f63
Update chain/messagepool/messagepool.go 
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
 2023-03-29 05:44:25 -10:00
Fridrik Asmundsson
2c98a187dd Use defaultExecTraceCacheSize in case env cannot be parsed 2023-03-29 17:30:00 +02:00
vyzo
52d70d563a fix tests 2023-03-29 16:46:37 +03:00
vyzo
4184ce9c75 refactor execution lanes: hide the lock 2023-03-29 16:45:45 +03:00
Fridrik Asmundsson
f44fa5d96f Use MessageIndex in WaitForMessage 2023-03-29 13:35:27 +02:00
Fridrik Asmundsson
8aa4917741 Fix bug in searchForIndexedMsg which always returns an error 2023-03-29 13:18:26 +02:00
Fridrik Asmundsson
17505a0022 Make execution trace configurable via env variable 
We want to make the execution trace cache size configurable as SPs
may want to disable it while exchanges may want to crank it up.

We were also are going with intuition for this value, so having
ability to change it without a new build would help.

Fixes: https://github.com/filecoin-project/lotus/issues/10584
 2023-03-29 11:53:43 +02:00
Mike Seiler
3477f7ce57 use LRU cache for keyCache to make threadsafe, also have (high) upper bounds on size of cache 2023-03-28 13:14:33 -10:00
Mike Seiler
89b217ee21 move write lock to before verifyMsg 2023-03-28 12:35:09 -10:00
Mike Seiler
2fa95a09be clean up cache logic in addTs / fix a bug where the loop was incorrectly releasing the lock 2x 2023-03-28 11:14:13 -10:00
Aayush Rajasekaran
4b2e74a6df
Merge pull request #10535 from hsanjuan/filcryo-v1.20.0-branch 
Fix: export-range: Ignore ipld Blocks not found in Receipts.
 2023-03-28 16:01:11 -04:00
Aayush Rajasekaran
687f0c070f
Merge pull request #10556 from filecoin-project/10537-populate-index-after-snapshot 
feat: populate the index on snapshot import
 2023-03-28 11:41:43 -04:00
Fridrik Asmundsson
ecd13079e7 Address review comments 2023-03-28 17:08:53 +02:00
vyzo
71650cd8a4 rename newVM to makeVM for a happy yushie 2023-03-28 18:05:00 +03:00
vyzo
b27121612e rename confusing variable 2023-03-28 18:03:55 +03:00
vyzo
b2b78e9dfa
Update chain/vm/execution.go 
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
 2023-03-28 17:56:35 +03:00
Alfonso de la Rocha
b7c297c537 Merge branch 'master' into adlrocha/consistent-bcast 2023-03-28 16:55:06 +02:00
Alfonso de la Rocha
1a771e4310
include a deeper gc round 2023-03-28 16:52:32 +02:00
vyzo
dcd9869842 make gen 2023-03-28 16:58:09 +03:00
Hector Sanjuan
39b27e709e export-range: use debug log instead of warn for not found receipt events 2023-03-28 15:05:43 +02:00
Fridrik Asmundsson
2e45f6f778 Dont do locking using defer 2023-03-28 12:36:57 +02:00
Fridrik Asmundsson
f84f8a831a Use TipSetKey as key in cache and return copies 2023-03-28 12:36:57 +02:00
Fridrik Asmundsson
1ea7e05cde feat: Add small cache to execution traces 
This PR adds a small cache to calls to ExecutionTrace which helps
 improve performance for node operators like exchanges and block
explorers.

If items is in cache calls to this function will be 2-3x faster.

Fixes: https://github.com/filecoin-project/lotus/issues/10504
 2023-03-28 12:36:52 +02:00
Mikers
831f8a499d repub needs Lock not RLock 2023-03-28 08:40:41 +00:00
Alfonso de la Rocha
df82a8240e
add comments 2023-03-28 10:33:26 +02:00
Mikers
e9d2223025 disallow infinite loop, since in testing this only runs once 2023-03-28 08:11:23 +00:00
Mikers
ad81cd18c2 cache the tipset nonce calculation before holding the write lock, and also verify that the the calculation is cached after grabbing the write lock. if it is not cached, give up the lock, calculate, and then grab the write lock again 2023-03-28 08:11:00 +00:00
Alfonso de la Rocha
92f6d3e468
global locking strategy for blockInfo map 2023-03-28 10:01:43 +02:00
Fridrik Asmundsson
90171c8bab Addressing lint errors 2023-03-25 11:17:54 +00:00
Mike Seiler
6f440a6420 change highly contented message pool locks to RWMutexes for performance 2023-03-24 15:49:02 -10:00
Fridrik Asmundsson
b8137f6b4d Delete existing message index when loading from snapshot 2023-03-24 16:45:33 +00:00
Fridrik Asmundsson
48c57d394b Improve performance when populating message indax 2023-03-24 15:36:31 +00:00
vyzo
08134552a4 address review comments 2023-03-24 15:48:58 +02:00
vyzo
4b590e2102 add vm execution metrics 2023-03-24 15:16:22 +02:00
Fridrik Asmundsson
59640a8b22 Populate the index on snapshot import 
Fixes: https://github.com/filecoin-project/lotus/issues/10537
 2023-03-24 11:41:27 +00:00
Mikers
41fce94db4
perf: eth: gas estimate set applyTsMessages false (#10546) 
* have gas estimate call callInternal with applyTsMessages = false and other calls with applyTsMessages=true for gas caclulation optimization

* set applyTsMessages = true in CallWithGas call in shed

* update test with new callwithgas api optimization for eth call

* Update chain/stmgr/call.go

Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* env flag LOTUS_SKIP_APPLY_TS_MESSAGE_CALL_WITH_GAS must be 1 in order to have applyTsMessages change

* make sure that even if we arent apply ts messages we grab ts messages from the particular user who is requesting gas estimation

---------

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
Co-authored-by: Ubuntu <ubuntu@ip-10-0-4-29.us-east-2.compute.internal>
 2023-03-23 18:27:01 -04:00
Aayush
bc87017ea5 feat: supply: only grab genesis msig locks for writes 2023-03-23 17:04:59 -04:00
Aayush
dcdd1bc214 feat: supply: drop genesis market locked funds 2023-03-23 15:56:59 -04:00
vyzo
f11a7f8940 fix incorrect deferred vm release 2023-03-23 19:51:23 +02:00
vyzo
317a87d669 add some sanity checks for execution concurrency parameters 2023-03-23 18:52:37 +02:00
Aayush
b1669235f7 feat: chainstore: optimize BlockMsgsForTipset 2023-03-23 11:57:43 -04:00
vyzo
2a0660447a make token.Done idempotent 2023-03-23 17:38:36 +02:00
vyzo
2bb89d9c30 call Executor.Done where appropriate in stmgr uses 2023-03-23 17:34:59 +02:00
vyzo
ee6c0f8570 only call Atoi on non empty strings 2023-03-23 17:28:08 +02:00
vyzo
7362556c02 update VM interface references to use the executor, and call Done where appropriate 2023-03-23 17:17:46 +02:00
vyzo
6550abdfcc introduce execution lanes 2023-03-23 16:53:50 +02:00
vyzo
8abe0ea608
Merge pull request #10452 from filecoin-project/vyzo/feat/chain/msgindex 
feat:chain: Message Index
 2023-03-22 17:02:34 +02:00
Hector Sanjuan
6942020253 Ignore blockstore.Get errors when the top-level-task is Receipts 2023-03-21 21:25:50 +01:00
Alfonso de la Rocha
90c2f9dbe2
minor fix 2023-03-16 17:17:59 +01:00
Alfonso de la Rocha
8d260d7478
address review 2023-03-16 17:03:25 +01:00
vyzo
883bbf8701 add sanity check in SearchForIndexedMsg call site 
Check that receipt is non nil, and the message was indeed found.
 2023-03-16 17:51:28 +02:00
vyzo
3710768910 add TODO for WaitForMessage to use the index 2023-03-16 17:45:16 +02:00
vyzo
ef2f2b0f89 reword funny comment 2023-03-16 17:43:56 +02:00
vyzo
9c087cc52c second error variable name to avoid confusing they yushie. 2023-03-16 17:42:09 +02:00
vyzo
1e7f5c6a1e shut down the index if there is an error during head processing 2023-03-16 17:41:06 +02:00
vyzo
ff22a46253 complain if head change processing is building backlog 2023-03-16 17:38:46 +02:00
vyzo
90c4763e00 fix typos 2023-03-16 17:33:12 +02:00
vyzo
8bceaadda8 chain errors in searchForIndexedMsg 2023-03-16 17:32:06 +02:00
vyzo
91fccc421a add ON CONFLICT REPLACE clause in messages 2023-03-16 17:25:20 +02:00
adlrocha
f59c246c7a
Update chain/sub/bcast/consistent.go 
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
 2023-03-16 16:09:45 +01:00
Hector Sanjuan
4b8442d8b9 Chain export: record epoch for every task. 2023-03-16 15:31:58 +01:00
Hector Sanjuan
a3900caeb0 Improve logging by carrying some more information in tasks 2023-03-16 13:38:40 +01:00
Łukasz Magiera
7e414ac390 Merge remote-tracking branch 'origin/master' into feat/go-data-transfer-v2 2023-03-15 11:37:53 +01:00
Łukasz Magiera
162e07fc95
Merge pull request #10470 from filecoin-project/feat/applyblock-timing-log 
feat: consensus: log ApplyBlock timing/gas stats
 2023-03-14 15:25:31 +01:00
Łukasz Magiera
27e3c0b89f
fix: stmgr: don't attempt to lookup genesis state (#10472) 
* fix: stmgr: don't attempt to lookup genesis state

* stmgr: More correct fix for genesis state compute
 2023-03-14 06:45:23 -07:00
Łukasz Magiera
51c6b64c91 feat: consensus: log ApplyBlock timing/gas stats 2023-03-14 13:58:36 +01:00
Steven Allen
dcb49dc8ee
refactor: update cache to the new generic version (#10463) 
- Adds type safety.
- Reduces allocations.
- Fixes the drand cache (was storing by value, but retrieving by pointer)
 2023-03-13 15:29:09 -07:00
vyzo
5ade40cbac increase coalesce delays for test to deflake on CI 2023-03-13 17:27:12 +02:00
vyzo
47646cbd69 add optimization TODO comment 2023-03-13 15:19:07 +02:00
vyzo
1970792445 fix bug in searchForIndexedMsg 
Need to use and return the execution tipset
 2023-03-13 14:56:00 +02:00
vyzo
4b1a405002 basic msgindex itest 2023-03-13 14:55:26 +02:00
vyzo
05cb2428c1 add confidence check for indexed message epoch 2023-03-13 12:33:19 +02:00
vyzo
5e011d536b enhance comment about lookup cid semantics 2023-03-13 12:14:30 +02:00
vyzo
a11032b10a adjust coalescer delays 2023-03-13 11:44:10 +02:00
vyzo
bda7ef52da log rollback errors 2023-03-13 11:41:46 +02:00
vyzo
b90cfff0aa wire in lifecycle context 2023-03-13 10:42:48 +02:00
vyzo
5461548b7e fix comment typo 2023-03-13 05:57:26 +02:00
vyzo
4a20c9b60f cosmetics 2023-03-13 05:54:32 +02:00
vyzo
e7448b1bb5 fix tests 2023-03-13 05:51:33 +02:00
vyzo
df6dfdf8a9 refactor database 
- drop the execution index; we don't need it
- it is inclusion tipset
- use MessagesForTipset
- hoist db sql stuffs on top for clarity
- add index for tipset on messages
 2023-03-13 05:51:24 +02:00
vyzo
88d7a4e610 more lint 2023-03-12 16:13:05 +02:00
vyzo
1415391988 lint 
lint

lint
 2023-03-12 15:50:10 +02:00
vyzo
0077fa2a98 make gen 2023-03-12 15:30:05 +02:00
vyzo
171734ea31 hook the index into the rest of lotus 2023-03-12 15:25:07 +02:00
vyzo
0d274df977 use the transaction Luke! 2023-03-12 13:35:50 +02:00
vyzo
d9ca214309 test test test 2023-03-12 13:21:03 +02:00
vyzo
bf9ae23c98 implement Close 2023-03-11 22:09:31 +02:00
vyzo
b5dd4e31ac implement reconciliation 2023-03-11 21:26:11 +02:00
Raúl Kripalani
f7a979d825 eth_feeHistory: migrate to using TipSetState. 2023-03-11 17:25:00 +00:00
vyzo
d97c6b2f69 more implementation 2023-03-11 19:11:08 +02:00
vyzo
7fcf228bc4 better logger name 2023-03-11 18:23:05 +02:00
vyzo
3649ae373b implementation details 2023-03-11 18:21:16 +02:00
vyzo
9144f9cea7 abstract ChainStore interface 2023-03-11 17:56:23 +02:00
vyzo
eab728c4e7 we don't need the block 2023-03-11 17:25:57 +02:00
vyzo
cc83a7cd35 use tipset cid instead of key for posterity. 2023-03-11 17:12:28 +02:00
vyzo
a0e95ebe97 scaffolding 2023-03-11 16:50:29 +02:00
Steven Allen
1cf57ffe2d feat: eth: optimize receipt reading 
This optimizes the eth APIs (except the fee history one) to lookup the
tipset state/receipts instead of computing the state.
 2023-03-10 15:33:45 -08:00
Steven Allen
92bca485b6 feat: stmgr: skip tipset execution when possible 2023-03-10 13:21:17 -08:00
Steven Allen
f7603f6c13 feat: eth API: reject masked ID addresses embedded in f410f payloads 
We'll never get an actor/account deployed to one of these
addresses (although we might get a placeholder). However, converting
such an address to an f4 address is definitely wrong.
 2023-03-10 10:57:39 -08:00
Alfonso de la Rocha
c11ffa58a8
address review 2023-03-10 09:27:30 +01:00
Steven Allen
58900a7033
feat: mempool: Reduce minimum replace fee from 1.25x to 1.1x (#10416) 
However, we're leaving the default at 1.25x for backwards compatibility, for now.

Also:

1. Actually use the configured replace fee ratio.
2. Store said ratios as percentages instead of floats. 1.25, or 1+1/(2^2),
can be represented as a float. 1.1, or 1 + 1/(2 * 5), cannot.

fixes #10415
 2023-03-09 13:17:17 -08:00
Aayush Rajasekaran
b75ebcac81
Merge pull request #10423 from filecoin-project/asr/fast-index 
feat: chain: make chain tipset fetching 1000x faster
 2023-03-09 15:38:55 -05:00
Aayush
90c89288b6 feat: chain: make fetching tipset by height 1000x faster 2023-03-09 15:12:13 -05:00
Travis Person
f641139bf2
Merge pull request #10282 from filecoin-project/feat/cache-migrated-stateroot 
feat: stmgr: cache migrated stateroots
 2023-03-09 11:47:43 -08:00
Travis Person
b4e589a0f7 review comments 2023-03-09 18:54:29 +00:00
Łukasz Magiera
dcfbf7d0a6 fix TestPrintGroupInfo 2023-03-08 18:13:58 +01:00
Łukasz Magiera
d114d8fe7e deprecate MarketListRetrievalDeals, make gen 2023-03-08 17:15:16 +01:00
Marten Seemann
5b5ea083ce fix drand test 2023-03-08 17:50:58 +13:00
hannahhoward
7bc6185077
chore(deps): update to get to compile 2023-03-07 11:45:30 -08:00
Aayush
74d94af034 fix: state: short-circuit genesis state computation 2023-03-06 15:39:28 -05:00
Aayush
4e57626e42 feat: update renew-sectors with FIP-0045 logic 2023-03-02 16:38:05 -05:00
Steven Allen
dbbcf4b2ee
feat: vm: switch to the new exec trace format (#10372) 
This is now "FVM" native. Changes include:

1. Don't treat "trace" messages like off-chain messages. E.g., don't
include CIDs, versions, etc.
2. Include IPLD codecs where applicable.
3. Remove fields that aren't filled by the FVM (timing, some errors,
code locations, etc.).
 2023-03-01 16:02:18 -08:00
Aayush
19f1ad2762 Merge branch 'releases' into asr/merge-release-into-master 2023-03-01 14:18:35 -05:00
Travis Person
814c146626 feat: stmgr: cache migrated stateroots 2023-03-01 16:17:34 +00:00
Steven Allen
68b401a895 fix: cli: better handle sending from EthAccount actors 
This will make `lotus send` mostly just "do what the user wants" in this
case:

1. The user may not explicitly specify a method number.
2. Parameters are automatically cbor-encoded where applicable.
3. The method number is automatically selected based on the
   recipient (CreateExternal if sent to the EAM, InvokeEVM otherwise).
 2023-02-24 15:15:41 -08:00
Łukasz Magiera
ea57a1aec3 Merge remote-tracking branch 'origin/master' into hsanjuan/chain-export-range-rebased 2023-02-23 13:08:14 +01:00
Travis Person
bf2ac13609
Merge pull request #10283 from filecoin-project/feat/disable-pre-migrations 
feat: stmgr: add env to disable premigrations
 2023-02-21 09:23:13 -08:00
Travis Person
2dcaddf61d feat: stmgr: add env to disable premigrations 
Setting the environment variable `LOTUS_DISABLE_PRE_MIGRATIONS=1` will
discard premigrations for all upgrade.
 2023-02-17 23:12:26 +00:00
Aayush Rajasekaran
1ec02c5c95
Merge pull request #10296 from filecoin-project/asr/migration-test 
feat: shed: test the nv18 migration
 2023-02-17 16:20:51 -05:00