Commit Graph

925 Commits

Author SHA1 Message Date
Andrew Jackson (Ajax)
6fd468dfc0 harmonytask parallel test run gotcha 2023-08-21 17:47:43 -05:00
Andrew Jackson (Ajax)
415a0ac364 harmonytask more lints 2023-08-21 17:33:25 -05:00
Andrew Jackson (Ajax)
84f4cdfc15 harmonytask cleanups 2023-08-21 17:13:17 -05:00
Andrew Jackson (Ajax)
2912cf6f9c harmonytask review comments 2023-08-21 11:26:26 -05:00
Aayush
e8020e8f4e feat: test: Test that verified clients can directly transfer datacap, creating allocations 2023-08-17 09:40:29 -04:00
Phi
8b5da86727 feat: introduce local nv21 skeleton
Introduce nv21 skeleton for local testing:

- Use local go-state-types with actor_version_checklist changes: https://github.com/filecoin-project/go-state-types/blob/master/actors_version_checklist.md
- Imports mock v12-actors bundle
- Define upgrade heights
- Generate adapters
- Add upgrade schedule and migration
- Add actorstype to the NewActorRegistry in /chain/consensus/computestate.go
- Add upgrade field to api/types.go/ForkUpgradeParams
- Add upgrade to node/impl/full/state.go
- Add network version to chain/state/statetree.go
- make jen
- make docsgen-cli
2023-08-16 20:01:48 +02:00
Aayush
e6b0231ab5 Add more post-migration sanity checks 2023-08-14 18:11:47 -04:00
Łukasz Magiera
33182967eb
Merge pull request #11142 from filecoin-project/feat/act-precommv2
sealing: Switch to calling PreCommitSectorBatch2
2023-08-14 21:31:24 +02:00
Łukasz Magiera
b11fba3561 itests: Port deadline-toggling itest to nv20 2023-08-14 20:25:36 +02:00
Andrew Jackson (Ajax)
54dc885ae6 feat:miner:harmonytask 2023-08-14 11:40:12 -05:00
Shrenuj Bansal
7759444247 address review comments 2023-08-10 18:35:35 -04:00
Mikers
f57b5bdedc spelling 2023-08-09 14:28:29 -10:00
Mikers
0b68348738 initial skeleton for nv21 migrations test. migration succesfully executes. todo @aayush and @zen add relevant tests 2023-08-09 14:18:05 -10:00
Aayush
62032e7c2a feat: deflake: TestGetBlockByNumber 2023-08-09 19:25:32 -04:00
Shrenuj Bansal
652ee04ce2 working changes 2023-08-08 20:59:21 -04:00
Łukasz Magiera
aa7220d9e8 itests: Drop sealing itests for legacy network versions 2023-08-08 18:51:30 +02:00
Łukasz Magiera
41fac326b1 sealing pipeline: Fix unit tests 2023-08-08 15:06:43 +02:00
Łukasz Magiera
bb2ac5c22d ci: Drop legacy cc-replace itests 2023-08-08 13:53:24 +02:00
Łukasz Magiera
884246677b
Merge pull request #11046 from filecoin-project/fix/fsm-commfail-loop
fix: sealing: Fix RetryCommitWait loop when sector cron activation fails
2023-08-07 12:11:57 -07:00
Shrenuj Bansal
864e8219ae WIP: sector index yugabyte impl 2023-08-05 16:15:23 -04:00
Łukasz Magiera
f620310bfe address review 2023-08-02 17:04:58 +02:00
Łukasz Magiera
683432e6e0
Merge pull request #11119 from filecoin-project/chore/cleanup-compunscid
chore: legacy syscalls: Cleanup ComputeUnsealedSectorCID
2023-08-02 06:45:35 -07:00
Łukasz Magiera
6636da1299 itests: Use Must-Post mining in TestDealsWithSealingAndRPC 2023-08-01 16:34:21 +02:00
Aayush
286482fd99 fix: failing test: Tweak TestWindowPostV1P1NV20 test condition 2023-08-01 09:34:44 -04:00
Aayush
b88e4abe05 fix: failing test: Tweak TestWindowPostV1P1NV20 test condition 2023-07-31 16:07:09 -04:00
Aayush
4872ad66a7 feat: FVM: do not error on unsuccessful implicit messages 2023-07-31 15:37:54 -04:00
Aayush
cd958e4f6e feat: FVM: do not error on unsuccessful implicit messages 2023-07-31 15:26:37 -04:00
Andrew Jackson (Ajax)
d39e699e1f harmonydb: better API 2023-07-18 14:51:26 -07:00
Andrew Jackson (Ajax)
d17bfb14f8 feat: miner deps: harmonydb 2023-07-14 18:05:49 -05:00
Steven Allen
5743ed0151 test: fix fevm recursive call
We've now lost 2 recursive calls, but that really shouldn't make a
difference.
2023-07-13 22:26:25 -07:00
Łukasz Magiera
b649ebddd5 TestPledgeSynth: must-post mining 2023-07-13 15:48:20 +02:00
Łukasz Magiera
50cf19722a Make Synthetic PoRep work 2023-07-13 15:48:20 +02:00
Łukasz Magiera
f8c151094f itests: Add test for synth-porep pledge 2023-07-13 15:48:20 +02:00
Andrew Jackson (Ajax)
7f306a5b02 Synthetic PoRep Impl 2023-07-13 15:47:59 +02:00
Andrew Jackson (Ajax)
b43665a3c1 synthPoRep in PreferredSeal..() 2023-07-13 15:47:58 +02:00
Andrew Jackson (Ajax)
1db45f46ae synthetic-porep subpackages 2023-07-13 15:47:20 +02:00
Phi
c7b9633160 feat: introduce local nv21 skeleton
Introduce nv21 skeleton for local testing:

- Use local go-state-types with actor_version_checklist changes: https://github.com/filecoin-project/go-state-types/blob/master/actors_version_checklist.md
- Imports mock v12-actors bundle
- Define upgrade heights
- Generate adapters
- Add upgrade schedule and migration
- Add actorstype to the NewActorRegistry in /chain/consensus/computestate.go
- Add upgrade field to api/types.go/ForkUpgradeParams
- Add upgrade to node/impl/full/state.go
- Add network version to chain/state/statetree.go
- make jen
- make docsgen-cli
2023-07-13 15:47:18 +02:00
Aayush
5d072409f8 feat: deflake: Use MockProofs 2023-07-10 10:26:51 -04:00
Łukasz Magiera
402895fdf0 itests: Test too-slow sector sealing cleanup 2023-07-06 09:27:09 +02:00
Friðrik Ásmundsson
a2431ff70a
Merge pull request #10815 from filecoin-project/10814-eip-1891
Add EIP-1898 support needed for The Graph compatibility
2023-06-23 08:29:45 +00:00
Fridrik Asmundsson
dda2d7e023 Refactor EthBlockNumberOrHash and remove the number field 2023-06-22 17:18:50 +00:00
Fridrik Asmundsson
eed38a3c21 Fix lint errors 2023-06-21 15:34:39 +00:00
Fridrik Asmundsson
f358160cd5 Add EIP-1898 support needed for The Graph compatibility
Fixes: #10814

This PR updates the following RPC methods according to EIP-1898
specs.

The following RPC methods are affected:

- eth_getBalance
- eth_getStorageAt
- eth_getTransactionCount
- eth_getCode
- eth_call

Note that eth_getBlockByNumber was not included in this list in
the spec although it seems it should be affected also?

Currently these methods all accept a blkParam string which can be
one of "latest", "earliest", "pending", or a block number (decimal
or hex). The spec enables caller to additionally specify a json
hash which can include the following fields:

- blockNumber EthUint64: A block number (decimal or hex) which is
  similar to the original use of the blkParam string
- blockHash EthHash: The block hash
- requireCanonical bool) If true we should make sure the block is
  in the canonical chain

Since the blkParam needs to support both being a number/string and
a json hash then this to properly work we need to introduce a new
struct with pointer fields to check if they exist. This is done
in the EthBlockParamByNumberOrHash struct which first tries to
unmarshal as a json hash (according to eip-1898) and then fallback
to unmarshal as string/number.
2023-06-20 09:38:00 +00:00
Jorropo
6c01310728
chore: migrate to boxo
This migrates everything except the `go-car` librairy: https://github.com/ipfs/boxo/issues/218#issuecomment-1529922103

I didn't migrated everything in the previous release because all the boxo code wasn't compatible with the go-ipld-prime one due to a an in flight (/ aftermath) revert of github.com/ipfs/go-block-format. go-block-format has been unmigrated since slight bellow absolutely everything depends on it that would have required everything to be moved on boxo or everything to optin into using boxo which were all deal breakers for different groups.

This worked fine because lotus's codebase could live hapely on the first multirepo setup however boost is now trying to use boxo's code with lotus's (still on multirepo) setup: https://filecoinproject.slack.com/archives/C03AQ3QAUG1/p1685022344779649

The alternative would be for boost to write shim types which just forward calls and return with the different interface definitions.

Btw why is that an issue in the first place is because unlike what go's duck typing model suggest interfaces are not transparent https://github.com/golang/go/issues/58112, interfaces are strongly typed but they have implicit narrowing. The issue is if you return an interface from an interface Go does not have a function definition to insert the implicit conversion thus instead the type checker complains you are not returning the right type.

Stubbing types were reverted https://github.com/ipfs/boxo/issues/218#issuecomment-1478650351

Last time I only migrated `go-bitswap` to `boxo/bitswap` because of the security issues and because we never had the interface return an interface problem (we had concrete wrappers where the implicit conversion took place).
2023-06-19 14:45:05 -07:00
Aayush
44aadb7962 feat: deflake TestDealsWithFinalizeEarly 2023-06-14 11:29:14 -04:00
Aayush Rajasekaran
1ba0e4b997
feat: deflake TestDMLevelPartialRetrieval (#10972) 2023-06-12 16:49:13 -07:00
Łukasz Magiera
b997f4acec
Merge pull request #10854 from filecoin-project/revert-10852-sbansal/revert-10848
Revert "Revert "libp2p: add QUIC and WebTransport to default listen addresses""
2023-05-30 19:34:44 +02:00
Łukasz Magiera
96fa081ca6
Merge pull request #10913 from filecoin-project/feat/piecereader-perf
feat: piecereader: Allow parallel access
2023-05-30 18:49:41 +02:00
Łukasz Magiera
047898f64d
Merge pull request #10926 from filecoin-project/asr/deflake-miningloop
feat: itest: force PoSt more aggressively around deadline closure
2023-05-30 16:54:03 +02:00
Łukasz Magiera
2636815311 Merge remote-tracking branch 'origin/master' into revert-10852-sbansal/revert-10848 2023-05-30 15:53:31 +02:00
Aayush Rajasekaran
7b921c6761
Merge pull request #10732 from filecoin-project/tests/test-postv11-worker
itests: Test PoSt V1_1 on workers
2023-05-29 16:45:29 -04:00
Aayush
782e09a37d feat: itest: force PoSt more aggressively around deadline closure 2023-05-29 16:32:49 -04:00
Łukasz Magiera
4b5a665422 Merge remote-tracking branch 'origin/master' into feat/piecereader-perf 2023-05-26 13:00:42 +02:00
Łukasz Magiera
cd75ea0fe4 piecereader: Retune to allow parallel ReadAt calls 2023-05-23 12:26:13 +02:00
Jiaying Wang
befc82d3cd
Merge pull request #10861 from filecoin-project/asr/deflake-logs
feat: itests: add logs to blockminer.go failure case
2023-05-12 13:47:03 -04:00
Aayush
760a27dd4b feat: itests: add logs to blockminer.go failure case 2023-05-12 12:17:31 -04:00
Aayush
f39478140a feat: deflake sector_import_simple 2023-05-11 15:08:12 -04:00
Łukasz Magiera
825564a994 itests: Deflake TestNetBlockIPAddr 2023-05-10 22:35:36 +02:00
Łukasz Magiera
b412d8c179
Revert "Revert "libp2p: add QUIC and WebTransport to default listen addresses"" 2023-05-10 22:28:09 +02:00
Shrenuj Bansal
da2e155b72 Revert "libp2p: add QUIC and WebTransport to default listen addresses"
This reverts commit 0a064c1b3d.
2023-05-10 15:58:30 -04:00
Łukasz Magiera
6fd93ed170
fix: sched: Address GET_32G_MAX_CONCURRENT regression (#10850)
* Fix 1.21 regression: GET_32G_MAX_CONCURRENT + mixed prepared/executing leads to stuck scheduler

If you have 12 GET tasks and GET_32G_MAX_CONCURRENT=1, sealing jobs will only show assigned tasks for GET of the miner
and is stuck.
I believe this to be a regression of 1.21 unifying the counters, in the case of GETs where PrepType and TaskType
both being seal/v0/fetch leading to a state where tasks are blocked since already counted towards the limit.

* itests: Repro issue from PR #10633

* make counters int (non-working)

* fix: worker sched: Send taskDone notifs after tasks are done

* itests: Make TestPledgeMaxConcurrentGet actually reproduce the issue

* make the linter happy

---------

Co-authored-by: Steffen Butzer <steffen.butzer@outlook.com>
2023-05-10 15:43:42 -04:00
Marten Seemann
0a064c1b3d libp2p: add QUIC and WebTransport to default listen addresses 2023-05-10 12:07:20 +03:00
Friðrik Ásmundsson
ceb3f1e41e
Validate that FromBlock/ToBlock epoch is indeed a hex value (#10780)
* Validate that FromBlock/ToBlock epoch is indeed a hex value

* Adding tests
2023-05-09 17:17:23 -04:00
Łukasz Magiera
70d2899ead itests: wdpost: Address review 2023-05-09 19:11:15 +02:00
Łukasz Magiera
8fcf59facc itests: Test PoSt V1_1 on workers 2023-05-09 19:02:51 +02:00
Aayush
a4235e2b3d feat: deflake eth_balance_test 2023-05-09 12:35:09 -04:00
Aayush
c6fb2b416a feat: deflake paych_api_test 2023-05-08 14:36:43 -04:00
Aayush Rajasekaran
9ae42929d7
Merge pull request #10829 from filecoin-project/steb/fix-deploy-flake
fix: itest: fix eth deploy test flake
2023-05-05 09:58:39 -04:00
Steven Allen
d7deb9a3eb fix: itest: fix eth deploy test flake
This fixes the flakyness by:

1. Disconnecting the client from the miner before submitting the
message. That way, we force it to get stuck in the message pool.
2. Removing the logic that asks lotus for the "latest" block. We have
other tests that exercise "latest".

fixes #10824
2023-05-04 14:51:25 -07:00
Aayush Rajasekaran
9da534c75c
Merge pull request #10816 from filecoin-project/asr/deflake-ethhistory
feat: deflake TestEthFeeHistory
2023-05-04 13:38:09 -04:00
Aayush
b6b3998864 feat: deflake TestEthFeeHistory 2023-05-04 10:01:40 -04:00
Aayush
6a1b523ba1 feat: make RunClientTest louder when deals fail 2023-05-03 16:49:23 -04:00
Aayush
17c43caacf chore: drop flaky TestBatchDealInput subcase 2023-05-03 11:51:42 -04:00
Steven Allen
bb5ba64cca Revert "Merge pull request #9858 from adlrocha/adlrocha/consistent-bcast"
This reverts commit 8b2208fd9a, reversing
changes made to 2db6b12b78.

Unfortunately, this is rather tricky code. We've found several issues so
far and, while we've fixed a few, there are outstanding issues that
would require complex fixes we don't have time to tackle right now.

Luckily, this code isn't actually needed by the main Filecoin chain
which relies on consensus fault reporting to handle equivocation. So we
can just try again later.
2023-04-27 12:25:30 -07:00
Aayush
e91bb642e7 fix: deflake: use 2 miners for flaky tests 2023-04-27 08:43:03 -04:00
Steven Allen
e351d77ff8
test: eth: deflake multiblock lookup test (#10769)
- Increase epoch times to give the miners a chance to see each other's
blocks.
- Wait longer for a multi-block tipset.
- Reduce the initial wait (we're increasing the block times and I don't
really feel like waiting around).
2023-04-26 15:10:43 -07:00
Aayush
5a061e4677 Merge branch 'releases' into asr/merge-release-into-master 2023-04-23 14:58:50 -04:00
Aayush
5e6e011fe2 fix: FEVM_tests: update recursive limits 2023-04-23 14:21:26 -04:00
zenground0
f3969cdc25 Update splitstore test helpers usage of create miner 2023-04-23 12:14:52 -06:00
Aayush
4ff1d1dc8d feat: itests should run on latest network version 2023-04-23 13:59:14 -04:00
Aayush
df80c4cde2 Merge branch 'releases' into asr/release-1230 2023-04-23 12:10:41 -04:00
Aayush
dc0ce8f11a Update FFI to 1.22.1 2023-04-23 10:42:01 -04:00
Aayush
4430a138b6 feat: test disputing nv20 window posts fails 2023-04-23 10:41:57 -04:00
Aayush
8dec6a1672 fix: test: Simulate call on parents's parents 2023-04-21 16:55:55 -04:00
Aayush
2f48759457 Merge branch 'releases' into release/v1.23.0 2023-04-21 15:12:23 -04:00
zenground0
d45ee8dc54 Test that we reverted activation bug fix 2023-04-21 10:58:27 -06:00
ZenGround0
327760acff
chore: build: Merge/v22 into 21 for 23 (#10702)
* chore: update ffi to increase execution parallelism

* Don't enforce walking receipt tree during compaction

* fix: build: drop drand incentinet servers

* chore: release lotus v1.20.4

* Apply suggestions from code review

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

* feat: Introduce nv19 skeleton

Update to go-state-types v0.11.0-alpha-1

Introduce dummy v11 actor bundles

Make new actors adapters

Add upgrade to Upgrade Schedules

make jen

Update to go-state-types v0.11.0-alpha-2

* feat: vm: switch to the new exec trace format (#10372)

This is now "FVM" native. Changes include:

1. Don't treat "trace" messages like off-chain messages. E.g., don't
include CIDs, versions, etc.
2. Include IPLD codecs where applicable.
3. Remove fields that aren't filled by the FVM (timing, some errors,
code locations, etc.).

* feat: implement FIP-0061

* Address review

* Add and test the FIP-0061 migration

* Update actors bundles to fip/20230406

* Update to go-state-types master

* Update to actors v11.0.0-rc1

* - Update go state types
- Keep current expiration defaults on creation, extension some tests
- Update ffi

* ffi experiment

* Integration nv19 migration

- Open splitstore in migration shed tool
- Update state root version

* Post rebase fixup

* Fix

* gen

* nv19 invariant checking

* Try fixig blockstore so bundle is loaded

* Debug

* Fix

* Make butterfly upgrades happen

* Another ffi experiment

* Fix copy paste error

* Actually schedule migration (#10656)

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Butterfly artifacts

* Set calibration net upgrade height

* Review Response

* Fix state tree version assert

* Quick butterfly upgrade to sanity check (#10660)

* Quick butterfly upgrade to sanity check

* Update butterfly artifacts

* Revert fake fix

* Give butterfly net correct genesis

* Butterfly artifacts

* Give time before upgrade

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* chore:releasepolish v1.22 release (#10666)

* Update butterfly artifacts

* register actors v11

* Update calibration upgrade time

* State inspection shed cmds

* Fix

* make gen

* Fix swallowed errors

* Lint fixup

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* v1.22.0-rc3

* bundle fix

* Feat/expedite nv19 (#10681)

* Update go-state-types

* Modify upgrade schedule and params

* Revert fip 0052

* Update gst

* docsgen

* fast butterfly migration to validate migration

* Correct epoch to match specified date

* Update actors v11

* Update changelog build version

* Update butterfly artifacts

* Fix lotus-miner init to work after upgrade

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* fix:deps:stable ffi for stable release  (#10698)

* Point to stable ffi for stable lotus release

* go mod tidy

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>

* Update CHANGELOG.md

Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>

---------

Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
Co-authored-by: Steven Allen <steven@stebalien.com>
Co-authored-by: jennijuju <jiayingw703@gmail.com>
2023-04-19 18:40:18 -04:00
ZenGround0
ff2258ff7a
Feat/expedite nv19 (#10681)
* Update go-state-types

* Modify upgrade schedule and params

* Revert fip 0052

* Update gst

* docsgen

* fast butterfly migration to validate migration

* Correct epoch to match specified date

* Update actors v11

* Update changelog build version

* Update butterfly artifacts

* Fix lotus-miner init to work after upgrade

---------

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2023-04-19 10:45:22 -04:00
Jorropo
fe42d974a2 chore: all: migrate from go-libipfs to boxo
github.com/ipfs/libipfs/blocks was unmigrated to github.com/ipfs/go-block-format due to compatibility issues with the rest of the IPLD stack.
2023-04-18 17:22:18 +02:00
Aayush
82a3578849 Update actors bundles to fip/20230406 2023-04-06 16:44:20 -04:00
Aayush
74567a16d1 Add and test the FIP-0061 migration 2023-04-06 14:59:47 -04:00
Aayush
0356163a2e Address review 2023-04-06 14:39:57 -04:00
Aayush
6d45c37dc5 feat: implement FIP-0061 2023-04-06 11:48:48 -04:00
Aayush Rajasekaran
8b2208fd9a
Merge pull request #9858 from adlrocha/adlrocha/consistent-bcast
feat:networking:  (Synchronous) Consistent Broadcast for Filecoin EC
2023-03-30 09:32:09 -04:00
Aayush
3636af4cd9 feat: Introduce nv19 skeleton
Update to go-state-types v0.11.0-alpha-1

Introduce dummy v11 actor bundles

Make new actors adapters

Add upgrade to Upgrade Schedules

make jen

Update to go-state-types v0.11.0-alpha-2
2023-03-29 16:13:41 -04:00
Aayush
66fc6dc3e5 refactor: stop using deprecated io/ioutil 2023-03-29 15:40:49 -04:00
Aayush
a724a713da refactor: stop using deprecated io/ioutil 2023-03-29 15:38:46 -04:00
Alfonso de la Rocha
b7c297c537 Merge branch 'master' into adlrocha/consistent-bcast 2023-03-28 16:55:06 +02:00
Łukasz Magiera
6012e65319 debug batch deal test a bit more 2023-03-27 15:34:59 +02:00
Łukasz Magiera
aebe3d4cf7 fix: itests: Don't call t.Error in MineBlocks goroutine 2023-03-27 13:23:29 +02:00