cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
20,641 Commits 24 Branches 342 Tags 314 MiB
5cb25affdd
Commit Graph

924 Commits

Author SHA1 Message Date
Aayush
13f3e0aca4 fix: itest: check for closed connection 2022-09-27 15:34:01 +00:00
Aayush
5d28eea968 chore: deps: update to go-jsonrpc 0.1.8 2022-09-27 15:34:01 +00:00
Aayush
27880ece2b feat: support typed errors over RPC 2022-09-27 15:34:01 +00:00
Aayush
1ae896e73f deflake TestRetryDealsNoFunds: increase wait 10x 2022-09-23 12:43:00 -04:00
Aayush
3e81c56349 address magik supernit 2022-09-23 11:41:53 -04:00
Shrenuj Bansal
7470549199 Address moar comments 2022-09-22 16:27:15 -04:00
Aayush
06262868f5 fix: itest: check for closed connection 2022-09-22 13:58:07 -04:00
Aayush
63a02a6c74 fix: test: flaky TestDeadlineToggling 2022-09-22 13:12:17 -04:00
Aayush
6189932bef chore: deps: update to go-jsonrpc 0.1.8 2022-09-22 11:44:19 -04:00
Aayush
93f94d4ee9 feat: support typed errors over RPC 2022-09-22 11:44:19 -04:00
Aayush Rajasekaran
7a2bc47486
fix: test: flaky TestDeadlineToggling around nulls (#9354) 2022-09-21 21:16:24 +00:00
Geoff Stuart
37be57610c
feat: actors: Integrate datacap actor into lotus (#9348) 
* Integrate datacap actor

* Implement datacap actor in chain/builtin
 2022-09-21 15:51:28 -04:00
Aayush
32670e810c chore: refactor: rename NewestNetworkVersion 2022-09-21 15:48:51 -04:00
Łukasz Magiera
859c2606f0 sealing: Address review 2022-09-19 12:13:06 +02:00
Łukasz Magiera
fec9c0f981 sector import: Fix lint 2022-09-19 12:13:06 +02:00
Łukasz Magiera
061a990eb8 sector import: RemoteSealingDoneEndpoint 2022-09-19 12:13:06 +02:00
Łukasz Magiera
ef834b988c sector import: Fix test build 2022-09-19 12:13:06 +02:00
Łukasz Magiera
c17ab761c5 sector import: Remote Commit2 2022-09-19 12:13:06 +02:00
Łukasz Magiera
142894895d sector import: Implement seed/ticket/commr/commd checks 2022-09-19 12:13:06 +02:00
Łukasz Magiera
b2dfaae68c sector import: Test remote commit1 retry 2022-09-19 12:13:06 +02:00
Łukasz Magiera
9f03569cd0 sector import: Remote Commit1 2022-09-19 12:13:06 +02:00
Łukasz Magiera
830c2ffdf5 sector import: FinalizeInto, send finalized cacde in itest 2022-09-19 12:13:06 +02:00
Łukasz Magiera
3086e8aa01 sector import: Seal with the correct proof type in saas itest 2022-09-19 12:13:06 +02:00
Łukasz Magiera
9c6d531ae7 sector import: Plumb sector download task canfig 2022-09-19 12:13:06 +02:00
Łukasz Magiera
fef7232c6b sector import: Initial SectorReceive itest 2022-09-19 12:13:06 +02:00
Łukasz Magiera
5c485c3375 sealing: Drop redundant Piece type 2022-09-19 12:13:05 +02:00
Shrenuj Bansal
4be8861f41 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-09-13 17:41:06 -04:00
Shrenuj Bansal
3441224b2f WIP: rest of the stuff 2022-09-13 17:30:13 -04:00
Shrenuj Bansal
a50a21b076 cleanup 2022-09-13 16:52:47 -04:00
Shrenuj Bansal
4c4b10c617 Ignore uuid check for messages with uuid not set 2022-09-13 16:33:55 -04:00
Shrenuj Bansal
4171be0b98 Few more changes 2022-09-12 16:37:24 -04:00
Shrenuj Bansal
a1f2fdb706 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-09-12 16:23:24 -04:00
Shrenuj Bansal
8f1b1bb1ff WIP: Raft consensus for lotus nodes in a cluster 2022-09-12 16:10:15 -04:00
Aayush
19c3232d2f more fixes 2022-09-10 12:37:03 -04:00
Aayush
0efca4d266 introduce v9 actors and nv17 2022-09-09 21:40:15 -04:00
LexLuthr
67d4f905e6
feat: proving: Introduce manual sector fault recovery (#9144) 
* rebase

* fix lint error

* fix errors

* add itest

* make gen after rebase

* apply suggestion from review

* make gen after latest rebase

* gen after rebase

* fix waitgroup

* change as per review

* refactor Recovery Batches

* fix CLI

* gen after rebase

* close the channel
 2022-09-06 21:17:30 +05:30
Łukasz Magiera
08b22edd89 fix make gen 2022-08-29 16:25:30 +02:00
Jiaying Wang
1b5a1f692a
Merge pull request #9206 from filecoin-project/asr/verifreg-tuple 
fix: verifreg: serialize RmDcProposalID as int, not tuple
 2022-08-26 16:38:34 -04:00
Łukasz Magiera
45d1bd61ce
Merge pull request #9183 from filecoin-project/feat/sectornum-mgmt 
feat: sealing: Use bitfields to manage sector numbers
 2022-08-26 10:59:24 -04:00
Łukasz Magiera
2086b219d2 Don't use go-libp2p-core 2022-08-25 14:20:41 -04:00
Łukasz Magiera
335f5df3ec numassigner: SectorNumReserveCount 2022-08-24 11:25:37 -04:00
Aayush
9812792d46 fix: verifreg: serialize RmDcProposalID as int, not tuple 2022-08-23 16:58:10 -04:00
Łukasz Magiera
129eba2b5d sealing: numassign: itests/bug fixes 2022-08-22 16:58:36 -04:00
Łukasz Magiera
ca72590e49 sealing: Use bitfields to manage sector numbers 2022-08-22 16:55:08 -04:00
Shrenuj Bansal
f5ba7b958e Add stateWaitMsg before checking last balance 2022-08-17 14:48:44 -04:00
Shrenuj Bansal
cd5dd2727c fix lint 2022-08-17 14:14:07 -04:00
Shrenuj Bansal
087a838db5 make gen for tests 2022-08-17 14:02:53 -04:00
Shrenuj Bansal
99040a89c6 Add test for dedup mpool messages 2022-08-17 12:38:23 -04:00
Shrenuj Bansal
b7227d3a6b run make gen 2022-08-16 15:56:32 -04:00
Shrenuj Bansal
fa4a479b97 Add uuid to mpool message sent to chain node from miner 2022-08-16 15:39:06 -04:00
ZenGround0
8b7be6d47e
feat:chain:splitstore auto prune (#9123) 
Auto Prune

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
 2022-08-08 16:06:32 -04:00
ZenGround0
0c91b0dc10
feat:chain:splitstore chain prune (#9056) 
* Splitstore chain prune
* Protect on reification for simpler logic and sound cold compact protect
* Recovery from checkpoint during chain prune
* Splitstore (discard and universal mode) running in itests
* Add pause and restart functions to itest block miner
* Add config options to itest full nodes
* Add FsRepo support for itest full ndoes

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
 2022-08-05 16:34:16 -04:00
Łukasz Magiera
cb2b4fa4aa Merge remote-tracking branch 'origin/master' into feat/storage-redeclare 2022-08-05 21:26:15 +02:00
Łukasz Magiera
11e4914531 Merge branch 'feat/SchedRemoveRequest' of github.com:LexLuthr/lotus into LexLuthr-feat/SchedRemoveRequest 2022-08-05 10:40:45 +02:00
Łukasz Magiera
819314de46 Merge remote-tracking branch 'origin/master' into feat/storage-redeclare 2022-08-05 09:01:30 +02:00
LexLuthr
ffc10d80ec fix lint error after merge 2022-08-04 02:35:22 +05:30
LexLuthr
a59d73d64a
Merge branch 'master' into feat/SchedRemoveRequest 2022-08-04 02:24:14 +05:30
LexLuthr
23bb80ab2c fix lint errors 2022-08-04 02:00:24 +05:30
LexLuthr
c962fe9011 fix the itest parsing errors 2022-08-04 02:00:24 +05:30
LexLuthr
4e2536e0ef switch pledging func 2022-08-04 02:00:24 +05:30
LexLuthr
b15f1e5b93 fix itest 2022-08-04 02:00:24 +05:30
LexLuthr
0fce643a10 fix sectorstatus in itest 2022-08-04 02:00:24 +05:30
LexLuthr
fad79f3218 respond to caller. Add itest 2022-08-04 02:00:24 +05:30
Łukasz Magiera
ecddd08048 itests: Test custom worker names 2022-08-03 13:05:30 +02:00
Łukasz Magiera
92d27aa6d2 Merge remote-tracking branch 'origin/master' into feat/storage-redeclare 2022-08-03 00:56:23 +02:00
Aayush
784926b2b4 add a test for Post max sectors recovery 2022-08-02 15:28:43 -04:00
Łukasz Magiera
8294e03d4e itests: More fixes after fixing sector metadata import 2022-08-02 17:35:26 +02:00
Łukasz Magiera
6d2990314f storage: Fix detaching shared paths 2022-08-02 15:46:35 +02:00
Łukasz Magiera
d1173265b3 itests: Helper returning non-genesis sector list 2022-08-02 15:29:38 +02:00
Łukasz Magiera
1ee82dd604 fix lint 2022-08-01 15:58:06 +02:00
Łukasz Magiera
3cd8d9eb06 itests: Worker tests for storage detach/redeclare 2022-08-01 15:58:06 +02:00
Łukasz Magiera
3309e10b0d itests: Test storage detach/redeclare 2022-08-01 15:58:06 +02:00
Łukasz Magiera
0ccaa00969 itests ensemble: import presealed sector metadata 2022-08-01 15:58:06 +02:00
Aayush Rajasekaran
3c191d209d fix: test: deflake TestQuotePriceForUnsealedRetrieval 2022-07-25 11:17:07 -04:00
Aayush Rajasekaran
7972bd8446
Merge pull request #9036 from filecoin-project/asr/tracing 
feat: FVM: always enable tracing for user-triggered executions
 2022-07-15 15:09:58 -04:00
Łukasz Magiera
9eb8f4ee9d
Merge pull request #9013 from filecoin-project/feat/path-type-filters 
feat: storage: Path type filters
 2022-07-15 13:08:43 +02:00
Aayush
474a50252b FVM: always enable tracing for user-triggered executions 2022-07-14 15:41:26 -04:00
Aayush Rajasekaran
ca713c57e7
Merge pull request #8084 from filecoin-project/itest_net_tests 
test: net: net and conngater tests
 2022-07-13 15:59:17 -04:00
Łukasz Magiera
ac2ea039b5 path index: Raise alerts on bad filter config 2022-07-12 21:45:12 +02:00
Łukasz Magiera
91e474e72e sealer: Pass correct args to SubAllowed in selectors 2022-07-12 21:45:12 +02:00
Łukasz Magiera
59d069dc4d itests: Tests for storage path filters 2022-07-12 21:45:12 +02:00
Łukasz Magiera
0ca1cc2bcd itests: Test lotus-miner with no store/seal paths 2022-07-12 21:45:12 +02:00
Steven Allen
9b75390b24
address cr 2022-07-11 15:57:27 +00:00
Aayush
9662c1463d Fixup itest net tests 2022-07-10 11:09:54 -04:00
TheMenko
49f074144d change variable to better descriptive name and check addr count 2022-07-10 10:17:40 -04:00
TheMenko
23728626e2 add stm annotations 2022-07-10 10:17:40 -04:00
TheMenko
0d437f732d add tests for netstat and netlimit, fix error checking 2022-07-10 10:17:40 -04:00
TheMenko
1a0dd76ed1 testing network functions from node net and conngater 2022-07-10 10:17:40 -04:00
Steven Allen
d192b821a9
fix: gas: estimate gas with a zero base-fee 
Otherwise, an account will need funds to estimate the max possible gas a
message could take (which is usually the block gas limit).

This does mean gas estimation no longer checks if the sending account
has enough funds to cover the message cost, but MpoolPush will now do
this.
 2022-07-08 09:47:45 -07:00
Łukasz Magiera
9a97d83889 wdpost: itests for new config fields 2022-07-04 16:15:23 +02:00
Aayush
0c34699f23 unbreak itests 2022-06-29 14:55:35 -04:00
Aayush
f9cf25f966 address review 2022-06-29 11:57:46 -04:00
Aayush
a52d584d0c an attempt at cleanup 2022-06-28 19:24:47 -04:00
Aayush
7554991085 Add a reentrant multisig test 2022-06-20 17:22:45 -04:00
Łukasz Magiera
db676acdf8 storage: Cleanup interfaces a bit 2022-06-17 13:52:19 +02:00
Łukasz Magiera
58416d0881 storage: Don't depend on specs-storage, move to storiface 2022-06-17 13:36:33 +02:00
Łukasz Magiera
05cdeb80c3 chore: remove redundant import prefixes 2022-06-15 12:06:22 +02:00
Łukasz Magiera
39cede946c chore: Fix more imports with '2' suffix 2022-06-14 21:37:54 +02:00
Łukasz Magiera
28099a3905 storage: Move storage/sealer/stores to storage/paths 2022-06-14 20:25:52 +02:00
Łukasz Magiera
a9600b8a6f storage: Move extern/sector-storage to storage/sealer 2022-06-14 20:03:38 +02:00
Łukasz Magiera
98a48a47f8 storage: Move extern/storage-sealing to storage/pipeline 2022-06-14 19:41:59 +02:00
Łukasz Magiera
717bda63b1 storage: Move control address logic to storage/ctladdr 2022-06-14 19:32:29 +02:00
Łukasz Magiera
dff1bf2868 storage: Move window-post logic into a separate package 2022-06-14 19:27:09 +02:00
Łukasz Magiera
12c91bb41d storage: Move mock PreCommit helper to the mock package 2022-06-14 19:23:14 +02:00
Łukasz Magiera
6afb43afbb wallet: Move key.go into separate pkg to avoid import loops 2022-06-14 19:21:40 +02:00
Łukasz Magiera
e65fae28de chore: fix imports 2022-06-14 17:00:51 +02:00
Steven Allen
30981d0fdd
feat: refactor: actor bundling system (#8838) 
1. Include the builtin-actors in the lotus source tree.
2. Embed the bundle on build instead of downloading at runtime.
3. Avoid reading the bundle whenever possible by including bundle
   metadata (the bundle CID, the actor CIDs, etc.).
4. Remove everything related to dependency injection.
    1. We're no longer downloading the bundle, so doing anything ahead
       of time doesn't really help.
    2. We register the manifests on init because, unfortunately, they're
       global.
    3. We explicitly load the current actors bundle in the genesis
       state-tree method.
    4. For testing, we just change the in-use bundle with a bit of a
       hack. It's not great, but using dependency injection doesn't make
       any sense either because, again, the manifest information is
       global.
    5. Remove the bundle.toml file. Bundles may be overridden by
       specifying an override path in the parameters file, or an
       environment variable.

fixes #8701
 2022-06-13 10:15:00 -07:00
Geoff Stuart
b7010c9e60 Implement function to migrate actors with only code changes 2022-06-10 15:52:32 -04:00
Łukasz Magiera
ae42ed0bb1 Merge remote-tracking branch 'origin/master' into rate-limit-gateway 2022-06-10 13:01:20 +02:00
Aayush
b28c11a57d Merge branch 'feat/nv16' 2022-06-03 14:01:49 -04:00
Cory Schwartz
e645e8aa7c Merge branch 'master' into rate-limit-gateway 2022-06-01 00:58:23 -07:00
Jennifer Wang
2e59d0129d Merge branch 'release/v1.15.3' into jen/mergev1153to16 2022-05-31 16:33:18 -04:00
Łukasz Magiera
56bde5adf9
Merge pull request #8692 from filecoin-project/feat/health-endpoints 
feat: networking: add healthz and livez endpoints
 2022-05-30 15:42:06 +02:00
Geoff Stuart
775a94a239 Test deal label with invalid utf8 2022-05-27 16:43:06 -04:00
Łukasz Magiera
7836e20801
Merge pull request #8710 from filecoin-project/feat/stor-fin-move-selector 
feat: sched: Finalize* move selectors
 2022-05-26 21:20:48 +02:00
Łukasz Magiera
cfff877ec7
Merge pull request #8700 from filecoin-project/feat/multi-sched 
feat: sched: Add scheduler interfaces, configurable assigner
 2022-05-26 21:20:30 +02:00
Aayush
2b847a98ae fix wdpost_dispute tests 2022-05-24 13:58:30 -04:00
Łukasz Magiera
7054475c00 ffiwrapper: Handle null padding in DataCid automatically 2022-05-24 15:59:11 +02:00
Łukasz Magiera
8c081e271a itests: test TestWorkerPledgeLocalFin 2022-05-24 01:13:30 +02:00
Cory Schwartz
2b0d0ce224 fix itests 2022-05-23 14:59:57 -07:00
Łukasz Magiera
3de34ea3c0 Fix TestWorkerPledgeSpread in CI 2022-05-23 23:28:31 +02:00
Łukasz Magiera
58574554c1 itests: Test spread sched assigner 2022-05-23 22:31:06 +02:00
Cory Schwartz
44264dccba fix integration test 2022-05-20 03:48:10 -07:00
Cory Schwartz
c9d3652357 Merge branch 'master' into rate-limit-gateway 2022-05-19 14:25:36 -07:00
Aayush
b2b20e469f Fix tests 2022-05-17 15:21:27 -04:00
Aayush
8cca9b1970 Use new go-state-types accessors 2022-05-17 15:21:27 -04:00
Geoff Stuart
23378f8710 Fix flaky test 2022-05-04 15:48:44 -04:00
Cory Schwartz
35e114a3ea Merge branch 'master' into rate-limit-gateway 2022-05-02 08:29:38 -07:00
Łukasz Magiera
43436f7bc8 rpcenc: Fix double read at eof 2022-04-28 11:50:51 +02:00
Łukasz Magiera
779e923854 sealing: ComputeDataCid review 2022-04-27 20:44:02 +02:00
vyzo
53ca3b64e1 normalize itest PrecommitChallengeDelay to 10, matching the test bundles 2022-04-27 17:57:04 +03:00
vyzo
31d193702b don't parse error messages, just check exit codes 2022-04-27 17:57:04 +03:00
Łukasz Magiera
d7c52f8adf sealing: ComputeDataCid miner API 2022-04-26 21:37:48 +02:00
Cory Schwartz
9fd3f3f73a fix itests 2022-04-20 20:51:15 -07:00
vyzo
5ceaeafa6e fix TODO for proof type, now that we have the testing bundle 2022-04-20 12:47:06 +03:00
Geoff Stuart
49d6c07280 Implemented StateLoockupRobustAddress 2022-04-14 17:41:18 -04:00
vyzo
c5e3ca3855 fix more rebase issues 2022-04-14 19:52:58 +03:00
vyzo
dec2b312af fix lint 2022-04-14 19:33:24 +03:00
vyzo
73e7c42b7e mark TODO for fixing policy stuffs in test 2022-04-14 19:31:34 +03:00
vyzo
2c9ea8c46b fix test 2022-04-14 19:31:34 +03:00
vyzo
71913521f9 fix itest migrations 2022-04-14 19:30:47 +03:00
Aayush
3370847f3e Implement FIP-0031 2022-04-14 19:28:16 +03:00
jennijuju
137f494619 tidy and gen 2022-04-12 13:53:03 +02:00
jennijuju
0dcad9cc18 Merge branch 'releases' into jen/mergeback 2022-04-12 01:25:02 +02:00
dirkmc
c34ade9ffb
Update itests/kit/rpc.go 
Co-authored-by: Anton Evangelatov <anton.evangelatov@gmail.com>
 2022-04-07 13:32:00 +02:00
Dirk McCormick
5f0489ace0 fix: testkit: give up on waiting for the RPC server to shutdown after 1 second 2022-04-07 12:22:36 +02:00
Łukasz Magiera
ebd34f1884 feat: storage: Parallel proving checks 2022-03-29 11:45:08 -04:00
Łukasz Magiera
54cb55a7ae feat: miner: API/CLI to compute window-post 2022-03-28 17:06:51 -04:00
Łukasz Magiera
82343a06e7 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-25 16:49:46 -04:00
Łukasz Magiera
3f152ae2bf workers: cmd/lotus-seal-worker->cmd/lotus-worker 2022-03-24 18:37:01 -04:00
Łukasz Magiera
92e4caa151 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-21 12:09:47 +01:00
Łukasz Magiera
e767981aaf itests: Fix ccupgrade post test 2022-03-21 12:07:00 +01:00
Łukasz Magiera
0536eb898d itests: Share WaitForSectorActive correctly 2022-03-21 12:07:00 +01:00
Łukasz Magiera
433e9c351d config: Fix doc on PreferNewSectorsForDeals; gofmt 2022-03-21 12:07:00 +01:00
Łukasz Magiera
49a6d7ebcc itests: Breakup ccupgrade_test.go 2022-03-21 12:07:00 +01:00
Łukasz Magiera
4c25358a08 itests: Test Sealing.MakeCCSectorsAvailable 2022-03-21 12:07:00 +01:00
Łukasz Magiera
ecd1ab4b42 itests: Test Sealing.PreferNewSectorsForDeals 2022-03-21 12:07:00 +01:00
Łukasz Magiera
36f1dd7bb3 itests: Fix flaky TestWindowPostWorkerSkipBadSector 2022-03-18 20:51:47 +01:00
Łukasz Magiera
7b81369c8c wdpost: Don't attempt to snark with skipped sectors 2022-03-18 19:45:25 +01:00
Łukasz Magiera
32818c528f wdpost: Test repro broken challenge skipping 2022-03-18 12:32:16 +01:00
Anton Evangelatov
0d6493ec3f resolve merge conflicts with master 2022-03-18 12:28:22 +01:00
Łukasz Magiera
aa124cf414 fix lint issues 2022-03-18 11:38:27 +01:00
Łukasz Magiera
6ddbe41376 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-18 10:54:44 +01:00
Łukasz Magiera
42b2adddee fix: itests: Don't hang on exit in MineBlocksMustPost 2022-03-18 10:42:21 +01:00
Łukasz Magiera
5e14892ef8 sealing: fix lint issues 2022-03-16 22:30:57 +01:00
Łukasz Magiera
62a393bf12 sealing: Address snap queue review 2022-03-16 22:30:09 +01:00
Łukasz Magiera
1158a928ec itests: Test reverting Available sectors to Proving 2022-03-16 22:29:40 +01:00
Łukasz Magiera
a9d757f3a0 itests: TestTooManyMarkedForUpgrade is not applicable anymore 2022-03-16 22:29:40 +01:00
Łukasz Magiera
2cef55a4f6 feat: sealing: Sector upgrade queue 2022-03-16 22:29:37 +01:00
Łukasz Magiera
a3bdd29615
Merge pull request #8330 from filecoin-project/feat/snap-queue 
feat: sealing: Sector upgrade queue
 2022-03-16 22:24:45 +01:00
Łukasz Magiera
3b2a1ea41d sealing: fix lint issues 2022-03-16 20:28:15 +01:00
Łukasz Magiera
724d80cf02 sealing: Address snap queue review 2022-03-16 20:04:48 +01:00
Łukasz Magiera
799d3ebe5e itests: Test reverting Available sectors to Proving 2022-03-16 19:05:53 +01:00
Darko Brdareski
79453663b3
feat: Add additional test annotations (#8272) 
* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate repub and selection tests

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

* Fix markset annotations

* doc: test annotations for the markets dagstore wrapper

* Annotate miner_api test in dagstore

* Annotate more test files

* Remove bad annotations from fsrepo

* Annotate wdpost system

* Remove bad annotations

* Renamce "conformance" to "chaos_actor" tests

* doc: stm annotations for blockheader & election proof tests

* Annotate remaining "A" tests

* annotate: stm for error_test

* memrepo_test.go

* Annotate "b" file tests

* message_test.go

* doc: stm annotate for fsrepo_test

* Annotate "c" file tests

* Annotate "D" test files

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate u,v,wl tests

* doc: stm annotations for various test files

* Annotate "T" test files

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add mempool unit tests

* Add two more memPool Add tests

* Update submodules

* Add check function tests

* Add stm annotations, refactor test helper

* Annotate api,proxy_util,blockstore_badger, policy tests

* Annotate splitstore: bsbadger / markset

solving merge conflicts

* Annotate splitstore feature

* Annotate union/timed blockstore tests

* Annotate openrpc, diff_adt tests

* Annotate error,drand,events tests

* Annotate predicates_test

* Fix annotations

* Annotate tscache, gen tests

* Annotate fundmanager test

* Annotate statetree_test

* Annotate forks_test

* Annotate searchwait_test.go

* Fix duplicated @@ symbols

* Annotate chain stmgr/store tests

* Annotate more (types) tests

* More tests annotated

* Annotate conformance chaos actor tests

* Annotate more integration tests

* Annotate journal system tests

* Annotate more tests.

* Annotate gas,head buffer behaviors

solve merge conflict

* Fix markset annotations

* Annotate miner_api test in dagstore

* Annotate more test files

* doc: test annotations for the markets dagstore wrapper

* Annotate wdpost system

* Renamce "conformance" to "chaos_actor" tests

* Annotate remaining "A" tests

* doc: stm annotations for blockheader & election proof tests

* annotate: stm for error_test

* Annotate "b" file tests

* memrepo_test.go

* Annotate "c" file tests

* message_test.go

* Annotate "D" test files

* doc: stm annotate for fsrepo_test

* Annotate u,v,wl tests

* message_test.go

* doc: stm annotate for chain, node/config & client

* docs: stm annotate node_test

* Annotate "T" test files

* doc: stm annotations for various test files

* Add mempool unit tests

solve merge conflict

* doc: stm annotate for proxy_util_test & policy_test

* doc: stm annotate for various tests

* doc: final few stm annotations

* Add two more memPool Add tests

* Update submodules

* Add check function tests

solve conflict

* Add stm annotations, refactor test helper

solve merge conflict

* Change CLI test kinds to "unit"

* Fix double merged test

* Fix ccupgrade_test merge

* Fix lint issues

* Add stm annotation to types_Test

* Test vectors submodule

* Add file annotation to burn_test

Co-authored-by: Nikola Divic <divicnikola@gmail.com>
Co-authored-by: TheMenko <themenkoprojects@gmail.com>
 2022-03-16 18:37:34 +01:00
Łukasz Magiera
8600ae2ac4 itests: TestTooManyMarkedForUpgrade is not applicable anymore 2022-03-16 17:48:02 +01:00
Łukasz Magiera
aef2ec513f feat: sealing: Sector upgrade queue 2022-03-16 17:33:05 +01:00
Łukasz Magiera
135aef78d7 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-11 17:04:58 +01:00
Łukasz Magiera
ad66ad4f7d fix: sealing fsm: Handle inputLk correctly 2022-03-11 13:34:14 +01:00
Łukasz Magiera
231893a794 feat: shed: ItestD 2022-03-10 19:39:10 +01:00
Łukasz Magiera
046a9f8af0 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-09 16:27:03 +01:00
Anton Evangelatov
b9fe61f9da Merge branch 'master' into nonsense/refactor-nodetype 2022-03-07 11:51:28 +01:00
Anton Evangelatov
acbfc879aa config: extract DealmakingConfig and Sealing config from Lotus for Boost (#8172) 
* refactor sealingCfg and dealmakingCfg to be able to abstract Config

* fix error messages
 2022-03-03 15:37:23 +01:00
Anton Evangelatov
f1f878a000 unexport repo types; use a global var for every repo type 2022-03-03 15:37:23 +01:00
Anton Evangelatov
e3edab66e3 refactor RepoType 2022-03-03 15:37:23 +01:00
Masih H. Derkani
aacc246ba3
Merge branch 'master' into feat/cid-to-piece-idx 2022-03-02 14:06:30 +00:00
Masih H. Derkani
769b0f30ef
Address review comments 
- Add comment to clarify the reason for loop in testkit
- Trim common prefix in state printed in CLI commands for better
  readability
- Upgrade to a tagged release of `go-fil-markets` that includes indexing
  work; see: https://github.com/filecoin-project/go-fil-markets/pull/673
- Fix typo in CLI usage.
- Add comments to note that it is safe to use fx `OnStart` context when
  starting the provider engine.
- Fix string concatenation in error message formatting.
 2022-03-02 11:04:50 +00:00
zenground0
77a954c7c3 Merge branch 'master' into feat/cid-to-piece-idx 2022-02-28 12:57:11 -07:00
Łukasz Magiera
a0cb6094c6 Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse 2022-02-22 10:55:03 +01:00
Elijah Seed-Arita
e2a93a3feb add itests ensemble mocknet getter 2022-02-18 11:09:56 -10:00
Łukasz Magiera
681b90768a Merge remote-tracking branch 'origin/master' into feat/paych-avail-reuse 2022-02-16 10:06:36 +01:00
Łukasz Magiera
36a1934845 paychmgr: Fix tests after api changes 2022-02-14 20:16:30 +01:00
zenground0
977351f419 Fix from Magik to remove hanging behavior 2022-02-14 14:00:41 -05:00
Łukasz Magiera
10af768c60 Merge commit 'origin/release/v1.15.0~2' into feat/paych-avail-reuse 2022-02-14 19:27:12 +01:00
zenground0
2c06eb76d6 Improve MineBlocksMustPost use it in PaychAPI itest 2022-02-14 09:47:18 -05:00
Nikola Divic
34387326d1 test: fix flaky message pool integration tests 
Using the same pattern described in my previous commit.
I also added the CircuitBreaker to the itests kit as it may be useful
for other integration tests when debugging flakyness caused by timeouts.
 2022-02-12 19:52:51 +01:00
Nikola Divic
aca2a0fd1b test: fix flake in TestMemPoolBatchPushUntrusted integration test 
The flake was caused by improper waiting for certain chain operations
to finish:

- We didn't wait for messages to be registered as pushed
- We improperly waited for a fixed time (10 seconds) for messages to be
mined, which in the best case would wait longer than necessary and in the
worst case would cause the test to break.

What I did:
- fixed by waiting in a loop for "just enough time". This fixed the flake
and made the test run faster, on average, because we don't have unnecessary
waiting.
- I added a "circuit-breaker" where the wait loop will timeout after 10 seconds.
 2022-02-12 17:48:45 +01:00
Nikola Divic
0e8a709f92 Merge branch 'master' of https://github.com/filecoin-project/lotus into bloxico/mempool_tests 2022-02-12 17:11:24 +01:00
Jennifer Wang
de1f4792cd Merge remote-tracking branch 'origin/master' into jen/v15 2022-02-09 19:27:34 -05:00
Jennifer Wang
6ed97f0d04 Merge branch 'release/v1.14.0' into jen/v15 2022-02-09 03:17:38 -05:00
Łukasz Magiera
4ad97f6a36 make gen 2022-02-08 17:22:41 +01:00
Łukasz Magiera
09cfad9d71 Add FinalizeReplicaUpdate into some more places 2022-02-08 17:22:41 +01:00
Łukasz Magiera
7fa84518c8 itests: Print API info env vars in ThroughRPC tests 2022-02-08 17:22:41 +01:00
Łukasz Magiera
e271bae5ec try ClearCache for update cache 2022-02-08 17:22:41 +01:00
Aarsh Shah
399a1afd94 Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx 2022-02-08 12:37:57 +04:00
Anton Evangelatov
595ad44ee7 refactor indexprovider libp2p host connection to fullnode with meshcreator 2022-02-03 15:44:18 +01:00
Anton Evangelatov
20fc5ffa38 fixing tests 2022-02-03 14:43:20 +01:00
Darko Brdareski
008fbbd652 Add unit and integration tests for mempool 2022-02-02 17:08:50 +01:00
Łukasz Magiera
98f4d37f1a Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-01-31 19:50:52 +00:00
Darko Brdareski
6b8f526df3 Fix merge 2022-01-27 11:06:04 +01:00
Darko Brdareski
e51ce5c508 Merge remote-tracking branch 'upstream/master' into bloxico/system-test-matrix 2022-01-27 10:57:56 +01:00
Łukasz Magiera
55c9271695 post workers: Test skips 2022-01-21 13:33:47 +01:00
Łukasz Magiera
8b19b84140 paych: option to force off-chain get 2022-01-20 18:19:26 +01:00
Łukasz Magiera
533349cc0d paych: Fix cli tests 2022-01-20 18:17:18 +01:00
Łukasz Magiera
2e76375e8a Fix paych itests 2022-01-20 18:16:00 +01:00
Łukasz Magiera
33dfcf2663 stores: Fix single post proof reading with updated sectors 2022-01-20 15:27:42 +01:00
Aarsh Shah
9ee940098d Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx 2022-01-20 16:21:48 +04:00
Łukasz Magiera
e476cf7968 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-01-20 13:15:48 +01:00
Łukasz Magiera
b115e2879a don't defer in a loop 2022-01-20 11:19:57 +01:00
vyzo
c3fb68545a fix mocknet.New invocation 2022-01-20 11:36:11 +02:00
Łukasz Magiera
fa09b9afb1 post workers: Share resource accounting code 2022-01-18 16:02:37 +01:00
Łukasz Magiera
c79f2b9d38 post workers: Fix lint 2022-01-18 12:11:59 +01:00
Łukasz Magiera
b38141601c Untangle ffi from api 2022-01-18 11:57:04 +01:00
Łukasz Magiera
03fa955e9e Cleanup imports 2022-01-18 11:37:24 +01:00
Łukasz Magiera
0c062f9c08 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-01-18 10:49:42 +01:00
zenground0
a9a523d8c0 Fix TooManyMarkedForUpgrade 2022-01-14 17:14:32 -05:00
zenground0
c6277922ac Deflake more practically 2022-01-14 17:14:32 -05:00
zenground0
a82ac00d9a Deflake snap deals integration test 2022-01-14 17:14:32 -05:00
zenground0
d6aa17e21f Snap Deals Integration 
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
 2022-01-14 17:14:32 -05:00
Jennifer Wang
e24b454a1f Bump the master version to v1.13.3-dev 2022-01-14 17:14:32 -05:00
c r
7efaa708a6 reorder transfer checks so as to ensure sending more money than you have to yourself fails with an error (fixing issue 7596) 
PR #7637, also adds tests to make sure behavior is correct across versions.
 2022-01-14 17:14:32 -05:00
Aayush Rajasekaran
3a7c364562 Address review 2022-01-14 17:14:32 -05:00
Aayush Rajasekaran
393d8541e2 Update deps 2022-01-14 17:14:32 -05:00
Aayush Rajasekaran
1ef780d96f Plug in the FFI call 2022-01-14 17:14:32 -05:00
Aayush Rajasekaran
a8cb027c08 Integrate v7 actors 2022-01-14 17:14:32 -05:00
Łukasz Magiera
4a874eff70 post workers: Cleanup, tests 2022-01-14 14:17:52 +01:00
Aarsh Shah
b9d3585f4c get tests working 2022-01-12 16:55:45 +04:00
Aarsh Shah
3a52f05692 green ci 2022-01-12 15:53:15 +04:00
Aarsh Shah
edfc2c215c Merge remote-tracking branch 'origin/main' into feat/cid-to-piece-idx 2022-01-12 12:48:54 +04:00
zenground0
c309686679 Fix TooManyMarkedForUpgrade 2022-01-10 15:39:38 +05:30
zenground0
3dc3f55ecb Deflake more practically 2022-01-10 15:39:38 +05:30
zenground0
f59bfd65da Deflake snap deals integration test 2022-01-10 15:39:38 +05:30
zenground0
33f2d24f54 Snap Deals Integration 
- FSM handles the actual cc upgrade process including error states
- PoSting (winning and window) works over upgraded and upgrading sectors
- Integration test and changes to itest framework to reduce flakes
- Update CLI to handle new upgrade
- Update dependencies
 2022-01-10 15:39:38 +05:30
Darko Brdareski
dda1a42a2a Merge branch 'bloxico/system-test-matrix' of https://github.com/filecoin-project/lotus into merge_lotus 2021-12-20 15:48:16 +01:00
Dirk McCormick
bb56e97177 fix: rename query-retrieval-ask to retrieval-ask 2021-12-17 18:01:12 +01:00
Dirk McCormick
b1734f84b3 feat: retrieval ask CLI command 2021-12-17 15:55:12 +01:00
Łukasz Magiera
a4728d3c72
Merge pull request #7578 from filecoin-project/rvagg/SimultaneousTransfersForStoragePerClient 
feat(graphsync): allow setting of per-peer incoming requests for miners
 2021-12-17 14:27:15 +01:00
Łukasz Magiera
bc384c01e3 Merge remote-tracking branch 'origin/master' into deps/update-ctx-dsbs 2021-12-17 13:01:22 +01:00
Rod Vagg
9e7d9affbe
feat(graphsync): allow setting of per-peer incoming requests for miners 2021-12-17 15:04:16 +11:00
Darko Brdareski
3e32aa896c Annotate client storage deals feature 2021-12-16 14:36:02 +01:00
Darko Brdareski
2f1f35cc71 Annotate storage miner features 2021-12-15 15:30:42 +01:00
vyzo
fc7d4a13b0 fix itests 2021-12-14 16:06:59 +02:00
Darko Brdareski
a64f2421d2 Annotate 'incoming' subsystem 2021-12-14 11:33:33 +01:00
Aayush Rajasekaran
80d5e52923 Merge branch 'master' into next 2021-12-13 13:24:28 -05:00
Darko Brdareski
0addca1070 Fix bad annotations 2021-12-13 13:41:04 +01:00
Darko Brdareski
0169d0dafd Annotate state feature tests 2021-12-10 16:08:25 +01:00
Darko Brdareski
95f86f9de0 Annotate feature syncer 2021-12-10 11:33:29 +01:00
zenground0
a5be80828a RemoveData and Decode 
- Unsealing replica update with sector key works and tested
- Sector key generation added and tested
 2021-12-03 15:21:06 -05:00
c r
e16e9ad343
reorder transfer checks so as to ensure sending more money than you have to yourself fails with an error (fixing issue 7596) 
PR #7637, also adds tests to make sure behavior is correct across versions.
 2021-11-29 17:26:47 -05:00
Łukasz Magiera
5b5e6b9e44 retrieval: DagSpec.MatchPath -> ExportMerkleProof 2021-11-29 21:14:00 +01:00
Łukasz Magiera
227188e908 retrieval: Test non-matching path traversal 2021-11-29 20:52:55 +01:00
Łukasz Magiera
61791b90ea retrieval: Only output matching nodes, MatchPath dagspec 2021-11-29 20:40:55 +01:00
Peter Rabbitson
407bf4961c Modify preexisting test to reflect the simple-path car export behavior 
I am open to having a flag or something, as long as the behavior reflected in
deals_partial_retrieval_dm-level_test.go can be preserved for v0, and opt-in for v1
 2021-11-24 15:31:19 +01:00
Peter Rabbitson
46937a1b9f Add tests cementing the DM-level behavior on simple paths 2021-11-24 15:31:15 +01:00
Łukasz Magiera
597b72e286 retrieval: Fix lint, cli docsgen 2021-11-22 12:46:29 +01:00
Łukasz Magiera
b83a9b902a gofmt 2021-11-22 12:46:17 +01:00
Łukasz Magiera
b26906963b retrieval: Support multi-root export 2021-11-22 12:46:17 +01:00
Łukasz Magiera
d0503d409f fix TestPartialRetrieval 2021-11-22 12:46:04 +01:00
Łukasz Magiera
b868769ec8 more retrieval api work 2021-11-22 12:46:02 +01:00
Łukasz Magiera
89138bab4d Simplify retrieval APIs 2021-11-22 12:45:42 +01:00
Aayush Rajasekaran
5f1783c9a5 Address review 2021-11-19 15:11:13 -05:00
Aarsh Shah
7655e660f3 integrate store-the-hash 2021-11-18 12:07:12 +04:00
Aayush Rajasekaran
8665e32221 Update deps 2021-11-17 15:55:14 -05:00
Aayush Rajasekaran
07f1be0ff6 Plug in the FFI call 2021-11-17 15:55:14 -05:00
Aayush Rajasekaran
1449644c34 Integrate v7 actors 2021-11-17 15:55:14 -05:00
Aarsh Shah
60fe55cdc0 fix lint and gen 2021-11-17 15:25:25 +04:00
Aarsh Shah
caf3cfa7d8 Merge branch 'feat/cid-to-piece-idx' into feat/indexer-provider 2021-11-17 11:43:49 +04:00
Aarsh Shah
bfee6699d2 update go mod and inject store the hash index in dagstore 2021-11-17 11:42:14 +04:00
Dirk McCormick
627d4fbf74 wip: storage miner index provider 2021-11-16 17:45:42 +01:00
Aarsh Shah
189f5f8aae fix deals test 2021-11-12 09:43:42 +01:00
Dirk McCormick
e309293a47 fix: lint fixes 2021-11-12 09:43:42 +01:00
Aarsh Shah
4d87f16e95 better comparision in test 2021-11-12 09:43:42 +01:00
Aarsh Shah
053317e334 test for retrieval by any cid 2021-11-12 09:43:42 +01:00
Dirk McCormick
662c51605f feat: retrieve from any cid (not just root cid) 2021-11-12 09:43:42 +01:00
Łukasz Magiera
19f52b4541 inline-gen: Slightly nicer comment format 2021-10-18 17:21:53 +02:00
Łukasz Magiera
9606dce425 Add inline-gen to itests 2021-10-18 17:17:54 +02:00
Anton Evangelatov
a19dbb8c91 remove log 2021-10-11 14:03:56 +02:00
Anton Evangelatov
6a3bb184b7 more tests for deals when provider has no funds in control addr 2021-10-11 14:03:56 +02:00
Anton Evangelatov
1acb6b4328 update CLI command to take arg; reduce time for itest 2021-10-11 14:03:56 +02:00
Anton Evangelatov
6e5ccc87cf cli: add retry for deals stuck in Publish with no funds 2021-10-11 14:03:55 +02:00
Peter Rabbitson
f8ac98bf46 Adjust error wording 2021-10-05 18:16:14 +02:00
Peter Rabbitson
5e6aceeb0e Merge remote-tracking branch 'origin/master' into feat/datamodel-selector-retrieval 2021-10-05 18:15:51 +02:00
Jiaying Wang
cbb147de23
Merge pull request #7405 from filecoin-project/feat/update-graphsync-0.10.0 
Update to graphsync to v0.10.0, enable seperate storage and retrieval transfer limits
 2021-10-05 11:50:59 -04:00
hannahhoward
368d72ebfe
feat(graphsync): update to v0.10.0-rc1 
also add config changes
 2021-10-05 14:13:58 +11:00
Peter Rabbitson
af0d9b650d Error on corner cases with corresponding negative tests 2021-10-04 23:21:24 +02:00
Peter Rabbitson
2a1094fc17 Merge banch feat/go-fil-markets-v1.13.0 into feat/datamodel-selector-retrieval 2021-10-04 21:56:11 +02:00
Jennifer Wang
38e01cba83 Merge branch 'release/v1.12.0' into jen/12tomaster 2021-10-04 02:38:00 -04:00
Aayush Rajasekaran
f004d036dc Set BatchPreCommitAboveBaseFee correctly 2021-10-01 14:23:41 -04:00
Łukasz Magiera
8e0314b3a3
Merge pull request #7362 from filecoin-project/chore/update-libp2p 
Update go-libp2p to v0.15.0
 2021-09-23 14:24:12 +01:00
Aayush Rajasekaran
b9bfcc4b69 Randomness: Move getters from ChainAPI to StateAPI 2021-09-21 14:20:15 -04:00
Aarsh Shah
89ac8b09b5 update libp2p 2021-09-21 15:10:04 +04:00
Aayush Rajasekaran
1d6db306ec Randomness: Move getters from ChainAPI to StateAPI 2021-09-15 00:03:13 +02:00
Peter Rabbitson
0444435589 Expose basic text-based datamodel selector on retrieval 
Syntaxt of selection is located at
https://pkg.go.dev/github.com/ipld/go-ipld-selector-text-lite#SelectorSpecFromPath

Example use, assuming that:
  - The root of the deal is a plain dag-pb unixfs directory
  - The directory is not sharded
  - The user wants to retrieve the first entry in that directory

  lotus client retrieve --miner f0XXXXX --datamodel-path-selector 'Links/0/Hash' bafyROOTCID ~/output

For a much more elaborate example see the top of ./itests/deals_partial_retrieval_test.go
 2021-09-10 09:44:11 +02:00
Anton Evangelatov
567d0546ec itests: remove equality comparison for cids 2021-09-07 10:57:10 +02:00
Anton Evangelatov
992cc3ffbf itests: MaxStagingDealsBytes integration test 2021-09-06 17:39:35 +02:00
Łukasz Magiera
95b128b7bc chain: Cleanup consensus logic 2021-09-02 18:09:37 +02:00
Steven Allen
003eae81ce fix: address review 2021-08-30 16:43:21 -07:00
Steven Allen
3846170302 refactor events system 2021-08-30 16:43:21 -07:00
Steven Allen
43bbde1e6b fix: close chain head subscription when the reader is slow 
The reader can just re-subscribe when they're ready to catch up. This
prevents a slow reader from bogging down the entire system.
 2021-08-30 16:43:21 -07:00
Dirk McCormick
7b7a5b0b21 revert: changes to OnDealExpiredOrChanged in #5431 #7201 2021-08-30 10:42:41 +02:00
Dirk McCormick
07dcb074f8 fix: catch deal slashed because sector was terminated 2021-08-27 12:56:23 +02:00
dirkmc
77a19774cf fix events API timeout handling for nil blocks (#7184) 2021-08-27 09:05:00 +02:00
Steven Allen
1105c24e8c test: disable flaky TestBatchDealInput 
See #4611.
 2021-08-24 15:37:52 -07:00
Steven Allen
b509f1f098
Merge pull request #7148 from filecoin-project/raulk/test-large-deals 
itests: support larger sector sizes; add large deal test.
 2021-08-24 12:21:11 -07:00
Steven Allen
2a4ad207a5 test: disable flaky TestSimultaneousTransferLimit 
See https://github.com/filecoin-project/lotus/issues/7152 for details.
 2021-08-20 14:59:01 -07:00
Raúl Kripalani
342134f853 EnableLargeSectors: restore previous supported proof types. 2021-08-20 18:07:25 +01:00
Raúl Kripalani
0030e208a0 avoid reordering imports. 2021-08-20 15:59:19 +01:00
Raúl Kripalani
2dde022a00 itests: support larger sector sizes; add large deal test. 2021-08-20 15:53:24 +01:00
Aarsh Shah
d7076778e2
integrate DAG store and CARv2 in deal-making (#6671) 
This commit removes badger from the deal-making processes, and
moves to a new architecture with the dagstore as the cental
component on the miner-side, and CARv2s on the client-side.

Every deal that has been handed off to the sealing subsystem becomes
a shard in the dagstore. Shards are mounted via the LotusMount, which
teaches the dagstore how to load the related piece when serving
retrievals.

When the miner starts the Lotus for the first time with this patch,
we will perform a one-time migration of all active deals into the
dagstore. This is a lightweight process, and it consists simply
of registering the shards in the dagstore.

Shards are backed by the unsealed copy of the piece. This is currently
a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so
when it's time to acquire a shard to serve a retrieval, the unsealed
CARv1 is joined with its index (safeguarded by the dagstore), to form
a read-only blockstore, thus taking the place of the monolithic
badger.

Data transfers have been adjusted to interface directly with CARv2 files.
On inbound transfers (client retrievals, miner storage deals), we stream
the received data into a CARv2 ReadWrite blockstore. On outbound transfers
(client storage deals, miner retrievals), we serve the data off a CARv2
ReadOnly blockstore.

Client-side imports are managed by the refactored *imports.Manager
component (when not using IPFS integration). Just like it before, we use
the go-filestore library to avoid duplicating the data from the original
file in the resulting UnixFS DAG (concretely the leaves). However, the
target of those imports are what we call "ref-CARv2s": CARv2 files placed
under the `$LOTUS_PATH/imports` directory, containing the intermediate
nodes in full, and the leaves as positional references to the original file
on disk.

Client-side retrievals are placed into CARv2 files in the location:
`$LOTUS_PATH/retrievals`.

A new set of `Dagstore*` JSON-RPC operations and `lotus-miner dagstore`
subcommands have been introduced on the miner-side to inspect and manage
the dagstore.

Despite moving to a CARv2-backed system, the IPFS integration has been
respected, and it continues to be possible to make storage deals with data
held in an IPFS node, and to perform retrievals directly into an IPFS node.

NOTE: because the "staging" and "client" Badger blockstores are no longer
used, existing imports on the client will be rendered useless. On startup,
Lotus will enumerate all imports and print WARN statements on the log for
each import that needs to be reimported. These log lines contain these
messages:

- import lacks carv2 path; import will not work; please reimport
- import has missing/broken carv2; please reimport

At the end, we will print a "sanity check completed" message indicating
the count of imports found, and how many were deemed broken.

Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>

Co-authored-by: Raúl Kripalani <raul@protocol.ai>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
 2021-08-16 23:34:32 +01:00
ZenGround0
ce58b119bf
Merge pull request #7038 from filecoin-project/spike/restrict-api-versions 
Strict major minor version checking on v0 and v1 apis
 2021-08-12 14:34:31 -04:00
ZenGround0
2d8c9bba99 more fix 2021-08-12 14:03:19 -04:00
ZenGround0
c8720fef02 Fix tests 2021-08-12 14:01:24 -04:00
Steven Allen
dcff06bd91 test: genesis actors version 
Properly handle genesis in actors tests. Fast-forward upgrading to
actors v13 doesn't work because there needs to be at least a day between
v0 genesis and v13 (due to differences in miner cron).
 2021-08-11 13:30:15 -07:00
Steven Allen
b885d69aaa chore: disable flaky test 2021-07-30 09:01:31 -07:00
ZenGround0
823bd54484 Review 2021-07-29 20:48:27 -07:00
ZenGround0
756e5ac0b8 Add a separate node to ensemble and check propagation reached 2021-07-29 20:48:27 -07:00
ZenGround0
7c99169fb4 Fix 2021-07-29 20:48:27 -07:00
ZenGround0
2380d252e1 Post rebase fix 2021-07-29 20:48:27 -07:00
ZenGround0
f2f6c209ea Integration test forcing max batch size 2021-07-29 20:48:27 -07:00
Peter Rabbitson
39325e5463
Merge branch 'master' into feat/allow_padding_redux 2021-07-26 20:53:27 +02:00
Peter Rabbitson
8d873e3edc Tests for online and offline dealpadding 
Also bump the times on several flaky tests that can not complete in time
on a typical laptop ( and fail half the time on CircleCI )
 2021-07-24 17:46:04 +02:00
Raúl Kripalani
8693df4656 fix racy TestSimultanenousTransferLimit. 2021-07-24 12:53:56 +01:00
Peter Rabbitson
2edf7fd25b Rewire itest's StartDeal to take the full API struct 
This allows one to use the harness for much more versatile deal conditions
 2021-07-24 04:17:42 +02:00
raulk
4ffbb03a82
Merge branch 'master' into raulk/merge-gs-fix 2021-07-22 23:37:17 +01:00
Łukasz Magiera
660829703a Merge remote-tracking branch 'origin/master' into feat/split-net-api 2021-07-22 15:38:06 +02:00
Łukasz Magiera
0236f2ec5d
Merge pull request #6802 from filecoin-project/fix/6786-rest 
fix: always check if StateSearchMessage returns nil
 2021-07-21 15:05:57 +02:00
Steven Allen
38919f59ad test: fix flaky window post tests 
Wait until the network upgrade is finished. If we try to seal while it's
happening, we have a few annoying edge cases that can fail if we try to
submit some messages right on the upgrade epoch (which is why everyone
turns that kind of stuff off for the upgrade epoch).
 2021-07-20 16:31:14 -07:00
Steven Allen
f9c759fcb2 fix: always check if StateSearchMessage returns nil 
It returns nil on "not found".

Fixes the cases not covered in:
https://github.com/filecoin-project/lotus/pull/6787
 2021-07-20 11:28:27 -07:00
Raúl Kripalani
5e66960ab6 fix merge error. 2021-07-20 17:51:09 +01:00
raulk
353ccf0c55
Merge branch 'raulk/merge-gs-fix' into raulk/merge-gs-fix-simultaneous-fix 2021-07-20 17:37:51 +01:00
Raúl Kripalani
c1b304b633 update graphsync. 2021-07-20 15:43:44 +01:00
hannahhoward
3b4df5e10d fix(itests): fix simultaneous transfer logic 
limit parallelism on client side and also fix the issue with how ongoing transfers are detected
 2021-07-16 11:46:03 -07:00
Raúl Kripalani
089bcceb33 remove test concurrency guardrails. 2021-07-16 19:09:56 +01:00
Raúl Kripalani
9042429abd Merge branch 'master' into raulk/merge-gs-fix 2021-07-16 18:19:50 +01:00
Raúl Kripalani
aeb8e57eef use https://github.com/ipfs/go-graphsync/pull/176. 2021-07-16 18:17:34 +01:00
Raúl Kripalani
b2feb7e960 upgrade go-fil-markets, go-data-transfer; add logging. 2021-07-16 17:30:45 +01:00
Łukasz Magiera
49e26cce7d api: Separate the Net interface from Common 2021-07-15 11:41:30 +02:00
hunjixin
b2f8b4011a fix imports 2021-07-15 14:52:12 +08:00
hunjixin
3e9ec01d04 rm fmt 2021-07-15 14:52:12 +08:00
hunjixin
a48a6a168a add test for new api 2021-07-15 14:52:12 +08:00
hunjixin
ac03c01e4a test for messages in tipset 2021-07-15 14:52:12 +08:00
Łukasz Magiera
583a8a13d9
Merge pull request #6629 from filecoin-project/feat/pledge-from-miner-balance 
Config for collateral from miner available balance
 2021-07-13 17:22:00 +02:00
Łukasz Magiera
7526a074d9 sealing: collateral buffer / falback config 2021-07-13 17:04:58 +02:00
Łukasz Magiera
83f2368507 Add CollateralFromMinerBalance config 2021-07-12 14:11:58 +02:00
Anton Evangelatov
4bc9fa04ba rename cfg.Subsystems.EnableStorageMarket to EnableMarkets 2021-07-12 12:12:29 +02:00
Anton Evangelatov
d89ddb9315 resolve conflicts 2021-07-12 11:34:37 +02:00
Raúl Kripalani
71ddf73f44 check if merging ipfs/go-graphsync#177 fixes deal concurrency. 2021-07-10 10:48:18 +01:00
Łukasz Magiera
9bdea91a70 itests: Some kit cleanup 2021-07-07 21:40:44 +02:00
Łukasz Magiera
d0e87a7487 dealpublisher: Fix provider address checks 2021-07-07 20:06:49 +02:00
Łukasz Magiera
2dc27d6ab4 itests: Fix deal provider collateral flakiness 2021-07-07 19:41:46 +02:00
Łukasz Magiera
ee97aa0a8a dealpublisher: Use address selector 2021-07-07 19:12:03 +02:00
Anton Evangelatov
6b014f57e5 pass Subsystems to StorageMiner option; add enableLibp2p bool in Settings 2021-07-07 13:56:37 +02:00
Anton Evangelatov
caa366bc6a variadic WithSubsystems 2021-07-07 13:35:40 +02:00
Anton Evangelatov
566a9d5541 fix signature 2021-07-06 18:12:30 +02:00
Anton Evangelatov
2a7af38538 fix testServ to Start with correct listener 2021-07-06 17:57:08 +02:00
Anton Evangelatov
7a78527fde remove not used sa; cleanup 2021-07-06 17:46:21 +02:00
Anton Evangelatov
8cf3accc53
Update itests/kit/ensemble.go 
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
 2021-07-06 17:25:33 +02:00
Anton Evangelatov
9839d5701b fix SectorsStatus to query full node for on-chain data 2021-07-06 15:41:14 +02:00
Anton Evangelatov
5a87f04e71 fix finalize sectors early test 2021-07-05 13:36:23 +02:00
Anton Evangelatov
4be0a7a215 resolve conflicts with master 2021-07-05 13:13:32 +02:00
Łukasz Magiera
13d4563c25
Merge pull request #6663 from filecoin-project/raulk/test-incremental-nonce 
add an incremental nonce itest.
 2021-07-02 23:33:52 +02:00
Łukasz Magiera
8a94ab676e storage: Fix FinalizeSector with sectors in stoage paths 2021-07-02 19:54:45 +02:00
Raúl Kripalani
592b8c4f4a add an incremental nonce itest. 2021-07-02 17:04:13 +01:00
Anton Evangelatov
724f728f60 enable more tests 2021-07-02 14:54:09 +02:00
Anton Evangelatov
9e85492b85 fix test 2021-07-02 12:24:07 +02:00
Anton Evangelatov
3aa6d03cc1 connect miner to full node , only when running markets 2021-06-30 16:45:11 +02:00
Anton Evangelatov
4f0a96c9c7 resolved conflicts 2021-06-30 13:16:52 +02:00
Anton Evangelatov
9ec7cd7239 enable libp2p options 2021-06-30 12:55:43 +02:00
Łukasz Magiera
e9dd3e8650 Test Miner SimultaneousTransfers 2021-06-28 18:17:29 +02:00
Raúl Kripalani
6a48fbbc11 increase suspension threshold to 300. 2021-06-23 19:21:42 +01:00
Raúl Kripalani
58f348cb7f add ability to suspend deal-making until CE is stable. 2021-06-23 19:14:27 +01:00
Raúl Kripalani
9b3188d110 split deals tests. 2021-06-23 18:13:29 +01:00
Raúl Kripalani
f4d0db83d8 Merge branch 'raulk/circleci-dynamic' into raulk/itests-refactor-kit 2021-06-23 17:33:17 +01:00
Raúl Kripalani
211751f8b8 declare some tests as expensive. 2021-06-23 16:37:43 +01:00
Anton Evangelatov
a8d5ae5213 enable more tests 2021-06-23 13:39:03 +02:00
Anton Evangelatov
4217ec5308 make auth on remote rpc server optional 2021-06-23 13:27:03 +02:00
Anton Evangelatov
21b51328f9 adding TestDealWithMarketAndMinerNode 2021-06-23 12:44:52 +02:00
Anton Evangelatov
99646d209e fix TestFirstDealEnablesMining 2021-06-23 11:56:09 +02:00
Anton Evangelatov
7fb93d4585 refactor kit.NewDealHarness to include main miner and market node 2021-06-23 11:24:55 +02:00
Anton Evangelatov
51a4657ef7 fix SDR test 2021-06-23 11:19:25 +02:00
Anton Evangelatov
d98eca7260 Merge branch 'raulk/itests-refactor-kit' into nonsense/split-market-miner-processes 2021-06-23 10:58:37 +02:00
Raúl Kripalani
b74ad796ce fix dangling import. 2021-06-22 21:39:20 +01:00
Raúl Kripalani
539f8c1f61 fix test that references private symbols; avoid hacky test selection flags. 2021-06-22 21:39:20 +01:00
Raúl Kripalani
2e9e1c2330 avoid double BlockMiner instantiation. 2021-06-22 18:15:38 +01:00
Raúl Kripalani
120dd14979 avoid double close. 2021-06-22 17:18:07 +01:00
Raúl Kripalani
9b2efd5ace try to deflake window post itests. 2021-06-22 17:07:14 +01:00
Anton Evangelatov
fc7feffaad Merge branch 'raulk/itests-refactor-kit' into nonsense/split-market-miner-processes 2021-06-22 17:37:50 +02:00
Raúl Kripalani
098eb6bfff try using bg context on constructor. 2021-06-22 16:36:06 +01:00
Raúl Kripalani
71a7270c98 cleanup gateway RPC. 2021-06-22 16:36:06 +01:00
Raúl Kripalani
2a58f830c0 fix sector_terminate_test.go flakiness. 2021-06-22 16:36:06 +01:00
Anton Evangelatov
aa584475cc fix paych and sdr tests 2021-06-22 16:48:07 +02:00
Anton Evangelatov
218c9199f3 fix testNonGenesisMiner 2021-06-22 15:51:50 +02:00
Anton Evangelatov
4f2d8b0a85 add all subsystems to deadlines and wdpost_dispute tests 2021-06-22 15:35:46 +02:00
Anton Evangelatov
a828b15fbe revert MockSectorMgr 2021-06-22 15:22:08 +02:00
Anton Evangelatov
9567807144 Merge branch 'raulk/itests-refactor-kit' into nonsense/split-market-miner-processes 2021-06-22 14:39:28 +02:00
Dirk McCormick
da789939b1 fix: bump blocktime of TestQuotePriceForUnsealedRetrieval to 1 second 2021-06-22 14:33:44 +02:00
Anton Evangelatov
b0d1668ab9 comment out preseal sectors 2021-06-22 13:34:45 +02:00
Anton Evangelatov
00618672f5 policy.SetPreCommitChallengeDelay to reduce WaitSeed stage time 2021-06-22 13:08:36 +02:00
Anton Evangelatov
5e10d53ca8 wip 2021-06-22 12:16:21 +02:00
Anton Evangelatov
6720463799 resolve merge conflicts 2021-06-22 11:28:23 +02:00
Raúl Kripalani
bb032b526c switch back to ws API. 2021-06-21 23:24:59 +01:00
Raúl Kripalani
0e2d06fc39 switch to http API. 2021-06-21 23:10:17 +01:00
Raúl Kripalani
502e104e6a typo. 2021-06-21 21:35:39 +01:00
Raúl Kripalani
e438ef99f8 fix merge error in window post dispute tests. 2021-06-21 21:25:55 +01:00
Raúl Kripalani
4fcd0b7aca disable resource filtering on scheduler. 2021-06-21 21:20:45 +01:00
Raúl Kripalani
b7a5e3cd0f Merge branch 'raulk/worker-disable-resource-filtering' into raulk/itests-refactor-kit 2021-06-21 21:09:04 +01:00
Raúl Kripalani
b6147fb27f extern/storage: retype resource filtering strategy to enum. 2021-06-21 20:28:15 +01:00
Raúl Kripalani
f3b6f8de1a add ability to ignore worker resources when scheduling. 2021-06-21 20:08:18 +01:00
Raúl Kripalani
c22e10c4a4 use mock proofs in TestQuotePriceForUnsealedRetrieval. 2021-06-21 18:29:25 +01:00
Raúl Kripalani
132b49728c Merge branch 'master' into raulk/itests-refactor-kit 2021-06-21 18:26:04 +01:00
Raúl Kripalani
70929a99e6 speed up test. 2021-06-21 18:24:39 +01:00
Raúl Kripalani
00fa3878d4 remove debug statement. 2021-06-21 18:22:08 +01:00
Raúl Kripalani
0879ac496f uncomment lines in TestDealCyclesConcurrent. 2021-06-21 18:21:10 +01:00
Raúl Kripalani
83ee345b1c Merge branch 'master' into raulk/itests-refactor-kit 2021-06-21 18:19:26 +01:00
Łukasz Magiera
b37a66c7c9 Merge remote-tracking branch 'origin/release/v1.10.0' into chore/merge-1.10 2021-06-21 17:03:03 +02:00
Raúl Kripalani
d1b291de5e fix proof types. 2021-06-21 13:24:25 +01:00
Raúl Kripalani
8a418bf982 rename {kit2=>kit}. 2021-06-18 19:45:29 +01:00
Raúl Kripalani
a7d8d15c13 kill old kit 🎉 2021-06-18 19:42:06 +01:00
raulk
a8bf7e7bad
Merge pull request #6510 from filecoin-project/refactor/itest-wdpost-dispute 2021-06-18 19:40:12 +01:00
Raúl Kripalani
718babd33a use miner owner address when posting proofs. 2021-06-18 19:38:17 +01:00
Raúl Kripalani
9604be6d2f Merge branch 'raulk/itests-refactor-kit' into refactor/itest-wdpost-dispute 2021-06-18 19:29:18 +01:00
Raúl Kripalani
3d8eb374bd cleaner instantiation of lite and gateway nodes + general cleanup. 2021-06-18 19:23:42 +01:00
Raúl Kripalani
3ae42d4648 Merge branch 'raulk/itests-refactor-kit' into refactor/itest-multisig 2021-06-18 19:08:07 +01:00
Raúl Kripalani
5548541e1a fix default proof type to use for non-genesis miners. 
We need to instantiate non-genesis miners with a _concrete_
proof type.
 2021-06-18 18:10:42 +01:00
Anton Evangelatov
577b730415 wip 2021-06-18 18:42:34 +02:00
Raúl Kripalani
4c2cc8e5f7 Merge branch 'raulk/itests-refactor-kit' into refactor/itest-deadling-toggling 2021-06-18 16:20:15 +01:00
Raúl Kripalani
e85af3cba7 fix merge error. 2021-06-18 16:19:58 +01:00
Raúl Kripalani
32327c7d43 Merge branch 'raulk/itests-refactor-kit' into refactor/itest-deadling-toggling 2021-06-18 11:22:12 +01:00
Anton Evangelatov
89df3cc207 Merge branch 'raulk/itests-refactor-kit' into nonsense/split-market-miner-processes 2021-06-18 12:07:15 +02:00
Raúl Kripalani
2548c224c7 switch to TestFullNode#WaitTillChain. 2021-06-18 10:27:20 +01:00
Raúl Kripalani
21848054d7 Merge branch 'raulk/itests-refactor-kit' into refactor/itest-wdpost 2021-06-18 10:26:17 +01:00
Raúl Kripalani
9436be5ff0 introduce TestFullNode#WaitTillChain(predicate). 2021-06-17 22:02:20 +01:00
Anton Evangelatov
338bd042b9 Merge branch 'raulk/itests-refactor-kit' into nonsense/split-market-miner-processes 2021-06-17 18:12:48 +02:00
Dirk McCormick
7a9769b807 refactor: deadline toggling test to kit2 2021-06-17 14:58:35 +02:00
Dirk McCormick
3de6beabe3 refactor: wdpost dispute test to use kit2 2021-06-17 14:31:57 +02:00
Dirk McCormick
e9325fecc5 refactor: wdpost test to kit2 2021-06-17 14:31:20 +02:00
Dirk McCormick
6567887e6e refactor: sector terminate test to kit2 2021-06-17 14:27:44 +02:00
Dirk McCormick
8f608dff45 refactor: sector pledge test to use kit2 2021-06-17 09:37:35 +02:00
Raúl Kripalani
f1bf080e04 Merge branch 'raulk/itests-refactor-kit' into refactor/itest-multisig 2021-06-16 20:00:11 +01:00
Raúl Kripalani
8ed753a712 rename RootVerifier option. 2021-06-16 18:16:59 +01:00
Raúl Kripalani
a3015ca9c2 Merge branch 'raulk/itests-refactor-kit' into refactor/itest-verifreg 2021-06-16 18:12:51 +01:00