Commit Graph

260 Commits

Author SHA1 Message Date
Łukasz Magiera
5f5cc794f0 make config not depend on ffi 2022-11-08 09:37:43 +00:00
ZenGround0
4ffded6fef
feat:splitstore:single compaction that can handle prune aka two marksets one compaction (#9571)
* begin

* rough draft -- this should probably actually work?

* WIP

* Start testing

* message mode

* Fix tests, make gen

* Better default

* docsgen-cli

* Review Response

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2022-11-07 16:31:12 -05:00
Shrenuj Bansal
15ed1ee33c Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-10-17 22:55:48 -04:00
Shrenuj Bansal
674427a8b2 fix lint and make gen 2022-10-17 22:44:00 -04:00
Shrenuj Bansal
900525f8c2 some cleanup 2022-10-17 16:13:30 -04:00
Shrenuj Bansal
17a77220c2 Ignore mpool msg existing errors for applying raft state 2022-10-06 11:44:13 +00:00
Shrenuj Bansal
559c2c6d34 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-09-27 16:29:03 +00:00
Shrenuj Bansal
99e7c322eb More wip 2022-09-27 16:08:04 +00:00
Aayush
9c242a8012 address magik supernit 2022-09-27 15:34:01 +00:00
Aayush
13f3e0aca4 fix: itest: check for closed connection 2022-09-27 15:34:01 +00:00
Aayush
3e81c56349 address magik supernit 2022-09-23 11:41:53 -04:00
Aayush
06262868f5 fix: itest: check for closed connection 2022-09-22 13:58:07 -04:00
Aayush
32670e810c chore: refactor: rename NewestNetworkVersion 2022-09-21 15:48:51 -04:00
Łukasz Magiera
061a990eb8 sector import: RemoteSealingDoneEndpoint 2022-09-19 12:13:06 +02:00
Łukasz Magiera
b2dfaae68c sector import: Test remote commit1 retry 2022-09-19 12:13:06 +02:00
Łukasz Magiera
9c6d531ae7 sector import: Plumb sector download task canfig 2022-09-19 12:13:06 +02:00
Łukasz Magiera
5c485c3375 sealing: Drop redundant Piece type 2022-09-19 12:13:05 +02:00
Shrenuj Bansal
4be8861f41 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-09-13 17:41:06 -04:00
Shrenuj Bansal
3441224b2f WIP: rest of the stuff 2022-09-13 17:30:13 -04:00
Shrenuj Bansal
4171be0b98 Few more changes 2022-09-12 16:37:24 -04:00
Shrenuj Bansal
a1f2fdb706 Merge branch 'master' into sbansal/nonce-coordination-and-consensus-for-chain-nodes 2022-09-12 16:23:24 -04:00
Shrenuj Bansal
8f1b1bb1ff WIP: Raft consensus for lotus nodes in a cluster 2022-09-12 16:10:15 -04:00
Aayush
0efca4d266 introduce v9 actors and nv17 2022-09-09 21:40:15 -04:00
Łukasz Magiera
08b22edd89 fix make gen 2022-08-29 16:25:30 +02:00
Łukasz Magiera
45d1bd61ce
Merge pull request #9183 from filecoin-project/feat/sectornum-mgmt
feat: sealing: Use bitfields to manage sector numbers
2022-08-26 10:59:24 -04:00
Łukasz Magiera
2086b219d2 Don't use go-libp2p-core 2022-08-25 14:20:41 -04:00
Łukasz Magiera
ca72590e49 sealing: Use bitfields to manage sector numbers 2022-08-22 16:55:08 -04:00
Shrenuj Bansal
b7227d3a6b run make gen 2022-08-16 15:56:32 -04:00
Shrenuj Bansal
fa4a479b97 Add uuid to mpool message sent to chain node from miner 2022-08-16 15:39:06 -04:00
ZenGround0
8b7be6d47e
feat:chain:splitstore auto prune (#9123)
Auto Prune

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2022-08-08 16:06:32 -04:00
ZenGround0
0c91b0dc10
feat:chain:splitstore chain prune (#9056)
* Splitstore chain prune
* Protect on reification for simpler logic and sound cold compact protect
* Recovery from checkpoint during chain prune
* Splitstore (discard and universal mode) running in itests
* Add pause and restart functions to itest block miner
* Add config options to itest full nodes
* Add FsRepo support for itest full ndoes

Co-authored-by: zenground0 <ZenGround0@users.noreply.github.com>
2022-08-05 16:34:16 -04:00
Łukasz Magiera
819314de46 Merge remote-tracking branch 'origin/master' into feat/storage-redeclare 2022-08-05 09:01:30 +02:00
Łukasz Magiera
ecddd08048 itests: Test custom worker names 2022-08-03 13:05:30 +02:00
Łukasz Magiera
8294e03d4e itests: More fixes after fixing sector metadata import 2022-08-02 17:35:26 +02:00
Łukasz Magiera
d1173265b3 itests: Helper returning non-genesis sector list 2022-08-02 15:29:38 +02:00
Łukasz Magiera
3cd8d9eb06 itests: Worker tests for storage detach/redeclare 2022-08-01 15:58:06 +02:00
Łukasz Magiera
0ccaa00969 itests ensemble: import presealed sector metadata 2022-08-01 15:58:06 +02:00
Łukasz Magiera
9eb8f4ee9d
Merge pull request #9013 from filecoin-project/feat/path-type-filters
feat: storage: Path type filters
2022-07-15 13:08:43 +02:00
Łukasz Magiera
91e474e72e sealer: Pass correct args to SubAllowed in selectors 2022-07-12 21:45:12 +02:00
Łukasz Magiera
59d069dc4d itests: Tests for storage path filters 2022-07-12 21:45:12 +02:00
Łukasz Magiera
0ca1cc2bcd itests: Test lotus-miner with no store/seal paths 2022-07-12 21:45:12 +02:00
Aayush
9662c1463d Fixup itest net tests 2022-07-10 11:09:54 -04:00
Aayush
0c34699f23 unbreak itests 2022-06-29 14:55:35 -04:00
Łukasz Magiera
db676acdf8 storage: Cleanup interfaces a bit 2022-06-17 13:52:19 +02:00
Łukasz Magiera
05cdeb80c3 chore: remove redundant import prefixes 2022-06-15 12:06:22 +02:00
Łukasz Magiera
39cede946c chore: Fix more imports with '2' suffix 2022-06-14 21:37:54 +02:00
Łukasz Magiera
28099a3905 storage: Move storage/sealer/stores to storage/paths 2022-06-14 20:25:52 +02:00
Łukasz Magiera
a9600b8a6f storage: Move extern/sector-storage to storage/sealer 2022-06-14 20:03:38 +02:00
Łukasz Magiera
98a48a47f8 storage: Move extern/storage-sealing to storage/pipeline 2022-06-14 19:41:59 +02:00
Łukasz Magiera
12c91bb41d storage: Move mock PreCommit helper to the mock package 2022-06-14 19:23:14 +02:00
Łukasz Magiera
6afb43afbb wallet: Move key.go into separate pkg to avoid import loops 2022-06-14 19:21:40 +02:00
Łukasz Magiera
e65fae28de chore: fix imports 2022-06-14 17:00:51 +02:00
Steven Allen
30981d0fdd
feat: refactor: actor bundling system (#8838)
1. Include the builtin-actors in the lotus source tree.
2. Embed the bundle on build instead of downloading at runtime.
3. Avoid reading the bundle whenever possible by including bundle
   metadata (the bundle CID, the actor CIDs, etc.).
4. Remove everything related to dependency injection.
    1. We're no longer downloading the bundle, so doing anything ahead
       of time doesn't really help.
    2. We register the manifests on init because, unfortunately, they're
       global.
    3. We explicitly load the current actors bundle in the genesis
       state-tree method.
    4. For testing, we just change the in-use bundle with a bit of a
       hack. It's not great, but using dependency injection doesn't make
       any sense either because, again, the manifest information is
       global.
    5. Remove the bundle.toml file. Bundles may be overridden by
       specifying an override path in the parameters file, or an
       environment variable.

fixes #8701
2022-06-13 10:15:00 -07:00
Aayush
b28c11a57d Merge branch 'feat/nv16' 2022-06-03 14:01:49 -04:00
Łukasz Magiera
8c081e271a itests: test TestWorkerPledgeLocalFin 2022-05-24 01:13:30 +02:00
Łukasz Magiera
58574554c1 itests: Test spread sched assigner 2022-05-23 22:31:06 +02:00
Aayush
8cca9b1970 Use new go-state-types accessors 2022-05-17 15:21:27 -04:00
vyzo
5ceaeafa6e fix TODO for proof type, now that we have the testing bundle 2022-04-20 12:47:06 +03:00
vyzo
c5e3ca3855 fix more rebase issues 2022-04-14 19:52:58 +03:00
vyzo
dec2b312af fix lint 2022-04-14 19:33:24 +03:00
vyzo
73e7c42b7e mark TODO for fixing policy stuffs in test 2022-04-14 19:31:34 +03:00
vyzo
2c9ea8c46b fix test 2022-04-14 19:31:34 +03:00
vyzo
71913521f9 fix itest migrations 2022-04-14 19:30:47 +03:00
Aayush
3370847f3e Implement FIP-0031 2022-04-14 19:28:16 +03:00
dirkmc
c34ade9ffb
Update itests/kit/rpc.go
Co-authored-by: Anton Evangelatov <anton.evangelatov@gmail.com>
2022-04-07 13:32:00 +02:00
Dirk McCormick
5f0489ace0 fix: testkit: give up on waiting for the RPC server to shutdown after 1 second 2022-04-07 12:22:36 +02:00
Łukasz Magiera
ebd34f1884 feat: storage: Parallel proving checks 2022-03-29 11:45:08 -04:00
Łukasz Magiera
82343a06e7 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-25 16:49:46 -04:00
Łukasz Magiera
3f152ae2bf workers: cmd/lotus-seal-worker->cmd/lotus-worker 2022-03-24 18:37:01 -04:00
Łukasz Magiera
92e4caa151 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-21 12:09:47 +01:00
Łukasz Magiera
0536eb898d itests: Share WaitForSectorActive correctly 2022-03-21 12:07:00 +01:00
Łukasz Magiera
ecd1ab4b42 itests: Test Sealing.PreferNewSectorsForDeals 2022-03-21 12:07:00 +01:00
Łukasz Magiera
32818c528f wdpost: Test repro broken challenge skipping 2022-03-18 12:32:16 +01:00
Łukasz Magiera
6ddbe41376 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-18 10:54:44 +01:00
Łukasz Magiera
42b2adddee fix: itests: Don't hang on exit in MineBlocksMustPost 2022-03-18 10:42:21 +01:00
Łukasz Magiera
aef2ec513f feat: sealing: Sector upgrade queue 2022-03-16 17:33:05 +01:00
Łukasz Magiera
135aef78d7 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-11 17:04:58 +01:00
Łukasz Magiera
231893a794 feat: shed: ItestD 2022-03-10 19:39:10 +01:00
Łukasz Magiera
046a9f8af0 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-03-09 16:27:03 +01:00
Masih H. Derkani
769b0f30ef
Address review comments
- Add comment to clarify the reason for loop in testkit
- Trim common prefix in state printed in CLI commands for better
  readability
- Upgrade to a tagged release of `go-fil-markets` that includes indexing
  work; see: https://github.com/filecoin-project/go-fil-markets/pull/673
- Fix typo in CLI usage.
- Add comments to note that it is safe to use fx `OnStart` context when
  starting the provider engine.
- Fix string concatenation in error message formatting.
2022-03-02 11:04:50 +00:00
zenground0
77a954c7c3 Merge branch 'master' into feat/cid-to-piece-idx 2022-02-28 12:57:11 -07:00
Elijah Seed-Arita
e2a93a3feb add itests ensemble mocknet getter 2022-02-18 11:09:56 -10:00
zenground0
977351f419 Fix from Magik to remove hanging behavior 2022-02-14 14:00:41 -05:00
zenground0
2c06eb76d6 Improve MineBlocksMustPost use it in PaychAPI itest 2022-02-14 09:47:18 -05:00
Nikola Divic
34387326d1 test: fix flaky message pool integration tests
Using the same pattern described in my previous commit.
I also added the CircuitBreaker to the itests kit as it may be useful
for other integration tests when debugging flakyness caused by timeouts.
2022-02-12 19:52:51 +01:00
Łukasz Magiera
4ad97f6a36 make gen 2022-02-08 17:22:41 +01:00
Łukasz Magiera
09cfad9d71 Add FinalizeReplicaUpdate into some more places 2022-02-08 17:22:41 +01:00
Łukasz Magiera
7fa84518c8 itests: Print API info env vars in ThroughRPC tests 2022-02-08 17:22:41 +01:00
Anton Evangelatov
595ad44ee7 refactor indexprovider libp2p host connection to fullnode with meshcreator 2022-02-03 15:44:18 +01:00
Anton Evangelatov
20fc5ffa38 fixing tests 2022-02-03 14:43:20 +01:00
Łukasz Magiera
55c9271695 post workers: Test skips 2022-01-21 13:33:47 +01:00
Łukasz Magiera
33dfcf2663 stores: Fix single post proof reading with updated sectors 2022-01-20 15:27:42 +01:00
Aarsh Shah
9ee940098d Merge remote-tracking branch 'origin/master' into feat/cid-to-piece-idx 2022-01-20 16:21:48 +04:00
Łukasz Magiera
e476cf7968 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-01-20 13:15:48 +01:00
vyzo
c3fb68545a fix mocknet.New invocation 2022-01-20 11:36:11 +02:00
Łukasz Magiera
c79f2b9d38 post workers: Fix lint 2022-01-18 12:11:59 +01:00
Łukasz Magiera
b38141601c Untangle ffi from api 2022-01-18 11:57:04 +01:00
Łukasz Magiera
03fa955e9e Cleanup imports 2022-01-18 11:37:24 +01:00
Łukasz Magiera
0c062f9c08 Merge remote-tracking branch 'origin/master' into feat/post-worker 2022-01-18 10:49:42 +01:00
Łukasz Magiera
4a874eff70 post workers: Cleanup, tests 2022-01-14 14:17:52 +01:00