cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
16,303 Commits 24 Branches 342 Tags 314 MiB
12c91bb41d
Commit Graph

151 Commits

Author SHA1 Message Date
Aayush Rajasekaran
a8cb027c08 Integrate v7 actors 2022-01-14 17:14:32 -05:00
Aarsh Shah
051d993d62 update markets 2022-01-12 17:48:12 +04:00
Aarsh Shah
07c842d6e0 lotus soup green 2022-01-12 17:43:56 +04:00
Aayush Rajasekaran
544cfa63ab cache added cids 2022-01-11 17:22:36 -05:00
Aayush Rajasekaran
19bd9cf945 Update to actors v7.0.0-rc1 2022-01-11 17:20:08 -05:00
Aayush Rajasekaran
8aabe1b488 Fast migration for v15 2022-01-11 17:19:38 -05:00
Jennifer Wang
edd3486d2c add temp snap params 2022-01-11 17:01:15 -05:00
Aayush Rajasekaran
207d33eaba
Merge pull request #7810 from filecoin-project/feat/snap-deals 
Snap Deals Lotus Integration: FSM Posting and integration test
 2022-01-11 12:46:55 -05:00
Aayush Rajasekaran
d645c5fbab Remove unnecessary params from VerifyWinningPost 2022-01-11 12:06:39 -05:00
Aayush Rajasekaran
9614f6db8d Update go-paramfetch 2022-01-10 19:03:26 -05:00
Aayush Rajasekaran
5ec5ebac31 Fix circsuypply calc around null blocks 2022-01-05 14:17:40 -05:00
Aayush Rajasekaran
aa1b770974 Rand: Refactor so that versioning lives in StateRand 2021-12-17 13:58:53 -05:00
Łukasz Magiera
bc384c01e3 Merge remote-tracking branch 'origin/master' into deps/update-ctx-dsbs 2021-12-17 13:01:22 +01:00
Aayush Rajasekaran
0e2278cc76 Merge branch 'master' into deps/update-ctx-dsbs 2021-12-14 18:08:29 -05:00
Aayush Rajasekaran
7fb590208c Deps: Update lotus-soup to point to tagged releases 2021-12-14 18:05:52 -05:00
vyzo
0eec00055f fix lotus-soup build 2021-12-14 18:23:43 +02:00
vyzo
e8df325796 update lotus-soup deps 2021-12-14 18:23:33 +02:00
Aayush Rajasekaran
80d5e52923 Merge branch 'master' into next 2021-12-13 13:24:28 -05:00
Łukasz Magiera
763659b8a3 retrieval: Update lotus-soup 2021-11-22 12:46:29 +01:00
Aayush Rajasekaran
1449644c34 Integrate v7 actors 2021-11-17 15:55:14 -05:00
Travis Person
2d4f5958e2 Add caches to lotus-stats and splitcode 2021-11-01 09:05:14 +00:00
Peter Rabbitson
778483166f Remove obsolete GS testplan - it now lives in go-graphsync 2021-10-07 12:23:43 +02:00
Jennifer Wang
38e01cba83 Merge branch 'release/v1.12.0' into jen/12tomaster 2021-10-04 02:38:00 -04:00
Aayush Rajasekaran
ec14259b5a Prep the butterfly network for v14 2021-10-02 17:27:39 -04:00
Aayush Rajasekaran
50ce475701 Upgrade to actors v6-rc1 2021-10-01 17:57:32 -04:00
Aayush Rajasekaran
a335cb3767 Update to latest actors 2021-10-01 17:11:05 -04:00
Aayush Rajasekaran
2bafdf7271 Modify tooling to support new WithdrawBalance return 2021-09-29 13:01:04 -04:00
Aarsh Shah
7694f895c2 update lotus soup dep 2021-09-23 09:15:47 +04:00
Aarsh Shah
9a07c2fd38 revert go car update 2021-09-22 12:55:22 +04:00
Aayush Rajasekaran
7b4c657e90 Add v6 actors 2021-09-21 14:23:14 -04:00
Aarsh Shah
7863a269ad fix lotus-soup 2021-09-21 15:31:16 +04:00
Aarsh Shah
89ac8b09b5 update libp2p 2021-09-21 15:10:04 +04:00
Steven Allen
0789a2927a fix: make lotus soup use the correct dependencies 
1. Add a replace directive for lotus.
2. Make sure we're not depending on commits that don't exist.
 2021-08-30 08:57:24 -07:00
Aarsh Shah
d7076778e2
integrate DAG store and CARv2 in deal-making (#6671) 
This commit removes badger from the deal-making processes, and
moves to a new architecture with the dagstore as the cental
component on the miner-side, and CARv2s on the client-side.

Every deal that has been handed off to the sealing subsystem becomes
a shard in the dagstore. Shards are mounted via the LotusMount, which
teaches the dagstore how to load the related piece when serving
retrievals.

When the miner starts the Lotus for the first time with this patch,
we will perform a one-time migration of all active deals into the
dagstore. This is a lightweight process, and it consists simply
of registering the shards in the dagstore.

Shards are backed by the unsealed copy of the piece. This is currently
a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so
when it's time to acquire a shard to serve a retrieval, the unsealed
CARv1 is joined with its index (safeguarded by the dagstore), to form
a read-only blockstore, thus taking the place of the monolithic
badger.

Data transfers have been adjusted to interface directly with CARv2 files.
On inbound transfers (client retrievals, miner storage deals), we stream
the received data into a CARv2 ReadWrite blockstore. On outbound transfers
(client storage deals, miner retrievals), we serve the data off a CARv2
ReadOnly blockstore.

Client-side imports are managed by the refactored *imports.Manager
component (when not using IPFS integration). Just like it before, we use
the go-filestore library to avoid duplicating the data from the original
file in the resulting UnixFS DAG (concretely the leaves). However, the
target of those imports are what we call "ref-CARv2s": CARv2 files placed
under the `$LOTUS_PATH/imports` directory, containing the intermediate
nodes in full, and the leaves as positional references to the original file
on disk.

Client-side retrievals are placed into CARv2 files in the location:
`$LOTUS_PATH/retrievals`.

A new set of `Dagstore*` JSON-RPC operations and `lotus-miner dagstore`
subcommands have been introduced on the miner-side to inspect and manage
the dagstore.

Despite moving to a CARv2-backed system, the IPFS integration has been
respected, and it continues to be possible to make storage deals with data
held in an IPFS node, and to perform retrievals directly into an IPFS node.

NOTE: because the "staging" and "client" Badger blockstores are no longer
used, existing imports on the client will be rendered useless. On startup,
Lotus will enumerate all imports and print WARN statements on the log for
each import that needs to be reimported. These log lines contain these
messages:

- import lacks carv2 path; import will not work; please reimport
- import has missing/broken carv2; please reimport

At the end, we will print a "sanity check completed" message indicating
the count of imports found, and how many were deemed broken.

Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>

Co-authored-by: Raúl Kripalani <raul@protocol.ai>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
 2021-08-16 23:34:32 +01:00
Anton Evangelatov
bb466ede66 fix cast on role_miner in testground lotus-soup plan 2021-07-07 16:04:01 +02:00
Anton Evangelatov
1fe08f5973 refactor testkit in lotus-soup to include Subsystems when instantiating miner 2021-07-07 14:21:28 +02:00
Anton Evangelatov
6b014f57e5 pass Subsystems to StorageMiner option; add enableLibp2p bool in Settings 2021-07-07 13:56:37 +02:00
Anton Evangelatov
8dbae00847 fix lotus-soup 2021-07-02 12:45:56 +02:00
Anton Evangelatov
4217ec5308 make auth on remote rpc server optional 2021-06-23 13:27:03 +02:00
Łukasz Magiera
b37a66c7c9 Merge remote-tracking branch 'origin/release/v1.10.0' into chore/merge-1.10 2021-06-21 17:03:03 +02:00
Aayush Rajasekaran
514107a75b Fix soup 2021-06-16 18:04:45 -04:00
Aayush Rajasekaran
35f76f58df
Merge pull request #6309 from filecoin-project/feat/better_mining_projection_and_accuracy 
Adjust various CLI display ratios to arbitrary precision
 2021-06-16 17:48:23 -04:00
Aayush Rajasekaran
520a0091f5 Merge branch 'releases' into release/v1.10.0 2021-06-15 17:40:36 -04:00
Peter Rabbitson
4efc3f8db0 Copy latest version to lotus-soup 2021-06-15 22:00:17 +02:00
Anton Evangelatov
c7c029ea91 testplans: lotus-soup: new images with filecoin-ffi ; use default WPoStChallengeWindow 2021-06-07 11:28:24 +02:00
Aayush Rajasekaran
4a321c6da2 Fix nerpa build 2021-06-02 19:19:32 -04:00
Aayush Rajasekaran
cd4505dd63 Update to specs-actors v5-rc-2 2021-06-02 19:11:47 -04:00
Łukasz Magiera
08b7ab90c1 mod tidy, fix testground build 2021-06-02 15:29:54 +02:00
Łukasz Magiera
6d9779ecae Merge remote-tracking branch 'origin/master' into feat/nv13-1.11 2021-06-02 15:07:13 +02:00
Łukasz Magiera
6ead83b605 Go 1.16 in Dockerfile 2021-06-02 14:42:48 +02:00
Łukasz Magiera
cba5c34aef Fix lotus-soup build 2021-06-02 12:38:07 +02:00
Dirk McCormick
133459756b fix: update lotus-soup 2021-06-01 13:44:42 -06:00
Dirk McCormick
6e92c43dd5 feat: add testground versions composition 2021-06-01 10:40:55 -06:00
Łukasz Magiera
1e4456138e Merge master into feat/nv13 2021-05-27 12:28:20 +02:00
Peter Rabbitson
c2e5a837e6 Adjust various CLI display ratios to arbitrary precision 
Originally the deviations from using float64 were insignificant, but at
exabyte scale they start to show up. Cleanup all displays, and clarify
the expectation text, adding an extra 99.9% probability calculator to
`lotus-miner info`
 2021-05-25 14:09:01 +02:00
Aayush Rajasekaran
b5da2655dc Introduce v5 actors 2021-05-10 19:44:28 -04:00
Dirk McCormick
c17a340f2e feat: deals end to end test with restarts 2021-05-06 09:49:44 +02:00
Łukasz Magiera
492b026031
Merge pull request #6124 from filecoin-project/testplans-go-mod 
go mod tidy for lotus-soup testplans
 2021-04-29 18:31:32 +02:00
Anton Evangelatov
5769a45380 use 1 miner for payment channel tests 2021-04-28 16:58:03 +02:00
Anton Evangelatov
bc8d6a1d87 go mod tidy for lotus-soup testplans 2021-04-28 14:36:20 +02:00
Anton Evangelatov
0304bebf20 remove deals concurrency 2021-04-28 13:38:18 +02:00
Anton Evangelatov
a9e4a0d22b upgrade docker images 2021-04-27 16:28:57 +02:00
Łukasz Magiera
a5921a9bf9 fix lotus-soup build 2021-04-06 13:30:36 +02:00
Łukasz Magiera
c9866da688 Merge remote-tracking branch 'origin/master' into feat/v1-api 2021-04-02 13:15:33 +02:00
Anton Evangelatov
1626dffd4c remove overwrite for go-data-transfer in testplans 2021-03-31 23:17:16 +03:00
Łukasz Magiera
9223a8e5ff Fix lotus-soup build 2021-03-25 15:29:20 +01:00
Anton Evangelatov
b4d4766615 use go-data-transfer from personal fork 2021-03-23 14:30:26 +02:00
Raúl Kripalani
7f0f7d0b36 Merge branch 'master' into refactor/lib/blockstore 2021-02-28 19:55:23 +00:00
Anton Evangelatov
59f814028f add storageadapter.PublishMsgConfig to miner in testkit 2021-02-09 19:12:59 +01:00
Anton Evangelatov
891971edec circleci: add task to get latest master for lotus 2021-02-08 17:48:37 +01:00
Anton Evangelatov
cc91eab0fd fix lr.Datastore interface to include context 2021-02-08 17:38:51 +01:00
Anton Evangelatov
764a08b980 fix testplans build 2021-02-08 16:32:30 +01:00
Łukasz Magiera
0afe7325f6
Merge pull request #4995 from filecoin-project/lock-lotus-version-in-testplan 
rename oni/lotus-soup imports to testplans/lotus-soup and go get lotus@master on trigger
 2021-02-08 15:39:15 +01:00
Łukasz Magiera
a541a2500a
Merge pull request #5282 from filecoin-project/chore/snake_context_through_blockstore_init 
Snake a context through the Chain-blockstore creation
 2021-01-30 11:55:17 +01:00
Łukasz Magiera
bb4e3e49ea Merge remote-tracking branch 'origin/master' into feat/no-timing-compute-state 2021-01-30 10:57:32 +01:00
Raúl Kripalani
af429dba0f Merge branch 'master' into chore/snake_context_through_blockstore_init 2021-01-29 21:21:46 +00:00
Raúl Kripalani
e02fdf5064 Merge branch 'master' into refactor/lib/blockstore 2021-01-29 21:21:31 +00:00
Anton Evangelatov
52a610650a merge master; resolve conflicts 2021-01-29 22:20:10 +01:00
Anton Evangelatov
1cca0ff9e2 upgrade docker; upgrade filecoin-ffi; bump up image versions 2021-01-29 22:02:45 +01:00
Anton Evangelatov
8ed726f9e3 bump up go to 1.15 2021-01-29 21:19:27 +01:00
Raúl Kripalani
b0cbc932bd consolidate all blockstores in blockstore package. 2021-01-29 20:01:00 +00:00
Anton Evangelatov
c09571984a remove blst replace; update go.mod to use lotus@master 2021-01-29 19:53:34 +01:00
Łukasz Magiera
389e6eb70b state compute-state ---html --no-timing flag 2021-01-28 19:37:32 +01:00
Raúl Kripalani
a1da1dab85 add context to LockedRepo#Datastore(). 2021-01-26 11:01:43 +00:00
Raúl Kripalani
dbbd1d7a45 fix stale imports in testplans/lotus-soup. 2021-01-26 10:53:27 +00:00
Anton Evangelatov
655277171b baseline 1 miner 1 cluster composition for local:docker 2020-12-17 16:19:41 +01:00
Anton Evangelatov
4d69947eed resolve conflicts with master 2020-12-11 10:01:14 +01:00
Anton Evangelatov
1cfd9555d5 reduce sectors ; less verbose logging for some subsystems 2020-12-07 15:11:41 +01:00
Anton Evangelatov
064b036000 update debug image to use 8MiB sectors 2020-12-07 14:14:21 +01:00
Anton Evangelatov
2829245a46 baseline k8s configs 2020-12-07 11:48:22 +01:00
Anton Evangelatov
b1dcf5fead 8MB sectors ; 5MB deal 2020-12-07 11:48:22 +01:00
Anton Evangelatov
d0599ff1e3 dummy delay for everyone but first miner 2020-12-07 11:48:22 +01:00
Anton Evangelatov
a53fd82379 add more time to miner to bootstrap 2020-12-07 11:48:22 +01:00
Anton Evangelatov
86af05ba6f update base image with latest filecoin-ffi 2020-12-07 11:48:22 +01:00
Anton Evangelatov
c543a10ce7 rename oni/lotus-soup to testplans/lotus-soup and lock lotus version 2020-12-07 11:48:22 +01:00
Aayush Rajasekaran
370817eb60 Allow miners to filter (un)verified deals 2020-12-02 01:32:34 -05:00
Anton Evangelatov
2e85b42665 add .sec scale to measurements; humanize for metric tags 2020-11-24 17:52:37 +01:00
Anton Evangelatov
52da5acc96 go mod replace lotus with ../../ 2020-11-23 20:43:18 +01:00
Anton Evangelatov
844c0a3527 bump docker golang version to 1.15:buster 2020-11-23 20:00:20 +01:00
Anton Evangelatov
bf500711b3 update Dockerfiles; integrate lotus-soup and graphsync in Lotus CI; update go.mod and references 2020-11-06 21:04:11 +01:00