Commit Graph

412 Commits

Author SHA1 Message Date
Aayush Rajasekaran
7b4c657e90 Add v6 actors 2021-09-21 14:23:14 -04:00
Łukasz Magiera
95b128b7bc chain: Cleanup consensus logic 2021-09-02 18:09:37 +02:00
Łukasz Magiera
b0f57d74e5
Merge pull request #7000 from filecoin-project/feat/refactor-events
Refactor events subsystem
2021-08-31 12:02:22 +02:00
Steven Allen
1cf556c3a2 feat: expose ChainGetPath on the gateway 2021-08-30 16:43:21 -07:00
Steven Allen
8e52bf30c1 test: re-enable disabled tests
1. Rewrite VRF test to actually test win counts, and enable it.
2. Stop skipping some tests that now pass.
2021-08-27 15:43:55 -07:00
Aarsh Shah
d7076778e2
integrate DAG store and CARv2 in deal-making (#6671)
This commit removes badger from the deal-making processes, and
moves to a new architecture with the dagstore as the cental
component on the miner-side, and CARv2s on the client-side.

Every deal that has been handed off to the sealing subsystem becomes
a shard in the dagstore. Shards are mounted via the LotusMount, which
teaches the dagstore how to load the related piece when serving
retrievals.

When the miner starts the Lotus for the first time with this patch,
we will perform a one-time migration of all active deals into the
dagstore. This is a lightweight process, and it consists simply
of registering the shards in the dagstore.

Shards are backed by the unsealed copy of the piece. This is currently
a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so
when it's time to acquire a shard to serve a retrieval, the unsealed
CARv1 is joined with its index (safeguarded by the dagstore), to form
a read-only blockstore, thus taking the place of the monolithic
badger.

Data transfers have been adjusted to interface directly with CARv2 files.
On inbound transfers (client retrievals, miner storage deals), we stream
the received data into a CARv2 ReadWrite blockstore. On outbound transfers
(client storage deals, miner retrievals), we serve the data off a CARv2
ReadOnly blockstore.

Client-side imports are managed by the refactored *imports.Manager
component (when not using IPFS integration). Just like it before, we use
the go-filestore library to avoid duplicating the data from the original
file in the resulting UnixFS DAG (concretely the leaves). However, the
target of those imports are what we call "ref-CARv2s": CARv2 files placed
under the `$LOTUS_PATH/imports` directory, containing the intermediate
nodes in full, and the leaves as positional references to the original file
on disk.

Client-side retrievals are placed into CARv2 files in the location:
`$LOTUS_PATH/retrievals`.

A new set of `Dagstore*` JSON-RPC operations and `lotus-miner dagstore`
subcommands have been introduced on the miner-side to inspect and manage
the dagstore.

Despite moving to a CARv2-backed system, the IPFS integration has been
respected, and it continues to be possible to make storage deals with data
held in an IPFS node, and to perform retrievals directly into an IPFS node.

NOTE: because the "staging" and "client" Badger blockstores are no longer
used, existing imports on the client will be rendered useless. On startup,
Lotus will enumerate all imports and print WARN statements on the log for
each import that needs to be reimported. These log lines contain these
messages:

- import lacks carv2 path; import will not work; please reimport
- import has missing/broken carv2; please reimport

At the end, we will print a "sanity check completed" message indicating
the count of imports found, and how many were deemed broken.

Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>

Co-authored-by: Raúl Kripalani <raul@protocol.ai>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
2021-08-16 23:34:32 +01:00
Łukasz Magiera
c094aa82ec commit batch: AggregateAboveBaseFee config 2021-07-01 13:33:54 +02:00
Aayush Rajasekaran
35f76f58df
Merge pull request #6309 from filecoin-project/feat/better_mining_projection_and_accuracy
Adjust various CLI display ratios to arbitrary precision
2021-06-16 17:48:23 -04:00
Łukasz Magiera
1e4456138e Merge master into feat/nv13 2021-05-27 12:28:20 +02:00
Peter Rabbitson
c2e5a837e6 Adjust various CLI display ratios to arbitrary precision
Originally the deviations from using float64 were insignificant, but at
exabyte scale they start to show up. Cleanup all displays, and clarify
the expectation text, adding an extra 99.9% probability calculator to
`lotus-miner info`
2021-05-25 14:09:01 +02:00
Łukasz Magiera
2afe725933 Get PreCommitting to work 2021-05-17 22:02:23 +02:00
Jakub Sztandera
cbfb4770fd
Add function to display nanoFIL
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-05-07 15:30:05 +02:00
Aayush Rajasekaran
d794b49df3 Use EmptyTSK where appropriate 2021-04-29 01:00:03 -04:00
Aayush Rajasekaran
f1ded63d60 v4 specs-actors integration, nv12 migration 2021-04-27 02:02:44 -04:00
Łukasz Magiera
26399dba70 Update markets, cbor-gen with soft map decoding 2021-02-19 20:11:43 +01:00
Peter Rabbitson
ac616eb340 Better document the block-header fields 2021-01-27 20:52:18 +01:00
Steven Allen
82eee15955 fix state tree version comment
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2021-01-21 15:21:19 -08:00
Steven Allen
9d2c430138 remove abstract map constructor/loader
Different maps have different parameters now so we just construct/load them manually where needed.
2021-01-21 15:21:19 -08:00
He Weidong
ab9b3b6851 enable parse and shorten negative FIL values 2021-01-09 17:48:36 +08:00
Łukasz Magiera
bc14d0820f fix lint 2020-12-01 20:00:55 +01:00
Łukasz Magiera
b28ed5887c gofmt 2020-12-01 19:23:43 +01:00
Łukasz Magiera
be9f7b88b2 uint64 is too small 2020-12-01 19:15:39 +01:00
Łukasz Magiera
00e5ace818 Print basefee in miner info 2020-12-01 18:56:43 +01:00
Aayush Rajasekaran
7838752352 Avoid sending messages to the zero BLS address 2020-11-17 01:42:11 -05:00
Lucas Molas
3ec25cbece
restrict parse fil string length (#4780) 2020-11-09 22:12:32 -03:00
Steven Allen
9d15a08d27 correctly mark new actors as "important" 2020-10-21 12:18:38 -07:00
Steven Allen
4e730b5ec8 port to v2 imports 2020-10-21 12:16:23 -07:00
Łukasz Magiera
698b49b7ef Improve gas defaults 2020-10-15 02:46:47 +02:00
Jakub Sztandera
8987defb9d
Fix BLS message ChainLength add more detail to ValidForBlockInclusion
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-10-13 20:02:42 +02:00
Łukasz Magiera
6c08120628 types: Add CID fields to messages in json marshalers 2020-10-12 20:46:47 +02:00
Jakub Sztandera
2b21fdef33
Refactor from crypto.SigType to types.KeyType
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-10-12 00:07:53 +02:00
Łukasz Magiera
fdaa9c14ad Merge remote-tracking branch 'origin/master' into feat/signing-backends 2020-10-09 20:36:51 +02:00
Dirk McCormick
7b1bec91ed feat: add tests for gateway ChainGetTipsetByHeight 2020-10-09 11:45:16 +02:00
Łukasz Magiera
7d2f06cfbd Fix WalletSign in tests 2020-10-09 01:34:48 +02:00
Łukasz Magiera
8783c7434e Merge remote-tracking branch 'origin/master' into feat/signing-backends 2020-10-09 00:50:41 +02:00
Łukasz Magiera
7332a39f00 Merge remote-tracking branch 'origin/master' into fix/balances-audit-no-suffix 2020-10-08 13:58:40 +02:00
whyrusleeping
cd9af278b4 more info in audit outputs 2020-10-07 17:01:01 -07:00
whyrusleeping
8f4911ac0d dont print fil suffix in lotus shed audit balances 2020-10-07 16:00:52 -07:00
Steven Allen
e803cf151f introduce separate state-tree versions
Instead of versioning the state tree along with the actors, version it
separately. This structure may not upgrade every time we update actors.
2020-10-07 12:04:32 -07:00
Aayush Rajasekaran
be9d23b329 Centralize some params in builtin 2020-09-29 02:25:43 -04:00
Steven Allen
233d8a9b72 update even more imports 2020-09-28 14:25:58 -07:00
zgfzgf
04876c663e modify tipset Equals 2020-09-27 10:17:06 +08:00
zgfzgf
70071e273d optimize tipset Equals func 2020-09-27 09:58:37 +08:00
Steven Allen
d33dd4f7bc more renames 2020-09-21 16:01:29 -07:00
Steven Allen
3f0106cfe5 migrate lotus-shed/genesis-verify to actor abstraction 2020-09-21 15:18:30 -07:00
Łukasz Magiera
82b95e34b7 cbor gen 2020-09-17 17:37:16 +02:00
Steven Allen
cc4d5306eb Progress 2020-09-14 15:43:12 -07:00
Aayush Rajasekaran
d678fe4bfa Fix tests 2020-09-07 15:48:42 -04:00
Aayush Rajasekaran
39755a294a Update to specs v0.9.6 2020-09-07 15:48:41 -04:00
vyzo
e24a29b146 fix tests that touch the mpool; mock block must have a ParentBaseFee 2020-09-06 11:55:43 +03:00