cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
16,974 Commits 24 Branches 342 Tags 314 MiB
347d92b29c
Commit Graph

3832 Commits

Author SHA1 Message Date
Łukasz Magiera
30fccaa0bd fix lint 2021-09-02 18:45:18 +02:00
Łukasz Magiera
95b128b7bc chain: Cleanup consensus logic 2021-09-02 18:09:37 +02:00
Steven Allen
6a02237f6f
Merge pull request #7210 from filecoin-project/fix/fork-check 
fix: correctly handle null blocks when detecting an expensive fork
 2021-08-31 11:01:45 -07:00
Steven Allen
91da70fb7d fix: comment spelling 2021-08-31 10:37:51 -07:00
Łukasz Magiera
daaa725e3b sectors expired: Handle precomitted and unproven sectors correctly 2021-08-31 16:24:29 +02:00
Łukasz Magiera
b0f57d74e5
Merge pull request #7000 from filecoin-project/feat/refactor-events 
Refactor events subsystem
 2021-08-31 12:02:22 +02:00
Steven Allen
d113813b33
Merge pull request #7211 from filecoin-project/test/audit-disabled-tests 
test: re-enable disabled tests
 2021-08-30 17:03:03 -07:00
dirkmc
1da59fa2fe fix events API timeout handling for nil blocks (#7184) 2021-08-30 17:00:24 -07:00
Steven Allen
1cf556c3a2 feat: expose ChainGetPath on the gateway 2021-08-30 16:43:21 -07:00
Steven Allen
003eae81ce fix: address review 2021-08-30 16:43:21 -07:00
Steven Allen
f518e34131 fix: atomically get head when registering an observer 
This lets us always call check (accurately).
 2021-08-30 16:43:21 -07:00
Steven Allen
82ac0a24a0 test: improve chain event tests 2021-08-30 16:43:21 -07:00
Steven Allen
3846170302 refactor events system 2021-08-30 16:43:21 -07:00
Steven Allen
a875e9ba73 fix: check parents when adding tipsets to the "cache" 2021-08-30 16:43:21 -07:00
Steven Allen
43bbde1e6b fix: close chain head subscription when the reader is slow 
The reader can just re-subscribe when they're ready to catch up. This
prevents a slow reader from bogging down the entire system.
 2021-08-30 16:43:21 -07:00
Steven Allen
14754f1b18 chore: dedup datastore import 2021-08-30 16:43:21 -07:00
Steven Allen
08207912a5 fix: check if at genesis when testing for expensive forks in Call 
Otherwise, we could try to lookup the parent of the block at height 0.
 2021-08-30 16:23:13 -07:00
Steven Allen
e6902304d5 fix: address expensive fork review 2021-08-30 16:20:23 -07:00
Dirk McCormick
7b7a5b0b21 revert: changes to OnDealExpiredOrChanged in #5431 #7201 2021-08-30 10:42:41 +02:00
Steven Allen
8e52bf30c1 test: re-enable disabled tests 
1. Rewrite VRF test to actually test win counts, and enable it.
2. Stop skipping some tests that now pass.
 2021-08-27 15:43:55 -07:00
Steven Allen
165735d01a fix: correctly handle null blocks when detecting an expensive fork 
Also improve/fix documentation to reflect the _actual_ tipset that's
passed into upgrades.

And update some comments.

fixes #7192
 2021-08-27 15:04:01 -07:00
dirkmc
77a19774cf fix events API timeout handling for nil blocks (#7184) 2021-08-27 09:05:00 +02:00
Aayush Rajasekaran
bdd1c36802 Incoming: improve a log message 2021-08-25 12:26:10 -04:00
Łukasz Magiera
927ef041f8 miner: Command to list expired sectors 2021-08-23 11:27:34 -07:00
Aarsh Shah
d7076778e2
integrate DAG store and CARv2 in deal-making (#6671) 
This commit removes badger from the deal-making processes, and
moves to a new architecture with the dagstore as the cental
component on the miner-side, and CARv2s on the client-side.

Every deal that has been handed off to the sealing subsystem becomes
a shard in the dagstore. Shards are mounted via the LotusMount, which
teaches the dagstore how to load the related piece when serving
retrievals.

When the miner starts the Lotus for the first time with this patch,
we will perform a one-time migration of all active deals into the
dagstore. This is a lightweight process, and it consists simply
of registering the shards in the dagstore.

Shards are backed by the unsealed copy of the piece. This is currently
a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so
when it's time to acquire a shard to serve a retrieval, the unsealed
CARv1 is joined with its index (safeguarded by the dagstore), to form
a read-only blockstore, thus taking the place of the monolithic
badger.

Data transfers have been adjusted to interface directly with CARv2 files.
On inbound transfers (client retrievals, miner storage deals), we stream
the received data into a CARv2 ReadWrite blockstore. On outbound transfers
(client storage deals, miner retrievals), we serve the data off a CARv2
ReadOnly blockstore.

Client-side imports are managed by the refactored *imports.Manager
component (when not using IPFS integration). Just like it before, we use
the go-filestore library to avoid duplicating the data from the original
file in the resulting UnixFS DAG (concretely the leaves). However, the
target of those imports are what we call "ref-CARv2s": CARv2 files placed
under the `$LOTUS_PATH/imports` directory, containing the intermediate
nodes in full, and the leaves as positional references to the original file
on disk.

Client-side retrievals are placed into CARv2 files in the location:
`$LOTUS_PATH/retrievals`.

A new set of `Dagstore*` JSON-RPC operations and `lotus-miner dagstore`
subcommands have been introduced on the miner-side to inspect and manage
the dagstore.

Despite moving to a CARv2-backed system, the IPFS integration has been
respected, and it continues to be possible to make storage deals with data
held in an IPFS node, and to perform retrievals directly into an IPFS node.

NOTE: because the "staging" and "client" Badger blockstores are no longer
used, existing imports on the client will be rendered useless. On startup,
Lotus will enumerate all imports and print WARN statements on the log for
each import that needs to be reimported. These log lines contain these
messages:

- import lacks carv2 path; import will not work; please reimport
- import has missing/broken carv2; please reimport

At the end, we will print a "sanity check completed" message indicating
the count of imports found, and how many were deemed broken.

Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>

Co-authored-by: Raúl Kripalani <raul@protocol.ai>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
 2021-08-16 23:34:32 +01:00
Steven Allen
3235dbfaee fix genesis for actors v2 2021-08-11 13:30:15 -07:00
Aayush Rajasekaran
a52352b13a PreCommitPolicy: Don't try to align expirations on proving period boundaries 2021-08-11 12:44:12 -04:00
ZenGround0
b914e95f34 revert pricelist by version to pricelist by epoch 2021-08-11 09:49:23 -04:00
Steven Allen
65e3540d03
Merge pull request #7007 from filecoin-project/fix/panic-less 
Dont panic if input is bad
 2021-08-10 16:22:56 -07:00
ZenGround0
234030218c Fix actor policy codegen 2021-08-10 13:32:00 -04:00
ZenGround0
6da26dcefc Remove network version panic 2021-08-10 13:07:30 -04:00
whyrusleeping
bfd69f5381 WIP: dont panic if input is bad 2021-08-09 12:17:54 +02:00
Steven Allen
5a321f970e fix: vet actors shims 
1. Check error in EraseAllUnproven
2. Avoid dead code.
 2021-08-06 14:56:29 -07:00
Steven Allen
fe74311209
Merge branch 'master' into feat/print-runtime-send-update 2021-07-28 18:55:38 -07:00
frrist
256d12773b polish(errors): better state tree errors 2021-07-28 17:50:22 -07:00
ZenGround0
e09a25ca68 Remove unnecessary ret log 2021-07-28 18:26:29 -04:00
ZenGround0
a968784908 fix new panics in invoker_test.go 2021-07-28 18:26:29 -04:00
ZenGround0
0202192fc0 Log more call context during errors 2021-07-28 18:26:29 -04:00
Łukasz Magiera
389f71251c Remove townhall 2021-07-28 17:58:46 +02:00
Łukasz Magiera
c57c20c6e4 fix lint 2021-07-27 15:58:18 +02:00
Łukasz Magiera
05a3710288 chainstore: Fix test build 2021-07-27 15:53:47 +02:00
Łukasz Magiera
ae63a4b33b fix lotus-sim build 2021-07-27 15:49:01 +02:00
Łukasz Magiera
ce82f2827b stmgr: Put actor state accessors in actors.go 2021-07-27 15:43:11 +02:00
Łukasz Magiera
e7d73cbe56 vm: Remove unused ActorBalance 2021-07-27 15:34:39 +02:00
Łukasz Magiera
9bd312881d VMSys doesn't belong in chainstore 2021-07-27 15:30:23 +02:00
Łukasz Magiera
ca1cd741c3 stmgr: Split stmgr.go into smaller subfiles 2021-07-27 15:15:50 +02:00
Łukasz Magiera
50b217817e stmgr: Split upgrade code from upgrade runtime 2021-07-27 14:48:30 +02:00
Łukasz Magiera
6ba42e3e6b chainstore: refactor store.go into more subfiles 2021-07-27 14:41:36 +02:00
Peter Rabbitson
ed387b43cc Bump go-multihash, adjust test for supported version 2021-07-26 17:50:19 +02:00
Aayush Rajasekaran
104626acb7 ValidateBlock: Assert that block header height's are greater than their parents 2021-07-26 11:09:23 -04:00
Łukasz Magiera
f9752d8a0a
Merge branch 'master' into feat/compact-sectors-numbers-cmd 2021-07-22 19:00:33 +02:00
Łukasz Magiera
0656b62176 Merge remote-tracking branch 'origin/master' into feat/compact-sectors-numbers-cmd 2021-07-22 16:04:25 +02:00
ZenGround0
d6a8a7aeb8 Remove validation because of sync TestDrandNull test 2021-07-22 09:49:47 -04:00
ZenGround0
4cd05fa39c Lint 2021-07-22 09:49:47 -04:00
ZenGround0
ed844c5283 Use current ntwk version in mpool message check 2021-07-22 09:49:47 -04:00
ZenGround0
9fc4a25bd1 Fix tests with bad network version refs 2021-07-22 09:49:47 -04:00
ZenGround0
cbc07cb939 Add 6.5 to VersionForNetwork commit updated docs 2021-07-22 09:49:47 -04:00
ZenGround0
545cc723e0 Fix test 2021-07-22 09:49:47 -04:00
ZenGround0
47b5afa84b Add version six and a half 2021-07-22 09:49:47 -04:00
ZenGround0
f49a8248f0 PriceListByVersion 2021-07-22 09:49:47 -04:00
Łukasz Magiera
3006dda786
Merge pull request #6760 from ipfs-force-community/feat/cache_blkmsg_events 
cache loaded block messages
 2021-07-22 13:54:51 +02:00
vyzo
839b00ab40 rename messagepool ProtectMessages to ForEachPendingMessage 2021-07-20 09:23:36 +03:00
vyzo
ebbaf23af8 support out-of-chain reference protection 2021-07-20 09:02:40 +03:00
hunjixin
eef3fd5d7a cache loaded block messages 2021-07-19 15:13:53 +08:00
Steven Allen
cf81c897df test: handle null blocks in TestForkRefuseCall 
Otherwise, this fails on rare occasions.
 2021-07-14 17:41:24 -07:00
Anton Evangelatov
d89ddb9315 resolve conflicts 2021-07-12 11:34:37 +02:00
Łukasz Magiera
79b0bfb272 policy: Add a docstring to SetProviderCollateralSupplyTarget 2021-07-08 18:56:03 +02:00
Łukasz Magiera
2dc27d6ab4 itests: Fix deal provider collateral flakiness 2021-07-07 19:41:46 +02:00
Anton Evangelatov
6b014f57e5 pass Subsystems to StorageMiner option; add enableLibp2p bool in Settings 2021-07-07 13:56:37 +02:00
Anton Evangelatov
ff2772a58c resolved conflicts 2021-07-06 16:00:41 +02:00
Łukasz Magiera
97207df705
Merge pull request #6639 from filecoin-project/frrist/context-in-statetree-diff 
polish(statetree): accept a context in statetree diff for timeouts
 2021-07-05 17:30:31 +02:00
Anton Evangelatov
4be0a7a215 resolve conflicts with master 2021-07-05 13:13:32 +02:00
Łukasz Magiera
c094aa82ec commit batch: AggregateAboveBaseFee config 2021-07-01 13:33:54 +02:00
frrist
d02a2a2cfa polish(statetree): accept a context in statetree diff for timeouts 
- this operation can take a while, this changes gives users the option
to abort if it takes "too long".
 2021-06-30 16:40:43 -07:00
Anton Evangelatov
4f0a96c9c7 resolved conflicts 2021-06-30 13:16:52 +02:00
Anton Evangelatov
9ec7cd7239 enable libp2p options 2021-06-30 12:55:43 +02:00
Łukasz Magiera
c10d99e3fd multiwallet: Don't fail if key is found in any wallet 2021-06-25 12:17:53 +02:00
Łukasz Magiera
ebd746cdec
Merge pull request #6406 from filecoin-project/feat/lotus-sim 2021-06-24 18:20:43 +02:00
Steven Allen
d6abcff63c
fix(lotus-sim): apply code review from magik6k 
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
 2021-06-21 09:05:48 -07:00
Łukasz Magiera
b37a66c7c9 Merge remote-tracking branch 'origin/release/v1.10.0' into chore/merge-1.10 2021-06-21 17:03:03 +02:00
Steven Allen
eb0a15faf0 fix(genesis): set initial balances to 0 2021-06-18 15:44:37 -07:00
Steven Allen
b30b5dd629 fix: move actors changes to template files 2021-06-18 15:44:37 -07:00
Steven Allen
eb2b706156 chore: fix lint errors in simulation 2021-06-18 15:44:37 -07:00
Steven Allen
22267eb45d feat(lotus-sim): split info command file 2021-06-18 15:44:36 -07:00
Steven Allen
8a215df46b fix(statetree): make StateTree.ForEach take layers into account 
This likely isn't used anywhere, but this _should_ take layers into
account (and I kind of just assumed it did).
 2021-06-18 15:44:36 -07:00
Steven Allen
e7b1e09ade feat(multisig): expose ApproveReturn 2021-06-18 15:44:35 -07:00
Steven Allen
0075abea5e fix(vm): always specify an ActorErr when ApplyMessage fails. 
This case shouldn't actually happen, but we might as well be consistent.
 2021-06-18 15:44:35 -07:00
Steven Allen
e2f5c494b0 feat: implement lotus-sim 2021-06-18 15:44:34 -07:00
Łukasz Magiera
2c2c37a138 chainstore: Don't take heaviestLk with backlogged reorgCh 2021-06-18 19:18:51 +02:00
Steven Allen
526674cefa fix: pick the correct partitions-per-post limit 2021-06-16 16:56:30 -07:00
Steven Allen
653b1d8231 fix: pick the correct partitions-per-post limit 2021-06-16 16:53:52 -07:00
Aayush Rajasekaran
35f76f58df
Merge pull request #6309 from filecoin-project/feat/better_mining_projection_and_accuracy 
Adjust various CLI display ratios to arbitrary precision
 2021-06-16 17:48:23 -04:00
Aayush Rajasekaran
b9f864ba9e Expand on Drand change testing 2021-06-16 14:17:55 -04:00
Peter Rabbitson
429419f210 Forgotten deadcode 2021-06-16 13:38:39 -04:00
Peter Rabbitson
3acd846dcd Fix logging around mineOne 
- A nil MiningBaseInfo is *NOT* unexpected: it happens when one is in penalty
  https://github.com/filecoin-project/lotus/blob/v1.9.0/chain/stmgr/utils.go#L500-L502

- Remove the log from IsRoundWinner(): all we care about is the randbase epoch
 2021-06-16 13:38:35 -04:00
Aayush Rajasekaran
520a0091f5 Merge branch 'releases' into release/v1.10.0 2021-06-15 17:40:36 -04:00
Łukasz Magiera
0514f38dc4 Merge remote-tracking branch 'origin/master' into chore/merge-1.10 2021-06-14 13:27:53 +02:00
Łukasz Magiera
534badad2a mpool: Add more metrics 2021-06-11 13:19:26 +02:00
wangchao
626d482990 correct the change of message size limit 2021-06-09 18:05:16 -04:00
Jakub Sztandera
18043810c0 Create MaxMessageSize constant 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-06-09 14:24:42 -04:00
Jakub Sztandera
0cd0faa6fb Increase message size limit 
Add test

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-06-09 14:24:38 -04:00
Aayush Rajasekaran
dede1b4758 Merge branch 'release/v1.10.0' into asr/merge-release 2021-06-09 14:05:41 -04:00
Aayush Rajasekaran
b0c9dd49f0 Fund miners with the aggregate fee when ProveCommitting 2021-06-09 13:08:54 -04:00
wangchao
4c87818de1 correct the change of message size limit 2021-06-09 15:19:35 +08:00
Jakub Sztandera
39bab148b8
Create MaxMessageSize constant 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-06-08 16:07:43 +02:00
Jakub Sztandera
4c605c349f
Increase message size limit 
Add test

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-06-08 15:20:10 +02:00
Łukasz Magiera
c98b6f6408
Merge pull request #6389 from filecoin-project/frrist/execmonitor-interface 
polish(stmgr): define ExecMonitor for message application callback
 2021-06-08 12:30:52 +02:00
Peter Rabbitson
29d7561dd1 Fix logging of stringified CIDs double-encoded in hex 2021-06-08 00:05:48 +02:00
frrist
0519c77c24 polish(stmgr): define ExecMonitor for message applicaiton callback 2021-06-07 11:03:18 -07:00
Aayush Rajasekaran
c66d66dfcb Fix state manager::Call() 2021-06-03 17:09:42 -04:00
Aayush Rajasekaran
a41a1cbd93 Gate runtime's GetRandomnessFromBeacon on HyperdriveHeight, not network version 2021-06-03 15:47:49 -04:00
Łukasz Magiera
c5797482b2 Revert CallWithGas hack 2021-06-03 18:51:01 +02:00
Łukasz Magiera
78c128f6a3 chain: Better logging in sync tests 2021-06-03 10:42:26 +02:00
Łukasz Magiera
ee21351aa0 statetree: Add missing version defs 2021-06-01 21:16:01 +02:00
Łukasz Magiera
a676da8a7e Merge remote-tracking branch 'origin/master' into feat/nv13-1.11 2021-06-01 21:07:04 +02:00
Łukasz Magiera
dc642d0b7b Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11 2021-06-01 21:06:58 +02:00
Łukasz Magiera
fa8beeac2b
Merge pull request #6371 from filecoin-project/fix/robustify-commit-batcher 
Fix tests
 2021-06-01 21:01:03 +02:00
Aayush Rajasekaran
39f2246ae6 Use build.UpgradeHyperDriveHeight directly when sourcing randomness in VM 2021-06-01 14:43:46 -04:00
Aayush Rajasekaran
c8cef1cb7e Fix chain/gen randomness getting (test only) 2021-06-01 13:44:08 -04:00
Łukasz Magiera
66c1554670 vm syscalls: fix typo 2021-06-01 19:23:12 +02:00
Aayush Rajasekaran
49fce48c3e Tweak CallVM to use current epoch, not future epoch 2021-06-01 12:43:16 -04:00
Łukasz Magiera
5a9e6c8142
Merge pull request #6364 from filecoin-project/asr/resolve-messages 
Improve address resolution for messages
 2021-06-01 14:38:31 +02:00
Aayush Rajasekaran
2ab24b358d Fix supported proof type manipulations for v5 actors 2021-05-31 18:50:29 -04:00
Mimir
ee508120d9
Typo fix in error message: "pubusb" -> "pubsub" 2021-05-31 15:30:10 -07:00
Aayush Rajasekaran
621e4eab0d Address review 2021-05-31 18:13:23 -04:00
Aayush Rajasekaran
9ceee6028b More tweaking of context handling in the messagepool 2021-05-31 18:13:23 -04:00
Aayush Rajasekaran
183c12db25 Make mempool reject ID addresses that are not reorg-stable 2021-05-31 18:13:23 -04:00
Aayush Rajasekaran
ed93d0725f Protect mp.localAddrs and mp.pending behind helper functions 2021-05-31 18:13:23 -04:00
Aayush Rajasekaran
1f03a618f9 Plumb contexts through 2021-05-31 18:13:23 -04:00
Aayush Rajasekaran
8d991283f4 Resolve to ID addresses when handling message selection 2021-05-31 18:13:23 -04:00
Łukasz Magiera
3671f2a6ff fix 2k build 2021-05-31 21:52:23 +02:00
Łukasz Magiera
ffa47659a1 Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11 2021-05-31 21:38:34 +02:00
Łukasz Magiera
c3e8eddb9b Merge remote-tracking branch 'origin/master' into feat/nv13-1.11 2021-05-31 21:24:56 +02:00
Łukasz Magiera
2a1b35e694
Merge pull request #6361 from filecoin-project/asr/fip-0015 
Implement FIP-0015
 2021-05-31 20:58:21 +02:00
Aayush Rajasekaran
61554cf3e0 Update to latest actors 2021-05-31 14:47:41 -04:00
Aayush Rajasekaran
7fca1c1ee7 Implement FIP-0015 2021-05-31 14:40:54 -04:00
Łukasz Magiera
8003a8a2d0 events: Fix handling of multiple matched events per epoch 2021-05-31 20:17:50 +02:00
Łukasz Magiera
3ea39f76e1 events: Fix handling of multiple matched events per epoch 2021-05-30 17:20:14 +02:00
Łukasz Magiera
dd79c2c41a
Merge pull request #6347 from filecoin-project/feat/agg-gas-price 
Introduce gas prices for aggregate verifications
 2021-05-28 12:40:16 +02:00
Aayush Rajasekaran
10b931312b Fix edgecase in tipset skipcache 2021-05-27 13:44:37 -04:00
Jakub Sztandera
cb59daf3c1
Introduce gas prices for aggregate verifications 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-27 17:51:01 +02:00
Łukasz Magiera
1e4456138e Merge master into feat/nv13 2021-05-27 12:28:20 +02:00
Aayush Rajasekaran
21b4741e30 Fix randomness fetching around null blocks 2021-05-26 21:06:06 -04:00
Aayush Rajasekaran
7714537239 Allow starting networks from arbitrary actor versions 2021-05-26 12:48:14 -04:00
Łukasz Magiera
43c62f4406
Revert "Allow starting networks from arbitrary actor versions" 2021-05-26 12:33:08 +02:00
Aayush Rajasekaran
cf574ca9a1 Allow starting networks from arbitrary actor versions 2021-05-25 19:30:20 -04:00
Łukasz Magiera
f5409845b5 Some review addressing 2021-05-25 16:07:45 +02:00
Peter Rabbitson
c2e5a837e6 Adjust various CLI display ratios to arbitrary precision 
Originally the deviations from using float64 were insignificant, but at
exabyte scale they start to show up. Cleanup all displays, and clarify
the expectation text, adding an extra 99.9% probability calculator to
`lotus-miner info`
 2021-05-25 14:09:01 +02:00
Peter Rabbitson
aed7017ab2 Forgotten deadcode 2021-05-21 15:30:08 +02:00
Peter Rabbitson
308d1e9b7c Fix logging around mineOne 
- A nil MiningBaseInfo is *NOT* unexpected: it happens when one is in penalty
  https://github.com/filecoin-project/lotus/blob/v1.9.0/chain/stmgr/utils.go#L500-L502

- Remove the log from IsRoundWinner(): all we care about is the randbase epoch
 2021-05-21 15:00:21 +02:00
Łukasz Magiera
678812f35b gofmt, lint 2021-05-20 12:05:04 +02:00
raulk
996feda1f7
typo. 
Co-authored-by: dirkmc <dirkmdev@gmail.com>
 2021-05-19 15:08:14 +01:00
Łukasz Magiera
a5677d1b7a ffiwrapper: Separate Prover interface 2021-05-19 15:20:23 +02:00
Łukasz Magiera
5112b9fe2b Lower default batch slack 2021-05-18 21:04:47 +02:00
Łukasz Magiera
81b5d8c671 Make things build with both batchers 2021-05-18 16:53:49 +02:00
Łukasz Magiera
c7ba083fa4 Import precommit batcher 2021-05-18 16:51:06 +02:00
Jennifer Wang
5af3af5a22 Merge branch 'release/v1.9.0' into releases 2021-05-17 16:16:34 -04:00
Łukasz Magiera
2afe725933 Get PreCommitting to work 2021-05-17 22:02:23 +02:00
Łukasz Magiera
578bef4f83 Update gen 2021-05-17 20:56:28 +02:00
Łukasz Magiera
6278bdc69a Make things build 2021-05-17 20:47:41 +02:00
Raúl Kripalani
c77f8fb382 adopt clearer method names; fix typo. 2021-05-15 15:07:02 +01:00
Raúl Kripalani
5daacc0f07 docs: add docs to chain store methods. 2021-05-13 13:08:52 +01:00
Łukasz Magiera
506f39b294 WIP: Integrate FIP0013 2021-05-11 22:10:29 -04:00
Aayush Rajasekaran
bf10b051ff
Merge pull request #6108 from filecoin-project/feat/actors-codegen 
wip actor wrapper codegen
 2021-05-10 20:12:57 -04:00
Aayush Rajasekaran
5f821cc733 Generate builtin.go 2021-05-10 19:53:07 -04:00
Aayush Rajasekaran
60446b46c8 Generate policy.go 2021-05-10 19:46:44 -04:00
Łukasz Magiera
cd2b959a88 wip partial codegen 2021-05-10 19:46:44 -04:00
Łukasz Magiera
a80259d986 Actor upgrade checklist 2021-05-10 19:46:44 -04:00
Łukasz Magiera
94d3c8bed7 fix lint 2021-05-10 19:46:44 -04:00
Łukasz Magiera
c6cebb448f Cleanup actor adapter templates 2021-05-10 19:46:44 -04:00
Łukasz Magiera
715b3dccec Fix error message 2021-05-10 19:46:44 -04:00
Łukasz Magiera
1a84bd5842 chain actors: codegen templates for all actors 2021-05-10 19:46:44 -04:00
Łukasz Magiera
0db070779f wip actor wrapper codegen 2021-05-10 19:46:44 -04:00
Aayush Rajasekaran
b5da2655dc Introduce v5 actors 2021-05-10 19:44:28 -04:00
Łukasz Magiera
63189cd81d Generate wrappers for new actor versions 2021-05-09 21:04:50 -04:00
Jakub Sztandera
18cbdcfc81
Disable checks API on Lotus Lite 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 17:19:17 +02:00
Jakub Sztandera
d777680449
Fix mpool.GetActor for lite node 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 15:30:06 +02:00
Jakub Sztandera
cbfb4770fd
Add function to display nanoFIL 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 15:30:05 +02:00
Jakub Sztandera
8d75da1629
Use MessagePrototype for check API 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 15:30:05 +02:00
Jakub Sztandera
7535c5bb53
Add mpool manage command 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 15:30:05 +02:00
Jakub Sztandera
87df73a455
Fix get nonce check 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 15:30:05 +02:00
Jakub Sztandera
86e90dc6f1
Message sending UI 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 15:30:04 +02:00
vyzo
d782250aba
implement MessagePool.CheckReplaceMessages 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 15:30:04 +02:00
vyzo
91e774063e
implement MessagePool.CheckMessages 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 15:30:04 +02:00
Peter Rabbitson
de60229957 mining lookback is effectively a constant - make it so 2021-05-06 10:18:08 +02:00
Peter Rabbitson
b1db3fee78 Log more ComputeVRF() inputs as per Why's request 2021-05-06 10:18:08 +02:00
Peter Rabbitson
72134ff458 Add a mining-heartbeat INFO line at every epoch 2021-05-06 10:18:08 +02:00
Peter Rabbitson
26841aa094 mining lookback is effectively a constant - make it so 2021-05-05 23:23:01 +02:00
Peter Rabbitson
3b0fc01e82 Log more ComputeVRF() inputs as per Why's request 2021-05-05 23:23:01 +02:00
Peter Rabbitson
14f6d170b2 Add a mining-heartbeat INFO line at every epoch 2021-05-05 23:23:01 +02:00
Aayush Rajasekaran
c074031fa1
Merge pull request #6167 from filecoin-project/asr/state-tree-version 
Allow creation of state tree v3s
 2021-05-03 19:16:44 -04:00
Aayush Rajasekaran
2a90de30bd
Merge pull request #6164 from filecoin-project/fix/drand-cache 
drand: fix beacon cache
 2021-05-01 12:21:38 -04:00
Aayush Rajasekaran
4193235b59 Allow creation of state tree v3s 2021-04-30 22:41:02 -04:00
Łukasz Magiera
04c71b2c6b mpool: Cleanup pre-nv12 selection logic 2021-04-30 10:10:42 +02:00
Łukasz Magiera
ab811e2e19 drand: fix beacon cache 2021-04-30 09:35:38 +02:00
Steven Allen
eb10918470
Merge pull request #6107 from filecoin-project/feat/checkpoint-sync 
feat: allow checkpointing to forks
 2021-04-29 16:30:52 -07:00
Aayush Rajasekaran
e6779b0b6f Add a command to get the fees of a deal 2021-04-29 13:29:17 -04:00
Steven Allen
f983fa8cd5
Merge pull request #6134 from filecoin-project/asr/emptytsk 
Use EmptyTSK where appropriate
 2021-04-29 09:35:43 -07:00
Steven Allen
4867425685 fix: use the parent state when listing actors 
To be consistent with other commands.
 2021-04-28 23:35:27 -07:00
Łukasz Magiera
4b04cac219
Merge pull request #4938 from filecoin-project/asr/total-power 
Return total power when GetPowerRaw doesn't find miner claim
 2021-04-29 07:45:43 +02:00
Aayush Rajasekaran
d794b49df3 Use EmptyTSK where appropriate 2021-04-29 01:00:03 -04:00
Steven Allen
8f309b214b chain: move checkpoint logic into chainstore 
That way, checkpoints can be enforced by the chainstore, removing a
potential race where an in-progress sync of a fork could bypass a sync
checkpoint.
 2021-04-28 15:06:29 -07:00
Steven Allen
0ad51f8a14 test(sync): fix tipset check and re-enable checkpoint tests 
We need to wait until the node _has_ the tipset, not until it doesn't.

This probably only worked before due to race conditions.

fixes #4716 (probably)
 2021-04-28 15:06:29 -07:00
Steven Allen
69da6a2a29 feat: allow checkpointing to forks 
Previously, `lotus sync checkpoint` would only checkpoint on the current
chain. Now, it can switch to a new fork.
 2021-04-28 13:54:26 -07:00
Łukasz Magiera
95e0c3df1d 1.9.0-rc1 2021-04-27 10:30:51 +02:00
Łukasz Magiera
a24a73bb52 Merge tag 'v1.8.0' into release/v1.9.0 2021-04-27 10:30:33 +02:00
Łukasz Magiera
c4137a438d Merge commit 'f1ded63d608e1b191d79c780b8065f8cff367c98' into feat/nv12-1.11 2021-04-27 08:22:55 +02:00
Aayush Rajasekaran
f1ded63d60 v4 specs-actors integration, nv12 migration 2021-04-27 02:02:44 -04:00
Łukasz Magiera
2e63690125
Merge pull request #6056 from filecoin-project/feat/vm-metrics 
stmgr: Improve ApplyBlocks metrics
 2021-04-18 17:07:16 +02:00
Łukasz Magiera
c7a67cf0e1
Merge pull request #6059 from filecoin-project/frrist/expose-filReserveDisbursed 
polish(api): expose filReserveDisbursed via CirculatingSupply API
 2021-04-18 16:31:03 +02:00
Łukasz Magiera
c72eb0ccaf
Merge pull request #6048 from filecoin-project/fix/nonce-getting 
Fix nonce getting on Lotus lite
 2021-04-18 16:25:47 +02:00
Łukasz Magiera
bb7801e6b7 fix lint 2021-04-18 16:17:31 +02:00
frrist
c118415b12 polish(api): expose filReserveDisbursed via CirculatingSupply API 
- motivated by: https://github.com/filecoin-project/sentinel-visor/issues/462
 2021-04-16 15:15:38 -07:00
Jakub Sztandera
66c408938b
Fix signature in messagepool, wire in context 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-04-16 18:43:28 +02:00
Łukasz Magiera
1b696aae5f stmgr: Fix VMApplyEarly metric 2021-04-16 14:57:59 +02:00
Jakub Sztandera
fd0eb2ec8f
Disable flaky checkpoint tests 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-04-15 19:59:25 +02:00
Jakub Sztandera
77d004ec06
Fix nonce getting on Lotus lite 
Resolves #5593 #5995

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-04-15 19:57:09 +02:00
Łukasz Magiera
53537a0af2 stmgr: Improve ApplyBlocks metrics 2021-04-15 19:44:13 +02:00
Łukasz Magiera
e76935147f Merge remote-tracking branch 'origin/master' into feat/v1-api 2021-04-13 14:24:22 +02:00
Aayush Rajasekaran
c1e4eb3b20 Add a whitelisted block 2021-04-10 02:12:38 -04:00
FinnZhang
9805758e8a remove messagepool CapGasFee duplicate code 2021-04-08 17:50:19 +08:00
Peter Rabbitson
a40fb1e1dd
Merge pull request #5974 from filecoin-project/chore/move_to_ipfs_log_v2 
Centralize everything on ipfs/go-log/v2
 2021-04-06 18:01:36 +02:00
Peter Rabbitson
25a77d905d Centralize everything on ipfs/go-log/v2 
I am not entirely sure this is right, but everything seems to build...
 2021-04-06 15:04:32 +02:00
Peter Rabbitson
7bced1532b Convert the chainstore lock to RW 
High read-API concurrency facilitates multiple calls to GetHeaviestTipSet
which in turn could slow down chainsync.
 2021-04-06 13:01:48 +02:00
Łukasz Magiera
e8f28d7b9f Fix tests 2021-04-06 12:24:58 +02:00
Aayush Rajasekaran
37ca23d9e2 Merge branch 'releases' into asr/release-master 2021-04-05 17:29:19 -04:00
Łukasz Magiera
81bd27911f Propagate StateMsg api changes 2021-04-05 19:56:53 +02:00
Łukasz Magiera
deb2b90b6a Fix lotus/miner build 2021-04-05 13:23:46 +02:00
Aayush Rajasekaran
72c410cd8f Address review on v1 sector extension tool 2021-04-03 22:19:36 -04:00
Łukasz Magiera
0103d2f621 v1 api: Cleanup message wait/search apis 2021-04-02 13:52:24 +02:00
Łukasz Magiera
c9866da688 Merge remote-tracking branch 'origin/master' into feat/v1-api 2021-04-02 13:15:33 +02:00
Łukasz Magiera
28a8636f0b
Merge pull request #5807 from filecoin-project/asr/remainder-acct-fix 
Fix creation of remainder account when it's not a multisig
 2021-04-01 16:31:19 +02:00
Łukasz Magiera
1a5c4d70f4 Merge remote-tracking branch 'origin/master' into feat/v1-api 2021-04-01 15:33:37 +02:00
Aayush Rajasekaran
cea46e5f53 Upgrade to version 11 at Norwegian height 2021-03-29 18:50:13 -04:00
Łukasz Magiera
a7122d424b
Merge pull request #5527 from filcloud/handle-ticket-expire 
fix precommit ticket expiration handling
 2021-03-29 22:55:43 +02:00
Łukasz Magiera
bf6ee88539
Merge pull request #5270 from filcloud/deal-duration 
use deal duration from actors
 2021-03-29 20:17:34 +02:00
Łukasz Magiera
089ff08d81
Merge pull request #5881 from filecoin-project/fix/lint-skip 
lint: don't skip builtin
 2021-03-27 10:09:44 +01:00
Steven Allen
f5029a0187 lint: don't skip builtin 2021-03-25 17:21:56 -07:00
Łukasz Magiera
e003977559 apiclient: Version client funcs 2021-03-25 15:39:48 +01:00
Łukasz Magiera
759ea5b14f Merge remote-tracking branch 'origin/master' into feat/v1-api 2021-03-25 14:13:21 +01:00
Łukasz Magiera
53cba157e2
Merge pull request #5804 from zl03jsj/fix/StateManager.Replay 
fix: StateManager.Replay
 2021-03-25 13:27:45 +01:00
Łukasz Magiera
fc4f60e528
Merge pull request #5850 from filecoin-project/feat/markets-dealid 
expose NextID from nice market actor interface
 2021-03-25 13:26:29 +01:00
Łukasz Magiera
65dcec0ebc api client: Use versioned api packages 2021-03-23 19:15:53 +01:00
Łukasz Magiera
c41777dcd2
API proxy struct codegen (#5854) 
* mostly working api proxy gen

* api: Consistent api names

* fix docsgen

* regenerate api struct

* api: expand external interfaces

* Add missing gen files

* apigen: fix perm detection

* api: Move perm tags to the interface

* gofmt

* worker perms

* docsgen

* docsgen: ignore tag comments

* apigen: add codegen warning

* gofmt

* missing actor type

* docsgen

* make linter happy

* fix lint

* apigen: use directives for tags

* docsgen

* regen openrpc docs
 2021-03-23 13:42:56 +01:00
frrist
0b579c1e03 feat(events): define Observer intreface for events 
- allows tipset apply and revert to be observed
 2021-03-22 15:37:44 -07:00
whyrusleeping
6bcf92f71a expose NextID from nice market actor interface 2021-03-21 00:11:42 -07:00
Peter Rabbitson
c340750672 Nerpa reset backport/cherry-pick from 73eabc310f 2021-03-18 20:55:15 +01:00
Aayush Rajasekaran
707219b7a1 Disable the Calico migration on not-mainnet 2021-03-17 20:46:00 -04:00
Aayush Rajasekaran
fe473e111c Fix creation of remainder account when it's not a multisig 2021-03-12 17:12:41 -05:00
zl
6f64d5db6f bugfix: StateManager.Replay 2021-03-13 00:12:37 +08:00
whyrusleeping
e0b650d4ec make rpcstatemanager its own package, for easy reuse 2021-03-11 18:21:18 -08:00
Łukasz Magiera
58d7627168
Merge pull request #5730 from filecoin-project/asr/genesis-rootkey 
Customize verifreg root key and remainder account when making genesis
 2021-03-08 22:42:09 +01:00
Aayush Rajasekaran
fc03a0773b gen/genesis: Allow remainder to be either account or msig 2021-03-06 01:43:48 -05:00
Aayush Rajasekaran
279f9e6fec gen/genesis: Allow verifreg rootkey to be either account or msig 2021-03-06 01:43:48 -05:00
Raúl Kripalani
b1b452bc0f remove dependency from blockstore/splitstore => chain/store. 2021-03-05 14:46:18 +02:00
Raúl Kripalani
1a804fbdec move splitstore into blockstore package. 2021-03-05 14:46:18 +02:00
Raúl Kripalani
1b51c10d78 split off lmdb support to a different branch. 2021-03-05 14:46:18 +02:00
vyzo
0fc2f3a26f fix post-rebase compilation errors 2021-03-05 14:46:18 +02:00
vyzo
3282f856ec fix tests 2021-03-05 14:46:18 +02:00
vyzo
99c6e4f48f adjust min bloom filter size 2021-03-05 14:46:18 +02:00
vyzo
cae5ddce88 dynamically size bloom filters 2021-03-05 14:46:18 +02:00
vyzo
5639261e44 make compaction parameters variable 2021-03-05 14:46:18 +02:00
vyzo
7587ab6234 quiet the stupid linter 2021-03-05 14:46:18 +02:00
vyzo
05fee27840 remove stale references to lmdb from splitstore implementation 2021-03-05 14:46:18 +02:00
vyzo
f62999d2b8 use named constants for bloom filter parameters 2021-03-05 14:46:18 +02:00
vyzo
44aadb931a rehash salted keys in bloom filter 2021-03-05 14:46:17 +02:00
vyzo
88849201ff fix tests 2021-03-05 14:46:17 +02:00
vyzo
f5ce7957f3 size bloom filter for 50M objects 2021-03-05 14:46:17 +02:00
vyzo
f4c6bc6a86 comment nomenclature 2021-03-05 14:46:17 +02:00
vyzo
4cc672d0c5 batch move objects from coldstore to hotstore 2021-03-05 14:46:17 +02:00
vyzo
97abbe1eca add (salted) bloom filter liveset 2021-03-05 14:46:17 +02:00
vyzo
aba6530411 batch deletion for purging the tracking store 2021-03-05 14:46:17 +02:00
vyzo
09cd1175a1 structured log for beginning of compaction 2021-03-05 14:46:17 +02:00
vyzo
e52c709d8a more accurate setting of skip params 2021-03-05 14:46:17 +02:00
vyzo
2426ffb277 better logging plus moving some code around 2021-03-05 14:46:17 +02:00
vyzo
2f26026991 compactSimple should walk the cold epoch at depth 1 
So that it finds reachable objects that should stay in the hotstore
 2021-03-05 14:46:17 +02:00
vyzo
783dcda19c add Sync to the tracking store 2021-03-05 14:46:17 +02:00
vyzo
364076c1ea set NoSync option for bolt livesets 2021-03-05 14:46:17 +02:00
vyzo
73259aa350 add configuration for splitstore and default to a simple compaction algorithm 2021-03-05 14:46:17 +02:00
vyzo
2e4d45ef07 test for bolt backed tracking store 2021-03-05 14:46:17 +02:00
vyzo
f1c61c4753 implement bolt backed tracking store 2021-03-05 14:46:17 +02:00
vyzo
2c1a9781cf add test for bolt liveset 2021-03-05 14:46:17 +02:00
vyzo
27a9b974db implement bolt-backed liveset 2021-03-05 14:46:17 +02:00
vyzo
68b6f913c7 propagate useLMDB option to splitstore through DI 2021-03-05 14:46:17 +02:00
vyzo
923a3db4b0 abstract tracking store and live set construction 2021-03-05 14:46:17 +02:00
vyzo
8f0ddac41a add comment 2021-03-05 14:46:17 +02:00
vyzo
e79445123f handle MDB_KEY_EXIST in tracking store Puts 2021-03-05 14:46:17 +02:00
vyzo
9977f5c3ec rewrite sweep logic to avoid doing writes/deletes nested in a read txn 2021-03-05 14:46:17 +02:00
vyzo
ee751f88cd refactor lmdb specific snoop/liveset code into their own files 
paves the way for different back ends
 2021-03-05 14:46:17 +02:00
vyzo
8e12377e69 handle consistency edge case 2021-03-05 14:46:17 +02:00
vyzo
31268ba685 walk snapshot the same way snapshot exporting does; skip old msgs and receipts by default. 
so that we don't panic with missing blocks in non-archival nodes
 2021-03-05 14:46:17 +02:00
vyzo
5068d51ac3 use CompactionCold epochs for delinating the cold epoch cliff 
this allows us to change the thresholds for testing.
 2021-03-05 14:46:17 +02:00
vyzo
d44719dbd3 amend confusing comment 2021-03-05 14:46:17 +02:00
vyzo
842ec43c2f get rid of goroutine iteration in tracking store; long live ForEach 2021-03-05 14:46:17 +02:00
vyzo
a586d42c3b make hot store DI injectable in the split store, default to badger. 2021-03-05 14:46:17 +02:00
vyzo
7044e623f9 flag to enable GC during compaction, disabled for now 2021-03-05 14:46:17 +02:00
vyzo
f6c930d0aa crank up blockstore max readers to 16K, reduce retry delays to 10us 2021-03-05 14:46:17 +02:00
vyzo
95befa1e41 set lmdb max readers retry delay to 1ms 2021-03-05 14:46:17 +02:00
vyzo
874ecd3573 adjust hot store options, redux. 2021-03-05 14:46:17 +02:00
vyzo
ca8a673b5f adjust hot store options 2021-03-05 14:46:17 +02:00
vyzo
69a88d41b6 fix snoop test 2021-03-05 14:46:17 +02:00
vyzo
cdf5bd0500 return annotated xerrors where appropriate 2021-03-05 14:46:17 +02:00
vyzo
ea05fd9d86 use xerrors instead of fmt.Errorf 2021-03-05 14:46:17 +02:00
vyzo
d91b60df91 fix potential panic with max readers retry and cursor channel 2021-03-05 14:46:17 +02:00
vyzo
b9f8a3d587 log MDB_READERS_FULL retries 2021-03-05 14:46:17 +02:00
vyzo
c89ab1a990 retry on MDB_READERS_FULL errors 2021-03-05 14:46:17 +02:00
vyzo
2080e467ba don't set MaxReaders for tracking store 2021-03-05 14:46:17 +02:00
Raúl Kripalani
877ecab960 update go-bs-lmdb and migrate to ledgerwatch/lmdb-go. 2021-03-05 14:46:17 +02:00
vyzo
5b4e6b7b26 don't set max readers for livesets 2021-03-05 14:46:17 +02:00
vyzo
58a8434825 temporary log level for splitstore to DEBUG 2021-03-05 14:46:17 +02:00
vyzo
8b0087524f adjust walk boundaries for marking 2021-03-05 14:46:17 +02:00
vyzo
76d6edbb52 fix max readers for tracking store 2021-03-05 14:46:17 +02:00
vyzo
1a23b1f6af make CompactionThreshold a var to fix lotus-soup build 
finality is not a constant there!
 2021-03-05 14:46:17 +02:00
vyzo
6e51e6db9a better handling of MDB_KEYEXIST in Put 2021-03-05 14:46:17 +02:00
vyzo
ce41e394dc handle MDB_KEYEXIST in liveset marking 2021-03-05 14:46:17 +02:00
vyzo
843fd09a62 deal with MDB_KEY_EXIST errors 2021-03-05 14:46:17 +02:00
vyzo
f44cf0f2c4 appease linter 2021-03-05 14:46:17 +02:00
vyzo
facdc555b1 add nil check for curTs -- some tests don't have chain state 2021-03-05 14:46:17 +02:00
vyzo
e07c6c71c0 splitstore constructor 2021-03-05 14:46:17 +02:00
vyzo
b0f48b500f use CAS for compacting state 2021-03-05 14:46:17 +02:00
vyzo
0af7b16ad5 simplify Has 2021-03-05 14:46:17 +02:00
vyzo
37e391f133 add TODO note about map size 2021-03-05 14:46:17 +02:00
vyzo
5db314f422 fallback to coldstore if snooping fails. 2021-03-05 14:46:17 +02:00
vyzo
d20cbc0c28 protect against potential data races 
overkill, but let's not have race detectors scream at us.
 2021-03-05 14:46:17 +02:00
vyzo
da478832cb quiet linter 2021-03-05 14:46:17 +02:00
vyzo
4763397281 add tracking store test 2021-03-05 14:46:17 +02:00
vyzo
0d7476c5b2 implement LMDB-backed tracking store 2021-03-05 14:46:17 +02:00
vyzo
83f8a0ab12 quiet linter 2021-03-05 14:46:17 +02:00
vyzo
5043f31adf liveset unit test 2021-03-05 14:46:17 +02:00
vyzo
3f92a000c7 implement lmdb-backed LiveSet 2021-03-05 14:46:17 +02:00
vyzo
0bf1a78b39 stubs for tracking store and live set 2021-03-05 14:46:17 +02:00
vyzo
17bc5fcd85 move splitstore implementation to its own directory 2021-03-05 14:46:17 +02:00
vyzo
2c9b58aaec add some logging 2021-03-05 14:46:17 +02:00
vyzo
c1b1a9ce2a avoid race with compacting state variable 2021-03-05 14:46:17 +02:00
vyzo
3083d80f5e no need to import go-ipfs-blockstore, lib/blockstore will do 2021-03-05 14:46:17 +02:00
vyzo
101e5c6540 close keys channel when dome emitting keys 2021-03-05 14:46:17 +02:00
vyzo
b945747eb2 satisfy linter 2021-03-05 14:46:17 +02:00
vyzo
2bed6c94cd use dual live set marking algorithm to keep all hotly reachable objects in the hotstore 2021-03-05 14:46:17 +02:00
vyzo
c2cc198316 fix off by 1 in marking 2021-03-05 14:46:17 +02:00
vyzo
fd08786048 track base epoch in metadata ds 2021-03-05 14:46:17 +02:00
vyzo
b192adfd2e trigger compaction from head changes 2021-03-05 14:46:17 +02:00
vyzo
c8f1139e0d compaction algorithm 2021-03-05 14:46:17 +02:00
vyzo
6577cc8ea6 splitstore struct and Blockstore interface implementation 2021-03-05 14:46:17 +02:00
Łukasz Magiera
d53c700d18
Merge pull request #5704 from filecoin-project/asr/log-large-delay 
Log block CID in the large delay warning
 2021-03-05 13:19:03 +01:00
Aayush Rajasekaran
374bd9b578
Update chain/sub/incoming.go 
Co-authored-by: raulk <raul@protocol.ai>
 2021-03-05 01:09:07 -05:00
Aayush Rajasekaran
22e465ba9c log block cid in the large delay warning 2021-03-02 16:46:10 -05:00
Raúl Kripalani
1ac0c9a926 address review comments. 2021-03-02 21:29:24 +00:00
Raúl Kripalani
2047a74958 implement blockstore.Union, a union blockstore. 
The union blockstore takes a list of blockstores. It returns the first
satisfying read, and broadcasts writes to all stores.

It can be used for operations that require reading from any two blockstores,
for example WalkSnapshot.
 2021-03-02 17:03:11 +00:00
Raúl Kripalani
b1c348b4a7 address review comments. 2021-03-02 16:31:01 +00:00
Raúl Kripalani
3795cc2bd2 segregate chain and state blockstores. 
This paves the way for better object lifetime management.

Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
  sizing caches differently.
- specifying different retention policies for chain and state data.

This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
  store, and only retain state objects reachable from the last finality
  in the state store.
 2021-02-28 22:49:44 +00:00
Raúl Kripalani
7f0f7d0b36 Merge branch 'master' into refactor/lib/blockstore 2021-02-28 19:55:23 +00:00
Łukasz Magiera
35759fa07e
Merge pull request #5635 from filecoin-project/deps/cbg-soft-map 
Update markets, cbor-gen with soft map decoding
 2021-02-22 18:27:38 +01:00
Aayush Rajasekaran
60b7f56297 Code review 2021-02-19 15:10:56 -05:00
Łukasz Magiera
26399dba70 Update markets, cbor-gen with soft map decoding 2021-02-19 20:11:43 +01:00
Aayush Rajasekaran
96b4b828f7 Remove the null bls actor from the init actor state 2021-02-18 19:01:28 -05:00
Aayush Rajasekaran
a0fedf8bdb Only delete the zero BLS actor on mainnet 2021-02-18 19:01:21 -05:00
Aayush Rajasekaran
2ca70e600d Move the zero bls actor creation check into the VM 2021-02-18 19:01:11 -05:00
Aayush Rajasekaran
5aff54161a Delete the null-bls actor 2021-02-18 19:01:03 -05:00
Łukasz Magiera
0c6aef221d Fix error logging format strings 2021-02-11 12:00:26 +01:00
Łukasz Magiera
fcbd9950ca
Merge pull request #5507 from filecoin-project/fix/coalescer 
refactor coalescing logic into its own function, take both cancellation sets into account
 2021-02-10 17:40:47 +01:00
Łukasz Magiera
9f546c2d1c
Merge pull request #5540 from austinabell/austin/beyondmaxio 
Remove unnecessary database reads in validation check
 2021-02-05 22:40:32 +01:00
austinabell
3228ae5672
Remove unnecessary database reads in validation check 2021-02-05 15:46:46 -05:00
Łukasz Magiera
8f603717a6 Merge remote-tracking branch 'origin/master' into steb/refactor-consistent-tipset-methods 2021-02-05 13:08:49 +01:00
Łukasz Magiera
06d3892e38 lotus-wallet: Add interactive mode 2021-02-05 12:44:01 +01:00
He Weidong
0e2e1125d3 fix precommit ticket expiration handling 2021-02-05 10:45:15 +08:00
vyzo
5c6988a7c1 refactor coalescing logic into its own function, take both cancellation sets into account 2021-02-02 21:26:50 +02:00
Łukasz Magiera
c59e2fea5b Use correct contexts in storageadapter 2021-02-02 18:43:49 +01:00
Łukasz Magiera
56a9d05491 Merge remote-tracking branch 'origin/master' into steb/refactor-consistent-tipset-methods 2021-01-30 12:05:21 +01:00
Łukasz Magiera
a541a2500a
Merge pull request #5282 from filecoin-project/chore/snake_context_through_blockstore_init 
Snake a context through the Chain-blockstore creation
 2021-01-30 11:55:17 +01:00
Łukasz Magiera
7c7301f701
Merge pull request #5350 from GFZRZK/GFZRZK/optmize/avoid_race_in_mpool_cfg 
avoid use mp.cfg directly to avoid race
 2021-01-30 11:31:56 +01:00
Łukasz Magiera
5cfae0f1e8
Merge pull request #5449 from filecoin-project/chore/document_blockheader_fields 
Better document the block-header fields
 2021-01-30 11:26:56 +01:00
Raúl Kripalani
d1104fec4c rename blockstores for consistency. 2021-01-29 23:17:25 +00:00
Raúl Kripalani
af429dba0f Merge branch 'master' into chore/snake_context_through_blockstore_init 2021-01-29 21:21:46 +00:00
Raúl Kripalani
e02fdf5064 Merge branch 'master' into refactor/lib/blockstore 2021-01-29 21:21:31 +00:00
Raúl Kripalani
b0cbc932bd consolidate all blockstores in blockstore package. 2021-01-29 20:01:00 +00:00
Steven Allen
53271729b6 tweak migration perf 2021-01-28 16:15:23 -08:00
Steven Allen
2f3c912ad9 improve upgrade logging 2021-01-28 16:15:23 -08:00
Steven Allen
a9a65e05d8 feat: avoid computing the tip-set state when looking up addresses 2021-01-27 17:54:42 -08:00
Steven Allen
4890d83acb test: assert that pre-migrations are actually run 2021-01-27 16:12:08 -08:00
Steven Allen
39d4f6780d pre-migration: refactor timing specification 
There are now three times:

1. StartWithin: start within X epochs of the upgrade.
2. DontStartWithin: don't start within X epochs of the upgrade.
3. StopWithin: stop within X epochs of the upgrade.
 2021-01-27 16:09:07 -08:00
Steven Allen
bceb246080 set worker counts for pre-migrations 2021-01-27 15:47:51 -08:00
Steven Allen
cffeb1a590
fix doc comments 
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
 2021-01-27 15:41:26 -08:00
Steven Allen
9574db6d0a log in migration 2021-01-27 15:05:00 -08:00
Steven Allen
9a03c003fe fix sorted test 2021-01-27 13:05:14 -08:00
Steven Allen
56054b0ad3 sort pre-migrations 2021-01-27 12:57:47 -08:00
Steven Allen
854385168d add additional pre-migration validations 2021-01-27 12:57:47 -08:00
Steven Allen
716f10e570 remove unnecessary wrapper store in tests 
This was causing a lot of warnings about not implementing View.
 2021-01-27 12:57:47 -08:00
Steven Allen
35d6a40071 cleanup pre-migration code a bit 2021-01-27 12:57:17 -08:00
Steven Allen
4072f24bf2 test pre-migrations 2021-01-27 12:57:17 -08:00
Steven Allen
ed3e0869c3 register tipsets with the chainstore when testing 2021-01-27 12:57:17 -08:00
Steven Allen
05026a23b9 actually use the temp cache for pre-migrations 2021-01-27 12:57:17 -08:00
Steven Allen
32059d0cbf stmgr: only persist the migration cache on success 
Otherwise, we may end up referencing blocks we never wrote to disk.
 2021-01-27 12:56:47 -08:00
Steven Allen
b08abc12ba fix error message 2021-01-27 12:55:56 -08:00
Steven Allen
6362887ce3 rename Upgrade to Migration where applicable 
This was really confusing.
 2021-01-27 12:55:56 -08:00
Steven Allen
77117a0be5 wait for pre-upgrades to exit on stop 2021-01-27 12:55:56 -08:00
Steven Allen
f65d179f2c make pre-migrations async 2021-01-27 12:55:56 -08:00
Steven Allen
8d05c5d62c validate pre-migrations 2021-01-27 12:55:56 -08:00
Steven Allen
8986a2002d wire up re-migration logic for nv10 2021-01-27 12:55:56 -08:00
Steven Allen
8d3cc632ac add support for running pre-migrations and caching their results 
This can significantly speedup state migrations.
 2021-01-27 12:55:53 -08:00
Peter Rabbitson
ac616eb340 Better document the block-header fields 2021-01-27 20:52:18 +01:00