Commit Graph

3229 Commits

Author SHA1 Message Date
Raúl Kripalani
3795cc2bd2 segregate chain and state blockstores.
This paves the way for better object lifetime management.

Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
  sizing caches differently.
- specifying different retention policies for chain and state data.

This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
  store, and only retain state objects reachable from the last finality
  in the state store.
2021-02-28 22:49:44 +00:00
Raúl Kripalani
7f0f7d0b36 Merge branch 'master' into refactor/lib/blockstore 2021-02-28 19:55:23 +00:00
Łukasz Magiera
35759fa07e
Merge pull request #5635 from filecoin-project/deps/cbg-soft-map
Update markets, cbor-gen with soft map decoding
2021-02-22 18:27:38 +01:00
Aayush Rajasekaran
60b7f56297 Code review 2021-02-19 15:10:56 -05:00
Łukasz Magiera
26399dba70 Update markets, cbor-gen with soft map decoding 2021-02-19 20:11:43 +01:00
Aayush Rajasekaran
96b4b828f7 Remove the null bls actor from the init actor state 2021-02-18 19:01:28 -05:00
Aayush Rajasekaran
a0fedf8bdb Only delete the zero BLS actor on mainnet 2021-02-18 19:01:21 -05:00
Aayush Rajasekaran
2ca70e600d Move the zero bls actor creation check into the VM 2021-02-18 19:01:11 -05:00
Aayush Rajasekaran
5aff54161a Delete the null-bls actor 2021-02-18 19:01:03 -05:00
Łukasz Magiera
0c6aef221d Fix error logging format strings 2021-02-11 12:00:26 +01:00
Łukasz Magiera
fcbd9950ca
Merge pull request #5507 from filecoin-project/fix/coalescer
refactor coalescing logic into its own function, take both cancellation sets into account
2021-02-10 17:40:47 +01:00
Łukasz Magiera
9f546c2d1c
Merge pull request #5540 from austinabell/austin/beyondmaxio
Remove unnecessary database reads in validation check
2021-02-05 22:40:32 +01:00
austinabell
3228ae5672
Remove unnecessary database reads in validation check 2021-02-05 15:46:46 -05:00
Łukasz Magiera
8f603717a6 Merge remote-tracking branch 'origin/master' into steb/refactor-consistent-tipset-methods 2021-02-05 13:08:49 +01:00
Łukasz Magiera
06d3892e38 lotus-wallet: Add interactive mode 2021-02-05 12:44:01 +01:00
vyzo
5c6988a7c1 refactor coalescing logic into its own function, take both cancellation sets into account 2021-02-02 21:26:50 +02:00
Łukasz Magiera
c59e2fea5b Use correct contexts in storageadapter 2021-02-02 18:43:49 +01:00
Łukasz Magiera
56a9d05491 Merge remote-tracking branch 'origin/master' into steb/refactor-consistent-tipset-methods 2021-01-30 12:05:21 +01:00
Łukasz Magiera
a541a2500a
Merge pull request #5282 from filecoin-project/chore/snake_context_through_blockstore_init
Snake a context through the Chain-blockstore creation
2021-01-30 11:55:17 +01:00
Łukasz Magiera
7c7301f701
Merge pull request #5350 from GFZRZK/GFZRZK/optmize/avoid_race_in_mpool_cfg
avoid use mp.cfg directly to avoid race
2021-01-30 11:31:56 +01:00
Łukasz Magiera
5cfae0f1e8
Merge pull request #5449 from filecoin-project/chore/document_blockheader_fields
Better document the block-header fields
2021-01-30 11:26:56 +01:00
Raúl Kripalani
d1104fec4c rename blockstores for consistency. 2021-01-29 23:17:25 +00:00
Raúl Kripalani
af429dba0f Merge branch 'master' into chore/snake_context_through_blockstore_init 2021-01-29 21:21:46 +00:00
Raúl Kripalani
e02fdf5064 Merge branch 'master' into refactor/lib/blockstore 2021-01-29 21:21:31 +00:00
Raúl Kripalani
b0cbc932bd consolidate all blockstores in blockstore package. 2021-01-29 20:01:00 +00:00
Steven Allen
53271729b6 tweak migration perf 2021-01-28 16:15:23 -08:00
Steven Allen
2f3c912ad9 improve upgrade logging 2021-01-28 16:15:23 -08:00
Steven Allen
a9a65e05d8 feat: avoid computing the tip-set state when looking up addresses 2021-01-27 17:54:42 -08:00
Steven Allen
4890d83acb test: assert that pre-migrations are actually run 2021-01-27 16:12:08 -08:00
Steven Allen
39d4f6780d pre-migration: refactor timing specification
There are now three times:

1. StartWithin: start within X epochs of the upgrade.
2. DontStartWithin: don't start within X epochs of the upgrade.
3. StopWithin: stop within X epochs of the upgrade.
2021-01-27 16:09:07 -08:00
Steven Allen
bceb246080 set worker counts for pre-migrations 2021-01-27 15:47:51 -08:00
Steven Allen
cffeb1a590
fix doc comments
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
2021-01-27 15:41:26 -08:00
Steven Allen
9574db6d0a log in migration 2021-01-27 15:05:00 -08:00
Steven Allen
9a03c003fe fix sorted test 2021-01-27 13:05:14 -08:00
Steven Allen
56054b0ad3 sort pre-migrations 2021-01-27 12:57:47 -08:00
Steven Allen
854385168d add additional pre-migration validations 2021-01-27 12:57:47 -08:00
Steven Allen
716f10e570 remove unnecessary wrapper store in tests
This was causing a lot of warnings about not implementing View.
2021-01-27 12:57:47 -08:00
Steven Allen
35d6a40071 cleanup pre-migration code a bit 2021-01-27 12:57:17 -08:00
Steven Allen
4072f24bf2 test pre-migrations 2021-01-27 12:57:17 -08:00
Steven Allen
ed3e0869c3 register tipsets with the chainstore when testing 2021-01-27 12:57:17 -08:00
Steven Allen
05026a23b9 actually use the temp cache for pre-migrations 2021-01-27 12:57:17 -08:00
Steven Allen
32059d0cbf stmgr: only persist the migration cache on success
Otherwise, we may end up referencing blocks we never wrote to disk.
2021-01-27 12:56:47 -08:00
Steven Allen
b08abc12ba fix error message 2021-01-27 12:55:56 -08:00
Steven Allen
6362887ce3 rename Upgrade to Migration where applicable
This was really confusing.
2021-01-27 12:55:56 -08:00
Steven Allen
77117a0be5 wait for pre-upgrades to exit on stop 2021-01-27 12:55:56 -08:00
Steven Allen
f65d179f2c make pre-migrations async 2021-01-27 12:55:56 -08:00
Steven Allen
8d05c5d62c validate pre-migrations 2021-01-27 12:55:56 -08:00
Steven Allen
8986a2002d wire up re-migration logic for nv10 2021-01-27 12:55:56 -08:00
Steven Allen
8d3cc632ac add support for running pre-migrations and caching their results
This can significantly speedup state migrations.
2021-01-27 12:55:53 -08:00
Peter Rabbitson
ac616eb340 Better document the block-header fields 2021-01-27 20:52:18 +01:00
Raúl Kripalani
a1da1dab85 add context to LockedRepo#Datastore(). 2021-01-26 11:01:43 +00:00
Łukasz Magiera
fabcbb621d Merge remote-tracking branch 'origin/master' into next 2021-01-26 00:47:22 +01:00
Aayush Rajasekaran
29d3d746a3 Build a WindowPoSt disputer 2021-01-25 17:26:09 -05:00
Łukasz Magiera
260908015f
Merge pull request #5393 from filecoin-project/feat-refactor-bls
refactor: switch to filecoin-ffi bls api for bls signatures
2021-01-25 20:56:18 +01:00
Raúl Kripalani
ce3af308ed Merge branch 'master' into chore/snake_context_through_blockstore_init 2021-01-25 19:31:41 +00:00
Łukasz Magiera
e4f4fa2c7d Update FFI more, fix zero aggregate sigs 2021-01-25 20:05:03 +01:00
Łukasz Magiera
bd6321d2c0 Fix lint 2021-01-25 15:35:00 +01:00
wangchao
ff62b64b4d avoid use mp.cfg directly to avoid race 2021-01-25 10:13:31 +08:00
Steven Allen
7c9b9e3657 explicitly check miner address protocol
This is implicitly checked when verifying the miner signature, but
explicitly checking it here makes it clear that this is a protocol
requirement, and reduces the chances that it will be accidentally
"refactored out".
2021-01-22 18:04:07 -08:00
Steven Allen
94089a6a15 remove todos 2021-01-22 11:15:43 -08:00
Steven Allen
cc423ee4d2 fix seal types length 2021-01-22 08:30:58 -08:00
Steven Allen
0ba1306969 test prove then dispute 2021-01-21 19:48:34 -08:00
Steven Allen
f10b13ff32 register v3 methods 2021-01-21 15:21:20 -08:00
Steven Allen
5a5e49dcd7 support paych v3 messages 2021-01-21 15:21:20 -08:00
Steven Allen
c5d3b979cb fix constructing new actors 2021-01-21 15:21:20 -08:00
Steven Allen
6ce5f2736e update policy for v3 2021-01-21 15:21:20 -08:00
Steven Allen
fe5b5702f9 fix data race in test 2021-01-21 15:21:20 -08:00
Steven Allen
f1719d1e23 mark v3 as expensive 2021-01-21 15:21:20 -08:00
Steven Allen
bf48728875 move winning post proof type logic into a helper 2021-01-21 15:21:20 -08:00
Steven Allen
03cd3760bb correctly pick the seal type based on the network version
Of course, we should really just run all of our tests post actors v2.
2021-01-21 15:21:20 -08:00
Steven Allen
82eee15955 fix state tree version comment
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2021-01-21 15:21:19 -08:00
Steven Allen
64e805926f update specs-actors for migration name change
Actors v3 is network version 10.
2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
95f68d0161 Register the v3 actor CIDs in the actor registry 2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
f6a0da3751 Add the migrationLogger struct 2021-01-21 15:21:19 -08:00
Steven Allen
99ea5b62ea wire up v3 migration 2021-01-21 15:21:19 -08:00
Steven Allen
1f62c07789 add back logging 2021-01-21 15:21:19 -08:00
Steven Allen
1a790f5ee3 use actors state-tree abstraction
Well, not fully, but this is enough to ensure we use the correct bitwidth and such.
2021-01-21 15:21:19 -08:00
Steven Allen
9d2c430138 remove abstract map constructor/loader
Different maps have different parameters now so we just construct/load them manually where needed.
2021-01-21 15:21:19 -08:00
Steven Allen
8dc49db30d window <-> winning 2021-01-21 15:21:19 -08:00
Steven Allen
c05df6dc1d fix: use the correct proof type in winning proofs 2021-01-21 15:21:19 -08:00
Steven Allen
7882b0163b update actor "is" functions for v3 2021-01-21 15:21:19 -08:00
Steven Allen
c31f5130e3 update imports to v3 2021-01-21 15:21:19 -08:00
Steven Allen
95c57d0ce3 and actually hook up loading logic 2021-01-21 15:21:19 -08:00
Steven Allen
be1f15052c register remaining state decoders 2021-01-21 15:21:19 -08:00
Steven Allen
9ce63c1211 simplify v3 claims map access 2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
06f13f8e00 Add v3 actors support for verifreg 2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
76abbb8938 Add v3 actors support for reward 2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
ee99e68614 Add v3 actors support for power 2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
abb388512e Add v3 actors support for paych 2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
9d06a8b116 Add v3 actors support for multisig 2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
0bb3aed53c Add v3 actors support for miner 2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
ab90a3b2bc Actors update: MinerInfo.SealProofType has been removed 2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
82635f2515 Rename Deadline.PoStSubmissions to PartitionsPoSted 2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
b4f479affd Add v3 actors support for market 2021-01-21 15:21:18 -08:00
Aayush Rajasekaran
378f169f14 Add v3 actors support for init 2021-01-21 15:21:18 -08:00
Aayush Rajasekaran
3961e3daf8 Add v3 actors support for account 2021-01-21 15:21:18 -08:00
Aayush Rajasekaran
ba3b32cfe7 Pull in actors v3 2021-01-21 15:21:18 -08:00
Steven Allen
4cdb7ba1d3 fix a flake in the sync manager edge case test
Depending on scheduling, we may see c2 before we see c1.
2021-01-20 16:08:29 -08:00
dignifiedquire
3d9eb226fa refactor: switch to filecoin-ffi bls api for bls signatures 2021-01-20 16:19:07 +01:00
Dirk McCormick
827a473391 feat: expose StateSearchMessage on gateway 2021-01-19 11:27:43 +01:00
Łukasz Magiera
2b3d66da3b
Merge pull request #5362 from filecoin-project/feat/fullnode-restore
Implement full-node restore option
2021-01-18 23:42:20 +01:00
Lucas Molas
69eb2163d2
fix(sync): enforce fork len when changing head (#5244) 2021-01-18 12:34:25 -03:00
Łukasz Magiera
38f9559080 Implement full-node restore option 2021-01-15 20:57:32 +01:00
Łukasz Magiera
476df99179
Merge pull request #5341 from filecoin-project/feat/sectors-terminate
Sector termination support
2021-01-14 22:00:18 +01:00
Łukasz Magiera
80b8d4b9d7 Address review 2021-01-14 20:27:15 +01:00
Łukasz Magiera
49abdd7d7d Sector termination support - address review 2021-01-14 15:46:57 +01:00
Ian Davis
7d2adc383f
chore: export vm.ShouldBurn 2021-01-14 11:29:57 +00:00
Łukasz Magiera
b0bb36909b
Merge pull request #4192 from filecoin-project/schomatis/sync/remove-self-bypass
fix(sync): remove checks bypass when we submit the block
2021-01-12 14:52:50 +01:00
He Weidong
ab9b3b6851 enable parse and shorten negative FIL values 2021-01-09 17:48:36 +08:00
Łukasz Magiera
32dcfb4842
Merge pull request #5303 from filecoin-project/feat/wallet-market-info
Better CLI for wallet market withdraw and client info
2021-01-08 22:54:12 +01:00
Dirk McCormick
c58086ee27 feat: better CLI for wallet market withdraw and client info 2021-01-07 10:18:29 +01:00
Łukasz Magiera
72871597cd
Merge pull request #5293 from filecoin-project/fix/fund-mgr-nice-err
Better error message when withdrawing funds
2021-01-06 20:23:50 +01:00
Dirk McCormick
e8e4b559e9 fix: format fund manager message fil amounts 2021-01-06 09:23:54 +01:00
Dirk McCormick
3287c621ad fix: better withdrawal error message 2021-01-05 16:58:12 +01:00
Peter Rabbitson
446a24ef3f Fix forgotten test-call 2021-01-04 15:46:09 +01:00
Peter Rabbitson
9334e73396 Snake a context through the Chain-blockstore creation 2021-01-04 15:27:52 +01:00
Ruslan Tushov
f9aca39caf
Buffered stream for chainxchg response 2020-12-25 16:03:46 +03:00
Lucas Molas
461a3cdebc
fix(sync): enforce ForkLengthThreshold for synced chain (#5182) 2020-12-22 12:29:36 -03:00
Jakub Sztandera
c7e3e9fa7f
Add IsNearUpgrade
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-12-19 21:51:07 +01:00
Łukasz Magiera
c1aafb7b71 Add missing v9 to chain/actors/version 2020-12-19 21:29:12 +01:00
Jakub Sztandera
a3b325db5d
Disable consensus faults during upgrade orange
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-12-19 21:00:12 +01:00
Łukasz Magiera
24dd6e01d7 disable SF for finality epochs 2020-12-19 20:53:08 +01:00
Łukasz Magiera
22884f18f1 define v9 upgrade 2020-12-19 20:44:21 +01:00
Steven Allen
1f754bdb78 disable gas burning for window post messages
While over-estimation fees and miner tips are still paid, gas is no longer burnt
for direct, successful window PoSt messages.

Usually, gas is burnt to prevent an attacker from spamming the network and to
allow clients to "price" messages (using the base fee cap) based on how urgently
they need them to be processed. However:

1. Window PoSt is already a "proof of work".
2. Miners need to submit WindowedPoSts on-time so all window post messages are urgent.
3. Work is already under way to move window post verification off-chain (making
it effectively free). This change simply introduces the "free" part a bit earlier.
2020-12-16 00:10:25 -05:00
Łukasz Magiera
4767773d1b
Merge pull request #5171 from filecoin-project/raulk/block-delay-metric
add metrics for delayed blocks.
2020-12-10 17:12:12 +01:00
vyzo
e07059e73d fix PushUntrusted publishing -- the message is local 2020-12-10 17:26:41 +02:00
Raúl Kripalani
0570306bb6 add metrics for delayed blocks. 2020-12-10 14:52:05 +00:00
Steven Allen
a47b063a24 Fix duplicate nonce test
StateGetActor now gets the actor at the tipset parent state.
2020-12-09 11:29:41 -08:00
Steven Allen
b8e3808c4f Make state tipset usage consistent in the API
_Always_ (almost) use the tipset's parent state, instead of computing.

Exceptions:

* MinerGetBaseInfo. Fixing this would break things so we need to be
careful (although we could bump the API version, fix it, then fix the call
sites).
* StateReplay. This is replaying a message on top of the given tipset.
* GasEstimateGasLimit. This executes the message on-top-of the tipset's
computed state (unlike call which executes it on the tipset's parent state).
  * Having this method and Call apply the message at different heights is really
  weird.
2020-12-09 11:29:40 -08:00
Łukasz Magiera
e1be89b442
Merge pull request #5159 from iand/fix/event-hang
fix(events): avoid potential hang when starting event listener
2020-12-09 19:40:02 +01:00
Łukasz Magiera
bc9b5cb5fc
Merge pull request #5160 from iand/fix/unlock-event-mutex
fix(events): unlock mutex before error returns
2020-12-09 19:25:07 +01:00
Ian Davis
c6f5214657
fix(events): unlock mutex before error returns 2020-12-09 13:37:56 +00:00
Ian Davis
696469aae7
fix: avoid potential hang when starting event listener
It was possible for NewEvents to never return, blocked on waiting for a WaitGroup to be done.
The call to Done was in a goroutine that could exit before reaching the Done call.

Replace the WaitGroup with a channel that is closed to signal that initialisation is complete.
Also, while we are waiting on the channel, wait on the context so we can exit clealy if the
context is canceled.
2020-12-09 11:29:48 +00:00
Jakub Sztandera
6ca5caef31
Refactor DefaultMessageSendSpec
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-12-08 20:51:27 +01:00
Jakub Sztandera
689b93dc0e
messagepool: small refactor
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-12-08 20:07:13 +01:00
Łukasz Magiera
b190c3492a
Merge pull request #5099 from zgfzgf/feat-chain-gen
delete repeat call func
2020-12-03 13:23:39 +01:00
Travis Person
a584fcb935 sync: log a message when there aren't enough peers 2020-12-03 01:27:07 +00:00
zgfzgf
a415e49fdb delete repeat call func 2020-12-02 20:05:01 +08:00
Łukasz Magiera
bc14d0820f fix lint 2020-12-01 20:00:55 +01:00
Łukasz Magiera
b28ed5887c gofmt 2020-12-01 19:23:43 +01:00
Łukasz Magiera
be9f7b88b2 uint64 is too small 2020-12-01 19:15:39 +01:00
Łukasz Magiera
00e5ace818 Print basefee in miner info 2020-12-01 18:56:43 +01:00
Łukasz Magiera
82b5cb89cd
Merge pull request #5061 from filecoin-project/fix/err-late-deals
Error out deals that are not activated by proposed deal start epoch
2020-12-01 17:02:06 +01:00
Jakub Sztandera
f1291f137c
Enable Callers tracing when GasTracing is enabled
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-12-01 13:53:30 +01:00
Dirk McCormick
e02a366419 feat: error out deals that are not activated by proposed deal start epoch 2020-11-30 15:27:40 +01:00
Jakub Sztandera
13ff98416f
Fix cs.tipsets being unbounded
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-11-26 23:37:59 +01:00
zgfzgf
32c159fa3a fix lint check 2020-11-26 21:36:40 +08:00
zgfzgf
fad250058d fix wallet delete address where address is default 2020-11-26 20:55:29 +08:00
zgfzgf
15aa8e5253 fix wallet dead lock 2020-11-25 12:58:17 +08:00
Łukasz Magiera
a63ef1dcd5
Merge pull request #4984 from yaohcn/fix-log-warn
fix log format
2020-11-24 18:01:56 +01:00