cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
10,794 Commits 24 Branches 342 Tags 314 MiB
9977f5c3ec
Commit Graph

351 Commits

Author SHA1 Message Date
vyzo
2bed6c94cd use dual live set marking algorithm to keep all hotly reachable objects in the hotstore 2021-03-05 14:46:17 +02:00
vyzo
c2cc198316 fix off by 1 in marking 2021-03-05 14:46:17 +02:00
vyzo
fd08786048 track base epoch in metadata ds 2021-03-05 14:46:17 +02:00
vyzo
b192adfd2e trigger compaction from head changes 2021-03-05 14:46:17 +02:00
vyzo
c8f1139e0d compaction algorithm 2021-03-05 14:46:17 +02:00
vyzo
6577cc8ea6 splitstore struct and Blockstore interface implementation 2021-03-05 14:46:17 +02:00
Raúl Kripalani
1ac0c9a926 address review comments. 2021-03-02 21:29:24 +00:00
Raúl Kripalani
2047a74958 implement blockstore.Union, a union blockstore. 
The union blockstore takes a list of blockstores. It returns the first
satisfying read, and broadcasts writes to all stores.

It can be used for operations that require reading from any two blockstores,
for example WalkSnapshot.
 2021-03-02 17:03:11 +00:00
Raúl Kripalani
b1c348b4a7 address review comments. 2021-03-02 16:31:01 +00:00
Raúl Kripalani
3795cc2bd2 segregate chain and state blockstores. 
This paves the way for better object lifetime management.

Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
  sizing caches differently.
- specifying different retention policies for chain and state data.

This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
  store, and only retain state objects reachable from the last finality
  in the state store.
 2021-02-28 22:49:44 +00:00
Raúl Kripalani
7f0f7d0b36 Merge branch 'master' into refactor/lib/blockstore 2021-02-28 19:55:23 +00:00
vyzo
5c6988a7c1 refactor coalescing logic into its own function, take both cancellation sets into account 2021-02-02 21:26:50 +02:00
Raúl Kripalani
d1104fec4c rename blockstores for consistency. 2021-01-29 23:17:25 +00:00
Raúl Kripalani
b0cbc932bd consolidate all blockstores in blockstore package. 2021-01-29 20:01:00 +00:00
Raúl Kripalani
a1da1dab85 add context to LockedRepo#Datastore(). 2021-01-26 11:01:43 +00:00
Raúl Kripalani
ce3af308ed Merge branch 'master' into chore/snake_context_through_blockstore_init 2021-01-25 19:31:41 +00:00
Łukasz Magiera
2b3d66da3b
Merge pull request #5362 from filecoin-project/feat/fullnode-restore 
Implement full-node restore option
 2021-01-18 23:42:20 +01:00
Lucas Molas
69eb2163d2
fix(sync): enforce fork len when changing head (#5244) 2021-01-18 12:34:25 -03:00
Łukasz Magiera
38f9559080 Implement full-node restore option 2021-01-15 20:57:32 +01:00
Peter Rabbitson
446a24ef3f Fix forgotten test-call 2021-01-04 15:46:09 +01:00
Jakub Sztandera
13ff98416f
Fix cs.tipsets being unbounded 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-11-26 23:37:59 +01:00
Łukasz Magiera
54637d5093
Merge pull request #4767 from zgfzgf/feat-print-multiple-blocks 
print multiple blocks from miner cid
 2020-11-20 16:14:37 +01:00
Raúl Kripalani
0c6072a1a0 chainstore lifecycle: close via Close() instead of context. 2020-11-16 22:28:59 +00:00
Raúl Kripalani
bac51dd4e3 daemon --import-snapshot: force head silently, without triggering reorg. 2020-11-16 16:03:47 +00:00
Raúl Kripalani
d7b4f92f1f add context to ChainStore. 2020-11-16 15:52:19 +00:00
Aayush Rajasekaran
1af1ceb1e0
Merge pull request #3646 from filecoin-project/feat/delayed-pubsub-subscribe 
delayed pubsub subscribe for messages topic
 2020-11-13 17:44:32 -05:00
Łukasz Magiera
99031bae17
Merge pull request #4709 from filecoin-project/feat/mute-logs 
Make some logs quieter
 2020-11-12 20:40:10 +01:00
Aayush Rajasekaran
735c04f999
Merge pull request #4681 from filecoin-project/badger-viewable 
move to native badger blockstore; leverage zero-copy View() to deserialize in-place
 2020-11-10 18:08:12 -05:00
Raúl Kripalani
38c404e04e vm: construct CBOR store properly. 2020-11-10 15:49:33 +00:00
Raúl Kripalani
af7236f7e7 migrate direct usages blockstore.{Get=>View}. 2020-11-10 13:35:36 +00:00
vyzo
185653f85a improve coalescing 2020-11-09 13:35:42 +02:00
zgfzgf
8c8396af59 print multiple blocks from miner 2020-11-09 10:57:36 +08:00
vyzo
378d7a1ad9 stupid linter 2020-11-06 22:26:47 +02:00
vyzo
9a76c648f1 unit test for head change coalescer 2020-11-06 22:26:47 +02:00
vyzo
08aab1c48a make Close satisfy io.Closer 2020-11-06 22:26:24 +02:00
vyzo
9149ae2164 godoc and final dispatch when closing the coalescer 2020-11-06 22:26:24 +02:00
vyzo
ec13c5f80d coalesce both reverts and applys in the same manner 2020-11-06 22:26:24 +02:00
vyzo
4058f5642f head change coalescer 2020-11-06 22:26:24 +02:00
Jakub Sztandera
5a34e5b2bf
Merge pull request #4599 from filecoin-project/feat/sync-manager-redux 
rewrite sync manager
 2020-11-06 21:01:26 +01:00
Raúl Kripalani
3577300aee Merge branch 'master' into badger-viewable 2020-11-06 19:34:25 +00:00
vyzo
32b7797394 only subscribe to messages topic if we are synced within 10 epochs 2020-11-05 18:08:13 +02:00
Raúl Kripalani
370ef93db6 chainstore: new ForceHeadSilent to support benchmarks. 2020-11-04 14:39:01 +00:00
Łukasz Magiera
a1e1b03ca4 Optionally allow bitswap for chainstore 2020-11-03 23:44:44 +01:00
Łukasz Magiera
9270ac6358 Make some logs quieter 2020-11-03 13:28:41 +01:00
Raúl Kripalani
099c4b5e1d migrate repo.Datastore(/chain) to repo.Blockstore(). 2020-11-01 13:03:21 +00:00
Jakub Sztandera
e575b5fe8a
Add error if weights are the same 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-10-28 16:53:13 +01:00
Jakub Sztandera
c7e6c8eacc
Fix flaky TestChainExportImportFull 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-10-23 21:45:08 +02:00
Łukasz Magiera
6b9a4c4a34
Merge pull request #4536 from filecoin-project/zgfzgf-fix-chain-export 
Small chain export optimization
 2020-10-22 17:35:58 +02:00
Łukasz Magiera
ae04f3029b chain: Add more robust test for ChainExport 2020-10-22 16:01:11 +02:00
Steven Allen
4e730b5ec8 port to v2 imports 2020-10-21 12:16:23 -07:00
zgfzgf
b7726c4ca8 optimize walkSnapshot recurseLinks 2020-10-12 16:44:39 +08:00
Peter Rabbitson
23f5a99117 Add comment clarifying current codepath 2020-10-10 15:36:32 +02:00
Peter Rabbitson
85abca4b16 Delint 2020-10-10 04:00:12 +02:00
Peter Rabbitson
6610a247cf Dump the block validation cache whenever we perform an import 
This solves a problem with folks resurrecting long-out-of-sync nodes via
snapshot imports.

The interface switch to Batching is necessary: startup is too long otherwise
( 8 minutes just to clear everything on a relatively old node )
 2020-10-10 03:41:04 +02:00
Steven Allen
748d2e82a7 unshare the journal 
Motivation:

* Run lotus with the race detector enabled (primary motivation).
* Allow multiple lotus nodes in a process (not a high priority).

Previously, the journal was shared between all lotus instances, but it was
initialized for every new node. This caused safety problems in tests (at a
minimum).

This patch explicitly passes the journal to all services that need it.
 2020-10-09 13:23:07 -07:00
Steven Allen
b1818a1055 fix base fee tests for different smoke heights 2020-10-06 13:12:16 -07:00
Steven Allen
256be285ba only enable gas tampering if the breeze upgrade is enabled 2020-10-06 13:00:06 -07:00
Steven Allen
8292d60196 Merge branch 'master' into asr/spec-v1 2020-10-02 18:03:03 -07:00
Łukasz Magiera
f147072284 Merge remote-tracking branch 'origin/master' into feat/pruning-command 2020-10-02 23:37:25 +02:00
Steven Allen
4989b1cc90 Merge branch 'master' into asr/spec-v1 2020-09-30 09:19:12 -07:00
Aayush Rajasekaran
a4e7117429 Add lotus shed util to validate a tipset 2020-09-30 01:29:04 -04:00
Steven Allen
233d8a9b72 update even more imports 2020-09-28 14:25:58 -07:00
Dirk McCormick
cfe6f595b0 fix: unsafe append in LoadTipSet 2020-09-28 16:54:18 +02:00
Dirk McCormick
6b16d48bad refactor: fetch tipset blocks in parallel 2020-09-28 15:56:44 +02:00
Travis Person
a13d10e42d add logging to chain export 2020-09-25 20:20:18 +00:00
Łukasz Magiera
17a84c9a2c
Merge pull request #3975 from filecoin-project/steb/abstract-actor-policy 
Add some actors policy setters for testing
 2020-09-24 15:51:41 +02:00
Łukasz Magiera
2867b31d2b
Merge pull request #3208 from austinabell/extratsload 
Update beacon entry load error message
 2020-09-24 14:19:00 +02:00
Steven Allen
32a699d6a3 Add some actors policy setters for testing 
Addresses:
* a307e4593a (r491966115)
* a307e4593a (r491966634)

Note: This puts everything into a policy package to avoid a dependency cycle
between the build package, the miner package, and the types package. This is
also why I introduced a GetPreCommitChallengeDelay function and removed the
variable.
 2020-09-23 15:00:52 -07:00
Aayush Rajasekaran
a876a0ba44 Use actor state addresses 2020-09-23 02:32:40 -04:00
Steven Allen
1bf3b4989d rename imports to match actors code 
`sed -i 's/\bv0\(\w\)\(\w*\)/\L\1\E\20/g' **/*.go`
 2020-09-18 14:59:27 -07:00
Łukasz Magiera
70faa36b7f Merge remote-tracking branch 'origin/master' into refactor/net-upgrade 2020-09-18 19:29:06 +02:00
Aayush Rajasekaran
053cfc1cc7 Migrate verifreg actor 2020-09-17 03:55:23 -04:00
Aayush Rajasekaran
b5ba7a0fad Miner migration 2020-09-17 02:21:16 -04:00
Aayush Rajasekaran
b4ee519282 Partial progress towards switching to miner and power interfaces 2020-09-16 01:37:49 -04:00
whyrusleeping
6670d22fb5 add command to (slowly) prune lotus chain datastore 2020-09-15 20:20:48 -07:00
Aayush Rajasekaran
952f2c2f1e Sever chainstore's dependence on the state tree 2020-09-15 20:41:34 -04:00
Steven Allen
cc4d5306eb Progress 2020-09-14 15:43:12 -07:00
Steven Allen
02dcb5e182 Merge branch 'master' into refactor/net-upgrade 2020-09-14 14:53:57 -07:00
Łukasz Magiera
06ec571c20 Merge remote-tracking branch 'origin/master' into refactor/net-upgrade 2020-09-14 15:09:52 +02:00
Łukasz Magiera
683a58195e More terraforming in chain/ 2020-09-14 14:46:38 +02:00
Raúl Kripalani
6d29d75724 Merge branch 'master' into inmem-journal 2020-09-14 12:17:45 +01:00
whyrusleeping
0e3dd3cb3b Add faster and slimmer option to chain export via lotus-shed 2020-09-10 17:28:25 -07:00
Jakub Sztandera
6813ab4e65 Add additional metrics 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-10 17:41:55 -04:00
Jakub Sztandera
fd714cbceb Remove PackingEfficiency consideration from base fee in an upgrade 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-10 17:41:55 -04:00
Aayush Rajasekaran
183df4c135 Fix IsAncestorOf 2020-09-09 19:09:55 -04:00
Aayush Rajasekaran
c77f5f6252
Revert "only subscribe to pubsub topics once we are synced" 2020-09-08 00:32:26 -04:00
whyrusleeping
5654322f54
introduce logic to perform base fee tamping around the upgrade time 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-07 22:31:36 +02:00
Aayush Rajasekaran
39755a294a Update to specs v0.9.6 2020-09-07 15:48:41 -04:00
vyzo
4f0ab4f226 remove notifee after subscribing 2020-09-07 15:55:53 +03:00
Raúl Kripalani
1ec534d607 Merge branch 'master' into inmem-journal 2020-09-04 10:55:40 +01:00
Raúl Kripalani
97ff90f8fa fix imports. 2020-09-03 15:51:26 +01:00
austinabell
6b63fa379f
format 2020-09-02 16:45:50 -04:00
austinabell
485f13de12
Remove load check 2020-09-02 16:45:23 -04:00
austinabell
7f3ba0e6eb Merge branch 'master' of github.com:filecoin-project/lotus into extratsload 2020-09-02 16:44:06 -04:00
Raúl Kripalani
3206f92063 Merge branch 'master' into inmem-journal 2020-09-02 19:50:52 +01:00
whyrusleeping
9a5542100a load fewer messages from disk when serving blocksync requests 2020-09-02 11:43:05 -07:00
whyrusleeping
2b16e69e90 allow snapshot importing 2020-09-02 17:15:10 +02:00
whyrusleeping
09ebd1a557 allow exporting a number of recent chain state trees 2020-09-02 17:15:10 +02:00
austinabell
ba881eb4cf
Remove height from chain rand 2020-09-01 15:48:16 -04:00
Raúl Kripalani
cb8e105a94 reduce diff noise. 2020-08-26 16:44:06 +01:00