Commit Graph

238 Commits

Author SHA1 Message Date
Łukasz Magiera
6b9a4c4a34
Merge pull request #4536 from filecoin-project/zgfzgf-fix-chain-export
Small chain export optimization
2020-10-22 17:35:58 +02:00
Steven Allen
4e730b5ec8 port to v2 imports 2020-10-21 12:16:23 -07:00
zgfzgf
b7726c4ca8 optimize walkSnapshot recurseLinks 2020-10-12 16:44:39 +08:00
Peter Rabbitson
23f5a99117 Add comment clarifying current codepath 2020-10-10 15:36:32 +02:00
Peter Rabbitson
85abca4b16 Delint 2020-10-10 04:00:12 +02:00
Peter Rabbitson
6610a247cf Dump the block validation cache whenever we perform an import
This solves a problem with folks resurrecting long-out-of-sync nodes via
snapshot imports.

The interface switch to Batching is necessary: startup is too long otherwise
( 8 minutes just to clear everything on a relatively old node )
2020-10-10 03:41:04 +02:00
Steven Allen
748d2e82a7 unshare the journal
Motivation:

* Run lotus with the race detector enabled (primary motivation).
* Allow multiple lotus nodes in a process (not a high priority).

Previously, the journal was shared between all lotus instances, but it was
initialized for every new node. This caused safety problems in tests (at a
minimum).

This patch explicitly passes the journal to all services that need it.
2020-10-09 13:23:07 -07:00
Steven Allen
8292d60196 Merge branch 'master' into asr/spec-v1 2020-10-02 18:03:03 -07:00
Łukasz Magiera
f147072284 Merge remote-tracking branch 'origin/master' into feat/pruning-command 2020-10-02 23:37:25 +02:00
Steven Allen
4989b1cc90 Merge branch 'master' into asr/spec-v1 2020-09-30 09:19:12 -07:00
Aayush Rajasekaran
a4e7117429 Add lotus shed util to validate a tipset 2020-09-30 01:29:04 -04:00
Steven Allen
233d8a9b72 update even more imports 2020-09-28 14:25:58 -07:00
Dirk McCormick
cfe6f595b0 fix: unsafe append in LoadTipSet 2020-09-28 16:54:18 +02:00
Dirk McCormick
6b16d48bad refactor: fetch tipset blocks in parallel 2020-09-28 15:56:44 +02:00
Travis Person
a13d10e42d add logging to chain export 2020-09-25 20:20:18 +00:00
Łukasz Magiera
2867b31d2b
Merge pull request #3208 from austinabell/extratsload
Update beacon entry load error message
2020-09-24 14:19:00 +02:00
whyrusleeping
6670d22fb5 add command to (slowly) prune lotus chain datastore 2020-09-15 20:20:48 -07:00
Aayush Rajasekaran
952f2c2f1e Sever chainstore's dependence on the state tree 2020-09-15 20:41:34 -04:00
Raúl Kripalani
6d29d75724 Merge branch 'master' into inmem-journal 2020-09-14 12:17:45 +01:00
whyrusleeping
0e3dd3cb3b Add faster and slimmer option to chain export via lotus-shed 2020-09-10 17:28:25 -07:00
Aayush Rajasekaran
183df4c135 Fix IsAncestorOf 2020-09-09 19:09:55 -04:00
Aayush Rajasekaran
c77f5f6252
Revert "only subscribe to pubsub topics once we are synced" 2020-09-08 00:32:26 -04:00
Aayush Rajasekaran
39755a294a Update to specs v0.9.6 2020-09-07 15:48:41 -04:00
vyzo
4f0ab4f226 remove notifee after subscribing 2020-09-07 15:55:53 +03:00
Raúl Kripalani
1ec534d607 Merge branch 'master' into inmem-journal 2020-09-04 10:55:40 +01:00
austinabell
6b63fa379f
format 2020-09-02 16:45:50 -04:00
austinabell
485f13de12
Remove load check 2020-09-02 16:45:23 -04:00
austinabell
7f3ba0e6eb Merge branch 'master' of github.com:filecoin-project/lotus into extratsload 2020-09-02 16:44:06 -04:00
Raúl Kripalani
3206f92063 Merge branch 'master' into inmem-journal 2020-09-02 19:50:52 +01:00
whyrusleeping
9a5542100a load fewer messages from disk when serving blocksync requests 2020-09-02 11:43:05 -07:00
whyrusleeping
2b16e69e90 allow snapshot importing 2020-09-02 17:15:10 +02:00
whyrusleeping
09ebd1a557 allow exporting a number of recent chain state trees 2020-09-02 17:15:10 +02:00
austinabell
ba881eb4cf
Remove height from chain rand 2020-09-01 15:48:16 -04:00
Raúl Kripalani
efdfd3ee3e Merge branch 'master' into inmem-journal 2020-08-26 16:38:23 +01:00
Raúl Kripalani
4e1ef09751 make journal a global var. 2020-08-26 16:09:37 +01:00
whyrusleeping
cf3298cd04 more correctly handle discrepancies between mempools head and the mining base 2020-08-24 17:21:03 -07:00
Steven Allen
5733c71c50 Lint everything
We were ignoring quite a few error cases, and had one case where we weren't
actually updating state where we wanted to. Unfortunately, if the linter doesn't
pass, nobody has any reason to actually check lint failures in CI.

There are three remaining XXXs marked in the code for lint.
2020-08-20 20:46:36 -07:00
austinabell
e4daadbfcf
Remove extra tipset load from checking beacon entries 2020-08-20 17:54:59 -04:00
whyrusleeping
13e5b72cdb proper genesis block history
Commit hash originally stamped into the ethereum blockchain:
https://etherscan.io/tx/0xe8f51c9eefb682cd866f059462577b6dd3d2685ff4b6437f6c940ff4f4aaf067
2020-08-17 21:13:43 -07:00
Łukasz Magiera
ba28169356 Merge remote-tracking branch 'origin/master' into next 2020-08-12 21:33:06 +02:00
Whyrusleeping
ed72aa1307 implement randomness distinction 2020-08-12 10:32:39 -07:00
whyrusleeping
1fc768fb32 add env vars to configure chain store cache sizes 2020-08-11 15:07:17 -07:00
Raúl Kripalani
92598237c3 Merge branch 'next' into inmem-journal 2020-08-11 12:41:11 +01:00
Aayush Rajasekaran
532c32a741 Add new BlockMsgsForTipset() 2020-08-10 15:55:44 -04:00
Steven Allen
b7a4dbb07f Support inline CIDs
And use the new CidBuilder from the spec actors.

This patch does not switch over to inline CIDs by default, but paves the way.
2020-07-23 23:12:32 -07:00
Steven Allen
5fc83f4d05 Refactor to use actor adt types instead of directly using HAMTs and AMTs
This way there's a single source of truth. Preparation for fixing
https://github.com/filecoin-project/specs-actors/issues/517 (requires changing
HAMT parameters).
2020-07-23 09:34:57 -07:00
Łukasz Magiera
d70edbcb7c Set things on syscallShim 2020-07-22 13:35:10 -04:00
Raúl Kripalani
b8475114ba Merge branch 'next' into inmem-journal 2020-07-21 17:42:51 +01:00
Raúl Kripalani
4e82cf369c rename journal entries to journal events. 2020-07-21 17:32:01 +01:00
whyrusleeping
42a7cb1248 fix a few multisig cli issues 2020-07-20 17:17:53 -07:00
Raúl Kripalani
226786c1da wip 2020-07-20 14:45:17 +01:00
Raúl Kripalani
4d2d8b2d11 Merge branch 'next' into inmem-journal 2020-07-20 10:38:58 +01:00
Raúl Kripalani
4bcf01938c use new journal from chain/store pkg. 2020-07-17 18:54:26 +01:00
Raúl Kripalani
695f6cfe45 wip in-memory journal. 2020-07-17 14:34:42 +01:00
Łukasz Magiera
faebc4c948 WIP Integrating specs-actors with refactored miner state 2020-07-14 13:45:45 +02:00
Łukasz Magiera
b049c5a7d1 Merge remote-tracking branch 'origin/master' into next 2020-07-08 14:35:40 +02:00
whyrusleeping
42bd4eccbe implement a persistent journal for lotus node operations 2020-07-06 09:42:00 -07:00
Łukasz Magiera
4c422c2d50 specs-actors v0.7.1 2020-07-01 13:47:40 +02:00
Raúl Kripalani
c8104a03e6
some initial godocs. (#2118) 2020-06-23 18:51:25 -03:00
Łukasz Magiera
ab4c11c096 Merge remote-tracking branch 'origin/master' into next 2020-06-16 12:04:29 +02:00
Howard Yeh
c822816f8e Cache tipset validation progress 2020-06-14 18:41:17 +08:00
Łukasz Magiera
8e025db656
Merge pull request #2014 from filecoin-project/fix/chain-reorgs-race
fix a potential race with chain reorgs notifees.
2020-06-12 20:35:16 +02:00
Raúl Kripalani
4e9293ba04 fix a potential race with chain reorgs notifees. 2020-06-12 19:18:23 +01:00
Jeromy
512270593b clean up some of the more spammy logs 2020-06-11 12:31:29 +02:00
Jeromy
686c543778 clean up some of the more spammy logs 2020-06-09 12:49:31 -07:00
Whyrusleeping
366359803a
Merge pull request #1903 from filecoin-project/feat/reuse-get-tipset-by-height
reuse GetTipsetByHeight in GetRandomness
2020-06-05 16:06:28 -07:00
Łukasz Magiera
163c2b2544 Merge remote-tracking branch 'origin/master' into next 2020-06-06 00:58:24 +02:00
Jeromy
73d928751d disallow sourcing randomness from future epochs 2020-06-05 15:12:13 -07:00
Aayush Rajasekaran
656b285195 Bugfix: Begin walkback when lookback target's height is too low 2020-06-04 18:17:54 -04:00
whyrusleeping
86083531c6 a couple bugfixes 2020-06-04 18:17:54 -04:00
Jeromy
c01f70105f implement chain index to make lookbacks faster 2020-06-04 18:17:54 -04:00
Jeromy
cd817906b2 handle sourcing randomness from a future block 2020-06-03 18:30:43 -07:00
Jeromy
2fdcf5347b reuse GetTipsetByHeight in GetRandomness 2020-06-02 17:35:23 -07:00
Jakub Sztandera
5605aae269
Fix even more lint warnings
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-06-02 16:29:39 +02:00
Jakub Sztandera
d6615b6286
Cleanup many lint warnings
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-05-27 22:53:20 +02:00
Łukasz Magiera
523a7a9d8d Merge pull request #1670 from filecoin-project/feat/update-markets-0.2.0
Update go-fil-markets
2020-05-06 01:45:33 +02:00
Jeromy
74510adaaa add flag to choose what to do when looking back tipsets in null rounds 2020-05-05 10:06:05 -07:00
hannahhoward
a9866c757e feat(deps): update go-fil-markets
Update to latest 0.2.0 release of go-fil-markets with less-blocking node io
2020-05-04 18:36:51 -07:00
Jeromy
ef1d485c6e move head change type out of chainstore package 2020-04-23 15:28:59 -07:00
Łukasz Magiera
ecc38d1bd8 sync: env var to fix drand 2020-04-20 20:33:53 +02:00
Jeromy
91f56742c4 Use lookback to get sector set and power for miners 2020-04-17 16:44:18 -07:00
Jeromy
4e6839b5ef some small cleanup 2020-04-14 15:13:52 -07:00
Jakub Sztandera
5ab9636365 Call VerifyBeacon, hack around genesis
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-04-14 16:52:18 +02:00
Jakub Sztandera
ce4978d8c3 Reintroduce correct ticket randomness alongside ElectionProof
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-04-08 21:06:41 +02:00
Jakub Sztandera
f1c890ca54 Tests pass
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-04-08 18:31:16 +02:00
Jeromy
8e0ca306d2 more wiring up of drandomness 2020-04-08 08:11:42 -07:00
Jakub Sztandera
9c250f24fa Nex wip
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-04-07 15:32:23 -07:00
Jakub Sztandera
afdb1db529 drand wip
,
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-04-07 15:32:23 -07:00
Jeromy
088d693eea code review changes 2020-03-25 12:13:09 -07:00
Aayush Rajasekaran
ebcefa80e3 provide msg length to vm::ApplyMessage 2020-03-25 08:25:03 -04:00
Jeromy
921f11db34 Rewrite chain export walker function to actually do the right thing 2020-03-20 17:18:57 -07:00
whyrusleeping
24bf720a9a Merge branch 'master' into testnet/3 2020-03-07 16:46:12 -08:00
Nate Walck
33af2409e8 Use head notif func for current node height, add pubsub metrics 2020-03-01 19:26:09 -05:00
whyrusleeping
cf2eefdfbe Fix up client storage deal matcher 2020-02-28 16:05:56 -08:00
Henri S
96555aa74b update to use VRF digest appropriately and simplify drawing rand past genesis 2020-02-28 11:31:28 -08:00
Łukasz Magiera
f1d9033194 Address some minor review nits 2020-02-25 21:25:52 +01:00
Łukasz Magiera
d787aa5007 Merge branch 'testnet/3' into feat/specs-actors 2020-02-24 18:32:02 +01:00
Łukasz Magiera
917df9a5b8 More unified randomness handling 2020-02-23 21:00:47 +01:00
Łukasz Magiera
69ec3c205a Merge remote-tracking branch 'origin/master' into testnet/3 2020-02-19 22:10:14 +01:00
whyrusleeping
6d0eec6395 working towards a working genesis generator 2020-02-17 23:15:30 -08:00