Commit Graph
12756 Commits
Author SHA1 Message Date
vyzo 669b47cfc9 do moving gc for hotstore every 20 compactions
that's about once a week
2021-07-14 22:59:53 +03:00
vyzo 818b8de182 keep track of the compaction serial (index)
it is useful so that:
- we only do slow (but very effective) moving gc every 10 compactions
- we can detect a splitstore v0 upgrade and re-warm up
2021-07-14 22:59:53 +03:00
vyzo c93328b036 use the new traits for hotstore gc 2021-07-14 22:59:52 +03:00
vyzo 35180b4761 merge Compact and CollectGarbage in badger 2021-07-14 22:59:52 +03:00
vyzo dc81c0e6a2 add blockstore traits related to gc 2021-07-14 22:59:52 +03:00
Łukasz Magiera f0c9e4e8f2 Merge pull request #6746 from filecoin-project/chore/add_power_to_shed_storage-stats 2021-07-14 11:54:43 +02:00
Peter Rabbitson a37ababc20 Minor rearrange 2021-07-14 08:32:07 +02:00
Peter Rabbitson 677c193e6c Add current power to the stats 2021-07-14 08:11:23 +02:00
Łukasz Magiera dd09666399 Merge pull request #6743 from filecoin-project/chore/centralize_all_cli_color_handling
Handle the --color flag via proper global state
2021-07-13 17:34:25 +02:00
Łukasz Magiera 583a8a13d9 Merge pull request #6629 from filecoin-project/feat/pledge-from-miner-balance
Config for collateral from miner available balance
2021-07-13 17:22:00 +02:00
Łukasz Magiera 837322ea59 Merge pull request #6356 from filecoin-project/nonsense/split-market-miner-processes
Support standalone miner-market process
2021-07-13 17:16:07 +02:00
Łukasz Magiera 0678b3fa05 sealing: Cleanup DisableCollateralFallback handling 2021-07-13 17:06:41 +02:00
Łukasz Magiera 7526a074d9 sealing: collateral buffer / falback config 2021-07-13 17:04:58 +02:00
Anton Evangelatov 2af02af1c1 rename --name to --type; provide empty stores.StorageConfig when init new service 2021-07-13 12:57:43 +02:00
Łukasz Magiera c37401a1a3 Merge pull request #6474 from filecoin-project/feat/splitstore-redux
Splitstore Enhanchements
2021-07-13 12:43:57 +02:00
Peter Rabbitson 6efd2e7335 Handle the --color flag via proper global state
Use `color.NoColor` as a global progressively modified as one descends
down command flags. Allows non-surprising operation when --color is
accepted at multiple layers, and even makes this behave predictably:

    lotus-miner --color actor control list --color=false
2021-07-13 12:22:29 +02:00
vyzo af399529ec finetune view waiting 2021-07-13 09:06:40 +03:00
vyzo 257423e917 fix view waiting issues with the WaitGroup
We can add after Wait is called, which is problematic with WaitGroups.
This instead uses a mx/cond combo and waits while the count is > 0.
The only downside is that we might needlessly wait for (a bunch) of views
that started while the txn is active, but we can live with that.
2021-07-13 09:01:50 +03:00
Steven Allen 04abd190ab nit: remove useless goto
Because stebalien has allergies.
2021-07-12 21:46:50 -07:00
vyzo 60212c86cb put a mutex around HeadChange 2021-07-13 03:14:13 +03:00
vyzo 759594d01c always return the waitgroup in protectView
so that we preclude the following scenario:
    Start compaction.
    Start view.
    Finish compaction.
    Start compaction.

which would not wait for the view to complete.
2021-07-13 03:11:40 +03:00
raulk 5c3e79e8d0 Merge pull request #6739 from filecoin-project/fix/cancel-stops-retrieval 2021-07-12 23:22:35 +01:00
hannahhoward 92ec239202 fix(node): ClientRetrieve stops on cancel
When a deal is cancelled, any in progress calls to ClientRetrieve or ClientRetrieveWithEvents should
return
2021-07-12 14:46:18 -07:00
Łukasz Magiera 4433c3cd7c Merge pull request #6708 from filecoin-project/feat/miner-info-deals
Add more deal details to lotus-miner info
2021-07-12 22:18:56 +02:00
Anton Evangelatov 7728d6bfc3 remove --enable-markets flag; add --name string slice flag 2021-07-12 14:38:29 +02:00
Łukasz Magiera 83f2368507 Add CollateralFromMinerBalance config 2021-07-12 14:11:58 +02:00
Anton Evangelatov 4bc9fa04ba rename cfg.Subsystems.EnableStorageMarket to EnableMarkets 2021-07-12 12:12:29 +02:00
Anton Evangelatov 056136ef5a rename argument from url to apiInfo 2021-07-12 11:36:22 +02:00
Anton Evangelatov a423c46f2a rename mock.go to net.go 2021-07-12 11:35:03 +02:00
Anton Evangelatov d89ddb9315 resolve conflicts 2021-07-12 11:34:37 +02:00
Anton Evangelatov 16784aa2cc remove pieceProvider from DI; small refactors 2021-07-12 11:30:26 +02:00
Aayush Rajasekaran 88ac2ebc0c Merge pull request #6710 from filecoin-project/arajasek-patch-1
Release template: Update all testnet infra at once
2021-07-11 17:26:43 -04:00
Aayush Rajasekaran 13dcfec8fe Release template: Update all testnet infra at once 2021-07-11 17:26:21 -04:00
Aayush Rajasekaran 20582037c5 Fix Lotus shed 2021-07-11 14:03:22 -04:00
Aayush Rajasekaran e983d90a28 Merge pull request #6066 from chadwick2143/extend-v1-sectors
Fix bugs in sectors extend --v1-sectors
2021-07-11 13:19:35 -04:00
Aayush Rajasekaran ea9ad8c5a8 Update cmd/lotus-storage-miner/sectors.go 2021-07-11 13:01:19 -04:00
Aayush Rajasekaran 93d929a10a Merge pull request #5950 from zgfzgf/feat-shed-election-backtest
add election backtest
2021-07-11 12:08:21 -04:00
vyzo df9670c58d fix lint 2021-07-10 16:38:40 +03:00
vyzo 0c5e336ff1 address review comments 2021-07-10 16:30:27 +03:00
Aayush Rajasekaran f7f5fd4809 Merge pull request #6620 from filecoin-project/asr/no-filter
Envvar to disable slash filter
2021-07-09 17:12:16 -04:00
Aayush RajasekaranandPeter Rabbitson dff097a992 Update miner/miner.go
Co-authored-by: Peter Rabbitson <ribasushi@protocol.ai>
2021-07-09 17:07:23 -04:00
vyzo 870a47f55d handle id cids in internal versions of view/get 2021-07-09 20:07:17 +03:00
vyzo f5ae10e3d1 refactor debug log code to eliminate duplication 2021-07-09 19:53:51 +03:00
vyzo 41290383e2 fix test 2021-07-09 19:24:44 +03:00
vyzo b9a5ea8f7b update wording around discard store 2021-07-09 19:23:55 +03:00
vyzo c0a1cfffa1 rename noopstore to discardstore 2021-07-09 19:19:37 +03:00
vyzo 18161fee38 remove unused lookback constructs 2021-07-09 19:12:58 +03:00
vyzo 095d7427ba make view protection optimistic again, as there is a race window 2021-07-09 15:41:10 +03:00
vyzo da0feb3fa4 dont mark references inline; instad rely on the main compaction thread to do concurrent marking
The problem is that it is possible that an inline marking might take minutes for some objects
(infrequent, but still possible for state roots and prohibitive if that's a block validation).
So we simply track references continuously and rely on the main compaction thread to trigger
concurrent marking for all references at opportune moments.

Assumption: we can mark references faster than they are created during purge or else we'll
never purge anything.
2021-07-09 15:10:02 +03:00
vyzo acc4c374ef properly handle protecting long-running views 2021-07-09 13:20:18 +03:00