Łukasz Magiera
b13046293f
Merge pull request #5735 from filecoin-project/feat/pubsub-ip-colocatio-whitelist
...
Add configuration option for pubsub IPColocationWhitelist subnets
2021-03-08 21:01:39 +01:00
Dirk McCormick
a6c8b56ab6
feat: Default the AlwaysKeepUnsealedCopy flag to true
2021-03-08 16:35:46 +01:00
vyzo
64646de761
add configuration option for pubsub IPColocationWhitelist subnets
2021-03-06 20:05:32 +02:00
vyzo
4a74f752c0
implement extended peer info in net peers cli
2021-03-06 19:14:13 +02:00
whyrusleeping
e7a1d72ba8
extract build/version to api/version, remove api package dep on build
2021-03-05 14:28:13 -08:00
vyzo
6b8c60a659
don't ID wrap the hotstore
2021-03-05 14:46:18 +02:00
vyzo
dd0c308427
move Blockstore config to FullNode, rename to Chainstore and add default for HotStoreType
2021-03-05 14:46:18 +02:00
vyzo
6014273e69
storage miner doesn't need a splitstore
2021-03-05 14:46:18 +02:00
Raúl Kripalani
4b1e1f4b52
rename liveset => markset; rename snoop => tracking store; docs.
2021-03-05 14:46:18 +02:00
Raúl Kripalani
8cfba5b092
renames and polish.
2021-03-05 14:46:18 +02:00
Raúl Kripalani
b1b452bc0f
remove dependency from blockstore/splitstore => chain/store.
2021-03-05 14:46:18 +02:00
Raúl Kripalani
1a804fbdec
move splitstore into blockstore package.
2021-03-05 14:46:18 +02:00
Raúl Kripalani
1b51c10d78
split off lmdb support to a different branch.
2021-03-05 14:46:18 +02:00
vyzo
0fc2f3a26f
fix post-rebase compilation errors
2021-03-05 14:46:18 +02:00
vyzo
97abbe1eca
add (salted) bloom filter liveset
2021-03-05 14:46:17 +02:00
vyzo
73259aa350
add configuration for splitstore and default to a simple compaction algorithm
2021-03-05 14:46:17 +02:00
vyzo
b83994797a
separate LMDB options for hotstore and tracking stores
2021-03-05 14:46:17 +02:00
vyzo
68b6f913c7
propagate useLMDB option to splitstore through DI
2021-03-05 14:46:17 +02:00
vyzo
99c7d8e3eb
more informative names for the hotstore directories
2021-03-05 14:46:17 +02:00
vyzo
a586d42c3b
make hot store DI injectable in the split store, default to badger.
2021-03-05 14:46:17 +02:00
vyzo
622b4f7d9d
hook splitstore into DI
2021-03-05 14:46:17 +02:00
Łukasz Magiera
448813d2fe
Merge pull request #5695 from filecoin-project/feat/segregate-blockstores
...
segregate chain and state blockstores
2021-03-03 10:53:46 +01:00
Dirk McCormick
ced455bba9
feat: add configurable max provider collateral
2021-03-02 10:24:57 +01:00
Łukasz Magiera
8bd5173a54
Merge branch 'next' into refactor/lib/blockstore
2021-03-01 19:04:40 +01:00
Łukasz Magiera
3f5b17dcfc
Merge pull request #5694 from filecoin-project/asr/nits
...
Correct some logs
2021-03-01 18:22:40 +01:00
Raúl Kripalani
3795cc2bd2
segregate chain and state blockstores.
...
This paves the way for better object lifetime management.
Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
sizing caches differently.
- specifying different retention policies for chain and state data.
This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
store, and only retain state objects reachable from the last finality
in the state store.
2021-02-28 22:49:44 +00:00
Raúl Kripalani
9f0c68bb70
fix unused import post-merge.
2021-02-28 19:57:37 +00:00
Raúl Kripalani
7f0f7d0b36
Merge branch 'master' into refactor/lib/blockstore
2021-02-28 19:55:23 +00:00
Aayush Rajasekaran
48bce386cd
Correct some logs
2021-02-28 01:18:29 -05:00
Łukasz Magiera
29c9fa3137
Merge pull request #5612 from filecoin-project/feat/cc-fsm-cleanup
...
storagefsm: Cleanup CC sector creation
2021-02-26 11:39:28 +01:00
Łukasz Magiera
e49a412f6d
Merge pull request #5375 from filecoin-project/feat/refactor-fsm-input
...
storagefsm: Rewrite input handling
2021-02-25 14:28:04 +01:00
Łukasz Magiera
26399dba70
Update markets, cbor-gen with soft map decoding
2021-02-19 20:11:43 +01:00
Łukasz Magiera
90b8612cf4
Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input
2021-02-16 22:39:06 +01:00
Łukasz Magiera
39ad3d3b27
Check format in client commP util
2021-02-16 20:48:31 +01:00
Łukasz Magiera
fd90c03018
Roturn SectorID from PledgeSector
2021-02-16 19:16:35 +01:00
Łukasz Magiera
f719765069
storageminer: exit PledgeSector after sectors enter sealing pipeline
2021-02-16 17:41:58 +01:00
Łukasz Magiera
46b880a6b9
storageminer: Skip uninitialized sectors in SectorsList
2021-02-16 17:34:06 +01:00
Łukasz Magiera
fc5e243c92
storagefsm: Cleanup CC sector creation
2021-02-16 17:14:59 +01:00
Łukasz Magiera
dd82729f60
Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input
2021-02-16 13:45:39 +01:00
Dirk McCormick
a6e3856776
Revert "feat: deals - show data transfer %"
...
This reverts commit b6c9ddccff
.
2021-02-16 12:32:45 +01:00
Łukasz Magiera
7111e924cc
Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input
2021-02-11 13:30:22 +01:00
Łukasz Magiera
95e47cf998
Merge pull request #5553 from filecoin-project/feat/data-xfer-percent
...
show data transfer % for storage deals
2021-02-11 13:29:43 +01:00
Łukasz Magiera
0c6aef221d
Fix error logging format strings
2021-02-11 12:00:26 +01:00
Dirk McCormick
b6c9ddccff
feat: deals - show data transfer %
2021-02-10 09:56:23 +01:00
whyrusleeping
e387f3810e
add code cid to stateReadState output
2021-02-09 14:18:19 -08:00
Łukasz Magiera
e92b8b24b1
Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input
2021-02-09 14:04:21 +01:00
Łukasz Magiera
adeb4cefee
node builder: Cleanup and add some docstrings
2021-02-09 13:52:03 +01:00
Łukasz Magiera
6de62411c6
Merge pull request #5538 from filecoin-project/feat/deal-publisher-force
...
add commands to list pending deals and force publish
2021-02-05 23:00:52 +01:00
Łukasz Magiera
b3f4e50c58
market: APIs to manage pending deals
2021-02-05 18:58:55 +01:00
Łukasz Magiera
2c772d6fda
skip TestDealMining in short mode
2021-02-05 13:18:41 +01:00
Łukasz Magiera
8f603717a6
Merge remote-tracking branch 'origin/master' into steb/refactor-consistent-tipset-methods
2021-02-05 13:08:49 +01:00
whyrusleeping
0efb4ca3a7
add an api endpoint to get pending transactions for multisigs
2021-02-03 20:46:10 -08:00
Łukasz Magiera
ba571794dc
Merge pull request #5309 from filecoin-project/feat/deal-batch-publish
...
batch publish deal messages
2021-02-04 01:19:56 +01:00
Łukasz Magiera
6e7fcb7ee6
Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input
2021-02-03 14:01:56 +01:00
Łukasz Magiera
9b65793509
Merge pull request #5404 from filecoin-project/feat/test-window-post
...
Retry proofs if we generate an incorrect one
2021-02-03 13:04:33 +01:00
Łukasz Magiera
1e7a107484
Merge pull request #5433 from filecoin-project/feat/keep-unsealed-cfg
...
Add miner config to always keep ensealed deal copies
2021-02-03 12:40:02 +01:00
Łukasz Magiera
332ea8a126
Merge remote-tracking branch 'origin/master' into feat/deal-batch-publish
2021-02-02 18:21:14 +01:00
Dirk McCormick
a19d6ce8a3
fix: zero price-per-byte retrieval
2021-02-01 16:55:18 +01:00
Dirk McCormick
987f41011a
refactor: move publish msg config inside deals config
2021-02-01 10:23:05 +01:00
Steven Allen
feb24c9684
choose the correct nonce
...
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2021-01-30 10:18:51 -08:00
Łukasz Magiera
56a9d05491
Merge remote-tracking branch 'origin/master' into steb/refactor-consistent-tipset-methods
2021-01-30 12:05:21 +01:00
Raúl Kripalani
d1104fec4c
rename blockstores for consistency.
2021-01-29 23:17:25 +00:00
Raúl Kripalani
af429dba0f
Merge branch 'master' into chore/snake_context_through_blockstore_init
2021-01-29 21:21:46 +00:00
Raúl Kripalani
b0cbc932bd
consolidate all blockstores in blockstore package.
2021-01-29 20:01:00 +00:00
Steven Allen
8d3cc632ac
add support for running pre-migrations and caching their results
...
This can significantly speedup state migrations.
2021-01-27 12:55:53 -08:00
Łukasz Magiera
d4c7b63aac
Add miner config to always keep ensealed deal copies
2021-01-26 17:50:31 +01:00
Raúl Kripalani
a1da1dab85
add context to LockedRepo#Datastore().
2021-01-26 11:01:43 +00:00
Łukasz Magiera
fabcbb621d
Merge remote-tracking branch 'origin/master' into next
2021-01-26 00:47:22 +01:00
Łukasz Magiera
422f0991e9
Merge pull request #5379 from filecoin-project/asr/disputer
...
Build a WindowPoSt disputer
2021-01-26 00:46:26 +01:00
Aayush Rajasekaran
29d3d746a3
Build a WindowPoSt disputer
2021-01-25 17:26:09 -05:00
Raúl Kripalani
ce3af308ed
Merge branch 'master' into chore/snake_context_through_blockstore_init
2021-01-25 19:31:41 +00:00
Dirk McCormick
adac340f3f
feat: batch publish deal messages
2021-01-25 14:25:57 +01:00
Steven Allen
0a294f146f
ensure we can't dispute a correct window post
2021-01-22 14:30:54 -08:00
Steven Allen
8de4e5e005
retry proofs if we generate an incorrect one
2021-01-22 13:31:47 -08:00
Steven Allen
0131b077be
re-enable watchdog
2021-01-22 11:19:37 -08:00
Jakub Sztandera
8c7fda4441
fix(gas): when estimating GasLimit only apply priors up to the nonce
...
The bug is applying all messages from given From address are priors
before appling the message that we are estimating.
If user tries replacing message in the middle with gas limit estimation
then message sequence is off and user will either get an execution error
or gas mis-esimation.
Resolves #5402
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-01-22 18:50:02 +01:00
Steven Allen
1b67eed82c
make the dispute test run by itself
...
It's kind of slow.
2021-01-21 20:16:03 -08:00
Steven Allen
0ba1306969
test prove then dispute
2021-01-21 19:48:34 -08:00
Steven Allen
d412ccdb3f
disable the watchdog
...
It's causing the tests to flake.
2021-01-21 18:28:37 -08:00
Steven Allen
026d236252
more reliably close in tests
2021-01-21 15:21:20 -08:00
Steven Allen
03cd3760bb
correctly pick the seal type based on the network version
...
Of course, we should really just run all of our tests post actors v2.
2021-01-21 15:21:20 -08:00
Aayush Rajasekaran
ab90a3b2bc
Actors update: MinerInfo.SealProofType has been removed
2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
82635f2515
Rename Deadline.PoStSubmissions to PartitionsPoSted
2021-01-21 15:21:19 -08:00
Łukasz Magiera
1070ad2289
storagefsm: Drop unused TargetWaitDealsSectors
2021-01-21 22:20:16 +01:00
Łukasz Magiera
542357a1df
storagefsm: Start packing correctly
2021-01-21 17:40:50 +01:00
Łukasz Magiera
239d6f8f4d
storagefsm: Rewrite input handling
2021-01-21 17:40:50 +01:00
Raúl Kripalani
b9adf95010
watchdog: increase heapprof capture threshold to 90%.
2021-01-21 14:06:18 +00:00
Raúl Kripalani
931cfe1ed1
upgrade to raulk/go-watchdog@v1.0.1
...
This pulls in the improvements introduced in:
- https://github.com/raulk/go-watchdog/releases/tag/v1.0.0
- https://github.com/raulk/go-watchdog/releases/tag/v1.0.1
Lotus tries to initialize the watchdog in the following order of precedence:
1. If a max heap limit has been provided, initialize a heap-driven watchdog.
2. Else, try to initialize a cgroup-driven watchdog.
3. Else, try to initialize a system-driven watchdog.
4. Else, log a warning that the system is flying solo, and return.
This PR also enabled automatic heap profile capture when memory usage
surpasses 90% of the limit. Profiles are written to <LOTUS_HOME>/heapprof.
A single heap profile is captured per episode, with a max of 10 episodes
captured during the lifetime of the process. Episode = instance of usage
climbing above the 90% threshold.
2021-01-20 18:09:19 +00:00
Łukasz Magiera
7a042ab21f
Merge pull request #5210 from filecoin-project/feat/mkts-miner-dont-dial-client
...
Miner should not dial client on restart
2021-01-20 13:37:09 +01:00
Dirk McCormick
1206154718
feat: update to go-fil-markets v1.1.0 for better retry config
2021-01-20 10:00:02 +01:00
Łukasz Magiera
0a85dc6232
Merge pull request #5369 from filecoin-project/feat/gas-premium-55
...
Use 55th percentile instead of median for gas-price
2021-01-19 19:10:47 +01:00
Dirk McCormick
827a473391
feat: expose StateSearchMessage on gateway
2021-01-19 11:27:43 +01:00
Jakub Sztandera
eeadfedbe6
Use 55th percentile instead of median for gas-price
...
The aim is to put some negative pressure on gas-premium instead of
maintining status quo.
55th percentile instead of median should not make much difference for
block inclusion timing.
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-01-18 15:44:20 +01:00
Dirk McCormick
835fd4b23c
feat: markets - miner should not dial client on restart
2021-01-14 16:38:56 +01:00
Łukasz Magiera
144b5a1350
perning termination API
2021-01-14 12:37:23 +01:00
Łukasz Magiera
1564db1fce
Sector termination test
2021-01-14 00:11:41 +01:00
Łukasz Magiera
3522c8d45a
SectorTerminateFlush API
2021-01-13 23:32:04 +01:00
Łukasz Magiera
52cc2cd3eb
Initial sector termination support
2021-01-13 00:42:01 +01:00
Peter Rabbitson
47141042b6
Add comment
...
Co-authored-by: raulk <raul@protocol.ai>
2021-01-12 18:11:18 +01:00
Dirk McCormick
c58086ee27
feat: better CLI for wallet market withdraw and client info
2021-01-07 10:18:29 +01:00
Dirk McCormick
c6820ec059
feat: lotus wallet market add (adds funds to storage market actor)
2021-01-06 14:32:46 +01:00
Peter Rabbitson
9334e73396
Snake a context through the Chain-blockstore creation
2021-01-04 15:27:52 +01:00
Łukasz Magiera
2e154ef6d0
Merge pull request #5176 from filecoin-project/shaodan-miner-sectors-info
...
Shaodan miner sectors info
2020-12-10 20:09:15 +01:00
Łukasz Magiera
0cb2f51549
Merge pull request #5094 from filecoin-project/asr/verified-only
...
Allow miners to filter (un)verified deals
2020-12-10 19:48:22 +01:00
Łukasz Magiera
564b88af52
docsgen, lint
2020-12-10 19:36:02 +01:00
Łukasz Magiera
d347e7ab8f
Merge branch 'miner-sectors-info' of github.com:shaodan/lotus into shaodan-miner-sectors-info
2020-12-10 19:32:26 +01:00
Steven Allen
e164dbb953
Compute the next nonce from the tipset, instead of actually executing the tipset
...
We could also do this in the message pool itself, but I'm not sure if it's worth it?
2020-12-09 11:29:41 -08:00
Steven Allen
b8e3808c4f
Make state tipset usage consistent in the API
...
_Always_ (almost) use the tipset's parent state, instead of computing.
Exceptions:
* MinerGetBaseInfo. Fixing this would break things so we need to be
careful (although we could bump the API version, fix it, then fix the call
sites).
* StateReplay. This is replaying a message on top of the given tipset.
* GasEstimateGasLimit. This executes the message on-top-of the tipset's
computed state (unlike call which executes it on the tipset's parent state).
* Having this method and Call apply the message at different heights is really
weird.
2020-12-09 11:29:40 -08:00
Łukasz Magiera
2ce5a29004
Merge pull request #5150 from filecoin-project/feat/list-deals-xfer-id
...
show data transfer ID in list-deals
2020-12-09 19:39:46 +01:00
Jakub Sztandera
6ca5caef31
Refactor DefaultMessageSendSpec
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-12-08 20:51:27 +01:00
Dirk McCormick
72cb130e17
feat: show data transfer ID in list-deals
2020-12-08 15:23:26 +01:00
Dan Shao
d56170d70e
Optimize sectors info loading
2020-12-07 13:42:37 +08:00
Ingar Shu
13c8a235b6
Use FundManager to withdraw funds, add MarketWithdraw to API
2020-12-03 13:08:18 -08:00
Łukasz Magiera
dc06d30f52
Merge pull request #5101 from filecoin-project/raulk/memory-watchdog
...
introduce memory watchdog; LOTUS_MAX_HEAP
2020-12-03 15:46:09 +01:00
Raúl Kripalani
f9cbf6d526
watchdog: add LOTUS_DISABLE_WATCHDOG escape hatch.
2020-12-03 13:16:10 +00:00
Raúl Kripalani
d033f8aab1
watchdog: remove silence period, reduce tick interval to 5 secs.
2020-12-02 22:31:57 +00:00
Raúl Kripalani
e34a759889
avoid global ResourceConstraints.
2020-12-02 22:26:30 +00:00
Raúl Kripalani
2ef8acdfc8
back out index cache usage in badger blockstore.
2020-12-02 21:35:13 +00:00
Łukasz Magiera
c3a0feef96
miner: Improve default address config
2020-12-02 21:54:38 +01:00
Łukasz Magiera
2fd93a55ac
plumb AddressSelector to stoage fsm
2020-12-02 21:47:54 +01:00
Łukasz Magiera
783971d2c8
docsgen, gofmt
2020-12-02 20:46:40 +01:00
Łukasz Magiera
f74a1b70ab
miner: Fix actor control list cli
2020-12-02 20:46:07 +01:00
Łukasz Magiera
7246ef273f
miner: Control address config for (pre)commits
2020-12-02 19:58:00 +01:00
Raúl Kripalani
e8a5e0d2fe
integrate memory watchdog; impose limits on badger caches.
2020-12-02 16:40:28 +00:00
Raúl Kripalani
655aea1018
Merge branch 'master' into raulk/badger-index-cache
2020-12-02 11:46:57 +00:00
Aayush Rajasekaran
370817eb60
Allow miners to filter (un)verified deals
2020-12-02 01:32:34 -05:00
Łukasz Magiera
95eaf13b5a
sectorstorage: Fix tests
2020-12-02 00:36:32 +01:00
Łukasz Magiera
c6f85886bf
miner: Add slow mode to proving check
2020-12-02 00:32:14 +01:00
Łukasz Magiera
82b5cb89cd
Merge pull request #5061 from filecoin-project/fix/err-late-deals
...
Error out deals that are not activated by proposed deal start epoch
2020-12-01 17:02:06 +01:00
Łukasz Magiera
614f45dcfb
Merge pull request #5041 from filecoin-project/feat/5013
...
Sector check command
2020-12-01 14:59:01 +01:00
Dirk McCormick
cb044f83a7
fix: ensure deal start is far enough in future for tests
2020-12-01 14:25:48 +01:00
Jakub Sztandera
65d50c76d6
bonus: fix WalletSign using t0 addresses in WalletSignMessage
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-11-27 17:19:41 +01:00
Łukasz Magiera
fcec665267
Merge remote-tracking branch 'origin/master' into feat/deal-await-precommit
2020-11-26 11:51:36 +01:00
s1m0n21
663b3f4f21
add a sector check command
2020-11-26 15:02:43 +08:00
Łukasz Magiera
2e544e3e6a
configure SimultaneousTransfers in node/builder
2020-11-25 19:57:38 +01:00
hannahhoward
694834e8d5
feat(graphsync): configure simultaneous requests
...
allow configuration of the number of simultaneous requests graphsync will process at once
2020-11-24 14:32:30 -08:00
Dirk McCormick
239e180683
feat: markets - separate watching for pre-commit from prove-commit
2020-11-24 13:24:41 -08:00
hannahhoward
afec58f780
feat(markets): add logging on data-transfer to miner
2020-11-23 12:43:30 -08:00
Łukasz Magiera
be2a5942fb
Merge pull request #4082 from leviok/fix-nil-pointer
...
fix nil pointer in StateSectorPreCommitInfo
2020-11-23 18:58:32 +01:00
Łukasz Magiera
0e39e760e5
Merge pull request #4929 from filecoin-project/feat/update-markets-commp-extraction
...
Remove all temp file generation for deals
2020-11-23 18:43:00 +01:00
Łukasz Magiera
7d540905ac
Don't return nil in StateSectorPreCommitInfo
2020-11-23 12:14:49 +01:00
Dirk McCormick
4a8df0c3fa
feat: add data transfer logging
2020-11-23 10:31:39 +01:00
leviok
9fe3649dd2
correct it
2020-11-23 11:08:21 +08:00
leviok
9b199f0476
fix nil pointer
2020-11-23 11:08:21 +08:00
hannahhoward
218f7c2c5d
feat(markets): update markets, extract commp
2020-11-20 13:29:14 -08:00
Łukasz Magiera
4483d94db3
Merge pull request #4902 from GFZRZK/GFZRZK/patch/remove_unuseful_code_in_newstorageask
...
just return storedask.NewStoredAsk to reduce unuseful code
2020-11-20 16:28:03 +01:00
Łukasz Magiera
94763c2aaa
Merge pull request #4849 from filecoin-project/feat/conngater
...
Connection Gater support
2020-11-20 16:25:02 +01:00
Raúl Kripalani
44040f0f3f
badger options: enable IndexCache and CompactL0OnClose.
2020-11-20 00:33:43 +00:00
Raúl Kripalani
55061ae6fb
fix blockstore directory not created automatically.
2020-11-19 15:55:02 +00:00
wangchao
49f8590578
just return storedask.NewStoredAsk to reduce unuseful code
2020-11-18 13:53:48 +08:00