vyzo
7512748b56
wire in sealing config values
2021-05-18 15:31:52 +03:00
Łukasz Magiera
506f39b294
WIP: Integrate FIP0013
2021-05-11 22:10:29 -04:00
Jakub Sztandera
18cbdcfc81
Disable checks API on Lotus Lite
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-05-07 17:19:17 +02:00
Jakub Sztandera
d777680449
Fix mpool.GetActor for lite node
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-05-07 15:30:06 +02:00
vyzo
ed61642b3a
implement NodeStatus API
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-05-07 15:30:04 +02:00
Dirk McCormick
5b1fab8ffa
feat: markets v1.3
2021-05-06 09:49:44 +02:00
Jakub Sztandera
77d004ec06
Fix nonce getting on Lotus lite
...
Resolves #5593 #5995
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-04-15 19:57:09 +02:00
Łukasz Magiera
e76935147f
Merge remote-tracking branch 'origin/master' into feat/v1-api
2021-04-13 14:24:22 +02:00
Łukasz Magiera
88bdb591d5
Merge pull request #6003 from filecoin-project/fix/fbs-di
...
Fix fallback chainstore
2021-04-10 01:58:38 +02:00
Łukasz Magiera
7e4cb9da84
Fix fallback chainstore
2021-04-09 16:11:02 +02:00
Dirk McCormick
4a3261a986
feat: go-fil-markets-v1.2.5-rc1
2021-04-08 14:34:35 +02:00
Łukasz Magiera
81bd27911f
Propagate StateMsg api changes
2021-04-05 19:56:53 +02:00
Łukasz Magiera
deb2b90b6a
Fix lotus/miner build
2021-04-05 13:23:46 +02:00
Łukasz Magiera
1a5c4d70f4
Merge remote-tracking branch 'origin/master' into feat/v1-api
2021-04-01 15:33:37 +02:00
Łukasz Magiera
7a23b411ae
Merge apistruct with the api package
2021-03-25 15:10:08 +01:00
Dirk McCormick
1359fb6cc4
fix: disable pull channel monitoring for now
2021-03-24 13:56:59 +01:00
Dirk McCormick
aee9890ee7
feat: update go-data-transfer and go-fil-markets
2021-03-23 13:28:30 +01:00
Dirk McCormick
9fb1dd43a9
feat: update to markets v1.2.0
2021-03-22 15:31:13 +01:00
whyrusleeping
cbc7f1c244
fix paychmgr constructor to take an easier to implement interface
2021-03-12 00:39:20 -08:00
whyrusleeping
e0b650d4ec
make rpcstatemanager its own package, for easy reuse
2021-03-11 18:21:18 -08:00
whyrusleeping
bf42a3be5e
move DI stuff for paychmgr into modules
2021-03-11 17:35:31 -08:00
Łukasz Magiera
bd8864ae43
Merge pull request #5755 from filecoin-project/feat/kvlog
...
Metadata datastore log
2021-03-10 14:13:01 +01:00
Łukasz Magiera
05274943c9
Merge pull request #5620 from filecoin-project/feat/disable-owner-worker-fallback
...
miner: Config to disable owner/worker address fallback
2021-03-10 11:21:39 +01:00
Łukasz Magiera
d4a981f327
fix lint
2021-03-09 23:38:28 +01:00
Łukasz Magiera
ca7e70bf3a
Metadata datastore log
2021-03-09 22:33:01 +01:00
Łukasz Magiera
6591af960f
Merge pull request #4992 from filecoin-project/feat/splitstore
...
hot/cold blockstore segregation (aka. splitstore)
2021-03-08 22:01:06 +01:00
Łukasz Magiera
305c2ec77d
miner: Config to disable owner/worker addcess fallback
2021-03-08 21:33:46 +01:00
vyzo
64646de761
add configuration option for pubsub IPColocationWhitelist subnets
2021-03-06 20:05:32 +02:00
vyzo
6b8c60a659
don't ID wrap the hotstore
2021-03-05 14:46:18 +02:00
vyzo
dd0c308427
move Blockstore config to FullNode, rename to Chainstore and add default for HotStoreType
2021-03-05 14:46:18 +02:00
Raúl Kripalani
4b1e1f4b52
rename liveset => markset; rename snoop => tracking store; docs.
2021-03-05 14:46:18 +02:00
Raúl Kripalani
8cfba5b092
renames and polish.
2021-03-05 14:46:18 +02:00
Raúl Kripalani
b1b452bc0f
remove dependency from blockstore/splitstore => chain/store.
2021-03-05 14:46:18 +02:00
Raúl Kripalani
1a804fbdec
move splitstore into blockstore package.
2021-03-05 14:46:18 +02:00
Raúl Kripalani
1b51c10d78
split off lmdb support to a different branch.
2021-03-05 14:46:18 +02:00
vyzo
0fc2f3a26f
fix post-rebase compilation errors
2021-03-05 14:46:18 +02:00
vyzo
97abbe1eca
add (salted) bloom filter liveset
2021-03-05 14:46:17 +02:00
vyzo
73259aa350
add configuration for splitstore and default to a simple compaction algorithm
2021-03-05 14:46:17 +02:00
vyzo
b83994797a
separate LMDB options for hotstore and tracking stores
2021-03-05 14:46:17 +02:00
vyzo
68b6f913c7
propagate useLMDB option to splitstore through DI
2021-03-05 14:46:17 +02:00
vyzo
99c7d8e3eb
more informative names for the hotstore directories
2021-03-05 14:46:17 +02:00
vyzo
a586d42c3b
make hot store DI injectable in the split store, default to badger.
2021-03-05 14:46:17 +02:00
vyzo
622b4f7d9d
hook splitstore into DI
2021-03-05 14:46:17 +02:00
Raúl Kripalani
3795cc2bd2
segregate chain and state blockstores.
...
This paves the way for better object lifetime management.
Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
sizing caches differently.
- specifying different retention policies for chain and state data.
This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
store, and only retain state objects reachable from the last finality
in the state store.
2021-02-28 22:49:44 +00:00
Raúl Kripalani
7f0f7d0b36
Merge branch 'master' into refactor/lib/blockstore
2021-02-28 19:55:23 +00:00
Łukasz Magiera
0c6aef221d
Fix error logging format strings
2021-02-11 12:00:26 +01:00
Łukasz Magiera
8f603717a6
Merge remote-tracking branch 'origin/master' into steb/refactor-consistent-tipset-methods
2021-02-05 13:08:49 +01:00
Łukasz Magiera
9b65793509
Merge pull request #5404 from filecoin-project/feat/test-window-post
...
Retry proofs if we generate an incorrect one
2021-02-03 13:04:33 +01:00
Łukasz Magiera
1e7a107484
Merge pull request #5433 from filecoin-project/feat/keep-unsealed-cfg
...
Add miner config to always keep ensealed deal copies
2021-02-03 12:40:02 +01:00
Steven Allen
feb24c9684
choose the correct nonce
...
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
2021-01-30 10:18:51 -08:00
Łukasz Magiera
56a9d05491
Merge remote-tracking branch 'origin/master' into steb/refactor-consistent-tipset-methods
2021-01-30 12:05:21 +01:00
Raúl Kripalani
d1104fec4c
rename blockstores for consistency.
2021-01-29 23:17:25 +00:00
Raúl Kripalani
af429dba0f
Merge branch 'master' into chore/snake_context_through_blockstore_init
2021-01-29 21:21:46 +00:00
Raúl Kripalani
b0cbc932bd
consolidate all blockstores in blockstore package.
2021-01-29 20:01:00 +00:00
Steven Allen
8d3cc632ac
add support for running pre-migrations and caching their results
...
This can significantly speedup state migrations.
2021-01-27 12:55:53 -08:00
Łukasz Magiera
d4c7b63aac
Add miner config to always keep ensealed deal copies
2021-01-26 17:50:31 +01:00
Raúl Kripalani
a1da1dab85
add context to LockedRepo#Datastore().
2021-01-26 11:01:43 +00:00
Raúl Kripalani
ce3af308ed
Merge branch 'master' into chore/snake_context_through_blockstore_init
2021-01-25 19:31:41 +00:00
Steven Allen
8de4e5e005
retry proofs if we generate an incorrect one
2021-01-22 13:31:47 -08:00
Steven Allen
0131b077be
re-enable watchdog
2021-01-22 11:19:37 -08:00
Steven Allen
d412ccdb3f
disable the watchdog
...
It's causing the tests to flake.
2021-01-21 18:28:37 -08:00
Steven Allen
03cd3760bb
correctly pick the seal type based on the network version
...
Of course, we should really just run all of our tests post actors v2.
2021-01-21 15:21:20 -08:00
Aayush Rajasekaran
ab90a3b2bc
Actors update: MinerInfo.SealProofType has been removed
2021-01-21 15:21:19 -08:00
Raúl Kripalani
b9adf95010
watchdog: increase heapprof capture threshold to 90%.
2021-01-21 14:06:18 +00:00
Raúl Kripalani
931cfe1ed1
upgrade to raulk/go-watchdog@v1.0.1
...
This pulls in the improvements introduced in:
- https://github.com/raulk/go-watchdog/releases/tag/v1.0.0
- https://github.com/raulk/go-watchdog/releases/tag/v1.0.1
Lotus tries to initialize the watchdog in the following order of precedence:
1. If a max heap limit has been provided, initialize a heap-driven watchdog.
2. Else, try to initialize a cgroup-driven watchdog.
3. Else, try to initialize a system-driven watchdog.
4. Else, log a warning that the system is flying solo, and return.
This PR also enabled automatic heap profile capture when memory usage
surpasses 90% of the limit. Profiles are written to <LOTUS_HOME>/heapprof.
A single heap profile is captured per episode, with a max of 10 episodes
captured during the lifetime of the process. Episode = instance of usage
climbing above the 90% threshold.
2021-01-20 18:09:19 +00:00
Dirk McCormick
1206154718
feat: update to go-fil-markets v1.1.0 for better retry config
2021-01-20 10:00:02 +01:00
Dirk McCormick
835fd4b23c
feat: markets - miner should not dial client on restart
2021-01-14 16:38:56 +01:00
Peter Rabbitson
9334e73396
Snake a context through the Chain-blockstore creation
2021-01-04 15:27:52 +01:00
Łukasz Magiera
0cb2f51549
Merge pull request #5094 from filecoin-project/asr/verified-only
...
Allow miners to filter (un)verified deals
2020-12-10 19:48:22 +01:00
Steven Allen
e164dbb953
Compute the next nonce from the tipset, instead of actually executing the tipset
...
We could also do this in the message pool itself, but I'm not sure if it's worth it?
2020-12-09 11:29:41 -08:00
Łukasz Magiera
dc06d30f52
Merge pull request #5101 from filecoin-project/raulk/memory-watchdog
...
introduce memory watchdog; LOTUS_MAX_HEAP
2020-12-03 15:46:09 +01:00
Raúl Kripalani
f9cbf6d526
watchdog: add LOTUS_DISABLE_WATCHDOG escape hatch.
2020-12-03 13:16:10 +00:00
Raúl Kripalani
d033f8aab1
watchdog: remove silence period, reduce tick interval to 5 secs.
2020-12-02 22:31:57 +00:00
Raúl Kripalani
e34a759889
avoid global ResourceConstraints.
2020-12-02 22:26:30 +00:00
Łukasz Magiera
2fd93a55ac
plumb AddressSelector to stoage fsm
2020-12-02 21:47:54 +01:00
Łukasz Magiera
7246ef273f
miner: Control address config for (pre)commits
2020-12-02 19:58:00 +01:00
Raúl Kripalani
e8a5e0d2fe
integrate memory watchdog; impose limits on badger caches.
2020-12-02 16:40:28 +00:00
Aayush Rajasekaran
370817eb60
Allow miners to filter (un)verified deals
2020-12-02 01:32:34 -05:00
Łukasz Magiera
2e544e3e6a
configure SimultaneousTransfers in node/builder
2020-11-25 19:57:38 +01:00
hannahhoward
694834e8d5
feat(graphsync): configure simultaneous requests
...
allow configuration of the number of simultaneous requests graphsync will process at once
2020-11-24 14:32:30 -08:00
hannahhoward
afec58f780
feat(markets): add logging on data-transfer to miner
2020-11-23 12:43:30 -08:00
Dirk McCormick
4a8df0c3fa
feat: add data transfer logging
2020-11-23 10:31:39 +01:00
Łukasz Magiera
4483d94db3
Merge pull request #4902 from GFZRZK/GFZRZK/patch/remove_unuseful_code_in_newstorageask
...
just return storedask.NewStoredAsk to reduce unuseful code
2020-11-20 16:28:03 +01:00
Łukasz Magiera
94763c2aaa
Merge pull request #4849 from filecoin-project/feat/conngater
...
Connection Gater support
2020-11-20 16:25:02 +01:00
wangchao
49f8590578
just return storedask.NewStoredAsk to reduce unuseful code
2020-11-18 13:53:48 +08:00
Aayush Rajasekaran
68a16afeda
Merge branch 'master' into release/v1.2.0
2020-11-17 19:40:05 -05:00
Łukasz Magiera
50146fb9b9
Merge pull request #4872 from filecoin-project/raulk/fix-blockstore-import
...
fix badger double open on daemon --import-snapshot; chainstore lifecycle
2020-11-18 01:36:16 +01:00
Łukasz Magiera
3536ec9f85
Merge pull request #4879 from filecoin-project/fix/4833
...
Fix MaxSealingSectrosForDeals getting reset to 0
2020-11-17 23:41:22 +01:00
hannahhoward
a2abeef727
feat(markets): upgrade to fix cid recording issue
2020-11-16 17:57:57 -08:00
Łukasz Magiera
dd9682f2be
Fix MaxSealingSectrosForDeals getting reset to 0
2020-11-17 00:28:49 +01:00
Raúl Kripalani
0c6072a1a0
chainstore lifecycle: close via Close() instead of context.
2020-11-16 22:28:59 +00:00
Steven Allen
638900f97f
Remove static proof type configuration from market subsystem
...
Instead, use proof type from miner actor. This will, in turn, use the upgraded
proof type if/when it's switched at runtime.
TODO: Consider making this some form of config option instead?
2020-11-16 19:04:17 +01:00
Steven Allen
931979809f
get lotus building
2020-11-16 19:03:30 +01:00
Łukasz Magiera
6bea9dd178
Making sealing logic work with multiple seal proof types
2020-11-16 19:03:30 +01:00
Raúl Kripalani
d7b4f92f1f
add context to ChainStore.
2020-11-16 15:52:19 +00:00
Łukasz Magiera
49d3bb9497
Merge pull request #4851 from filecoin-project/fix/daemon-test-mode-restart
...
Truncate genesis file before generating
2020-11-16 16:13:33 +01:00
Jakub Sztandera
afe6c51016
Merge pull request #4813 from filecoin-project/chore/blockstore_nits
...
Chore/blockstore nits
2020-11-15 19:52:10 +01:00
Aayush Rajasekaran
1af1ceb1e0
Merge pull request #3646 from filecoin-project/feat/delayed-pubsub-subscribe
...
delayed pubsub subscribe for messages topic
2020-11-13 17:44:32 -05:00
vyzo
91e2530e11
remove dtypes dependency from api; move NetBlockList type to api/types.go
2020-11-13 21:11:17 +02:00
Dirk McCormick
f9fc601567
fix: truncate genesis file before generating
2020-11-13 17:30:12 +01:00
vyzo
5a0b9f4197
connectiong gater API scaffolding
2020-11-13 12:17:48 +02:00
vyzo
4b1d1b6540
connection gater for libp2p host
2020-11-13 11:44:29 +02:00
Dirk McCormick
26e12e6f6a
fix: dont crash on startup if funds migration fails
2020-11-12 17:15:05 +01:00
Peter Rabbitson
29817b6617
Remove all Chain GC wrappings/types
...
There is no practical way to do GC against the chainstore given the current
state of affairs: remove all GC-related types for the time being.
2020-11-12 06:49:53 +01:00
hannahhoward
9f7204ee26
feat(builder): finish new FundManager setup
...
Finish setup of new FundManager and provide a migration for previously reserved funds
2020-11-10 21:11:05 -08:00
Dirk McCormick
0d243bb824
refactor: integrate new FundManager
2020-11-10 21:11:05 -08:00
Raúl Kripalani
3577300aee
Merge branch 'master' into badger-viewable
2020-11-06 19:34:25 +00:00
vyzo
dab20efc99
add env var controlling msgs sync epochs
2020-11-05 18:09:04 +02:00
vyzo
0e40a04975
nicer check using Time objects
2020-11-05 18:08:52 +02:00
vyzo
c6cd699f0d
fix nearsync check
2020-11-05 18:08:35 +02:00
vyzo
32b7797394
only subscribe to messages topic if we are synced within 10 epochs
2020-11-05 18:08:13 +02:00
Łukasz Magiera
a1e1b03ca4
Optionally allow bitswap for chainstore
2020-11-03 23:44:44 +01:00
Raúl Kripalani
a16d7f221e
Merge branch 'master' into badger-viewable
2020-11-01 13:10:56 +00:00
Raúl Kripalani
d2e2322fd2
make the lotus node use the new native badger blockstore.
2020-11-01 13:01:26 +00:00
Jakub Sztandera
22742a9ba0
bootstrap: don't return early when one drand resolution fails
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-10-30 21:58:17 +01:00
Łukasz Magiera
3b16faddad
Merge pull request #4652 from filecoin-project/feat/default-max-fee-cfg
...
Config for default max gas fee
2020-10-30 21:35:10 +01:00
Łukasz Magiera
c3d00b0ac6
Merge pull request #4650 from jsign/jsign/offlinemode
...
make IPFS online mode configurable
2020-10-29 22:24:58 +01:00
Łukasz Magiera
ae7889f830
Config for default max gas fee
2020-10-29 20:50:11 +01:00
Ignacio Hagopian
7d0f279353
make IPFS online mode configurable
...
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
2020-10-29 15:16:45 -03:00
Łukasz Magiera
e1da874258
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-27 03:31:07 +01:00
Steven Allen
0d06f8fa2b
Add commands to change the worker key
2020-10-26 18:16:24 -07:00
Łukasz Magiera
660236b224
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-23 23:25:35 +02:00
Dirk McCormick
92942d44d1
feat: lite-mode - market storage and retrieval clients
2020-10-23 15:02:26 +02:00
Steven Allen
0c2ec04c09
use upgrade schedule from node when getting the network name
...
It doesn't matter now, but may in the future.
2020-10-21 12:18:38 -07:00
Steven Allen
4e730b5ec8
port to v2 imports
2020-10-21 12:16:23 -07:00
Ingar Shu
1974490ab0
Don't overwrite previously-configured maxPieceSize for a persisted ask
2020-10-19 13:12:28 -07:00
Łukasz Magiera
7ac5dc55d0
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-17 13:45:11 +02:00
Łukasz Magiera
111942b50b
Merge pull request #4424 from filecoin-project/feat/retrieval-cli-deal-filter
...
Custom filters for retrieval deals
2020-10-16 18:41:40 +02:00
Ingar Shu
0b7dc6971d
Rebasing
2020-10-15 09:33:01 -07:00
hannahhoward
b29e3b5242
feat(markets): use build.BlockDelaySecs for start buffer
...
instead of constant from spec-actors which uses hardcoded start epoch, use one that factors in
possibility of different block time
2020-10-14 21:28:36 -07:00
Steven Allen
ddade32bd3
write bitswap blocks into a temporary, in-memory block cache
...
If they end up validating, we'll write them back to the underlying blockstore.
Otherwise, there's no reason to keep them.
2020-10-14 14:43:58 -07:00
Aarsh Shah
30dd5d9644
use updated stored ask api
2020-10-14 11:39:57 +02:00
hannahhoward
58662b79b3
fix(deps): use tagged go-fil-markets 0.9.0
2020-10-13 19:50:13 -07:00
Łukasz Magiera
71b3b9075d
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-13 21:33:21 +02:00
hannahhoward
4edebcec2b
feat(markets): update markets 0.9.0 and add data transfer restart command
2020-10-13 03:41:08 -07:00
Ingar Shu
49155ace62
Use seal-duration in calculating the earliest StartEpoch that we will consider ( #4337 )
2020-10-12 17:45:37 -03:00
Jakub Sztandera
2b21fdef33
Refactor from crypto.SigType to types.KeyType
...
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2020-10-12 00:07:53 +02:00
Steven Allen
c463582528
fix a race and optimize hello messages
...
LifecycleCtx can _only_ be called during startup as it appends an fx hook.
Worse, this was causing us to append an fx hook on every single hello message,
leaking memory (and probably causing other shutdown issues...).
2020-10-10 08:33:06 -07:00
Steven Allen
748d2e82a7
unshare the journal
...
Motivation:
* Run lotus with the race detector enabled (primary motivation).
* Allow multiple lotus nodes in a process (not a high priority).
Previously, the journal was shared between all lotus instances, but it was
initialized for every new node. This caused safety problems in tests (at a
minimum).
This patch explicitly passes the journal to all services that need it.
2020-10-09 13:23:07 -07:00
Dirk McCormick
ef73b964fb
feat: add end-to-end test for lite mode
2020-10-09 11:48:35 +02:00
Dirk McCormick
2719adc1b1
feat: lite-mode - thin client for chan & state
2020-10-09 11:43:03 +02:00
Dirk McCormick
8fa4c0a970
feat: gateway - MpoolPush
2020-10-09 11:43:03 +02:00
Dirk McCormick
00a14de094
feat: gateway - StateLookupID
2020-10-09 11:43:03 +02:00
Dirk McCormick
e19cd9ed01
feat: lotus-lite - replace wallet StateManager with thin client to gateway
2020-10-09 11:43:03 +02:00
vyzo
7933cf094f
allow all drand topics
2020-10-08 21:48:55 +03:00
vyzo
4cd73f1560
use subscription filter in pubsub
2020-10-08 21:40:36 +03:00
Łukasz Magiera
0de3051821
Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers
2020-10-08 13:10:41 +02:00
vyzo
de390749fc
set pubsub IPColocationFactor to 5
2020-10-06 09:11:44 +03:00
Łukasz Magiera
0e2f697217
Merge pull request #4133 from filecoin-project/feat/backup
...
Miner backup/restore commands
2020-10-06 02:05:03 +02:00
Ingar Shu
1227bf2893
Change max time in future for StartEpoch to 7 days
2020-10-05 12:35:58 -07:00