cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
20,235 Commits 24 Branches 342 Tags 314 MiB
05cd57a7a9
Commit Graph

444 Commits

Author SHA1 Message Date
vyzo
622b4f7d9d hook splitstore into DI 2021-03-05 14:46:17 +02:00
Dirk McCormick
ced455bba9 feat: add configurable max provider collateral 2021-03-02 10:24:57 +01:00
Raúl Kripalani
3795cc2bd2 segregate chain and state blockstores. 
This paves the way for better object lifetime management.

Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
  sizing caches differently.
- specifying different retention policies for chain and state data.

This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
  store, and only retain state objects reachable from the last finality
  in the state store.
 2021-02-28 22:49:44 +00:00
Łukasz Magiera
adeb4cefee node builder: Cleanup and add some docstrings 2021-02-09 13:52:03 +01:00
Łukasz Magiera
332ea8a126 Merge remote-tracking branch 'origin/master' into feat/deal-batch-publish 2021-02-02 18:21:14 +01:00
Dirk McCormick
987f41011a refactor: move publish msg config inside deals config 2021-02-01 10:23:05 +01:00
Steven Allen
8d3cc632ac add support for running pre-migrations and caching their results 
This can significantly speedup state migrations.
 2021-01-27 12:55:53 -08:00
Dirk McCormick
adac340f3f feat: batch publish deal messages 2021-01-25 14:25:57 +01:00
Łukasz Magiera
0cb2f51549
Merge pull request #5094 from filecoin-project/asr/verified-only 
Allow miners to filter (un)verified deals
 2020-12-10 19:48:22 +01:00
Łukasz Magiera
dc06d30f52
Merge pull request #5101 from filecoin-project/raulk/memory-watchdog 
introduce memory watchdog; LOTUS_MAX_HEAP
 2020-12-03 15:46:09 +01:00
Raúl Kripalani
e34a759889 avoid global ResourceConstraints. 2020-12-02 22:26:30 +00:00
Łukasz Magiera
7246ef273f miner: Control address config for (pre)commits 2020-12-02 19:58:00 +01:00
Raúl Kripalani
e8a5e0d2fe integrate memory watchdog; impose limits on badger caches. 2020-12-02 16:40:28 +00:00
Aayush Rajasekaran
370817eb60 Allow miners to filter (un)verified deals 2020-12-02 01:32:34 -05:00
Łukasz Magiera
2e544e3e6a configure SimultaneousTransfers in node/builder 2020-11-25 19:57:38 +01:00
Łukasz Magiera
94763c2aaa
Merge pull request #4849 from filecoin-project/feat/conngater 
Connection Gater support
 2020-11-20 16:25:02 +01:00
Steven Allen
931979809f get lotus building 2020-11-16 19:03:30 +01:00
vyzo
4b1d1b6540 connection gater for libp2p host 2020-11-13 11:44:29 +02:00
Peter Rabbitson
29817b6617 Remove all Chain GC wrappings/types 
There is no practical way to do GC against the chainstore given the current
state of affairs: remove all GC-related types for the time being.
 2020-11-12 06:49:53 +01:00
hannahhoward
9f7204ee26 feat(builder): finish new FundManager setup 
Finish setup of new FundManager and provide a migration for previously reserved funds
 2020-11-10 21:11:05 -08:00
Dirk McCormick
0d243bb824 refactor: integrate new FundManager 2020-11-10 21:11:05 -08:00
Łukasz Magiera
a1e1b03ca4 Optionally allow bitswap for chainstore 2020-11-03 23:44:44 +01:00
Raúl Kripalani
656ece06e5 fix metrics wiring. 
Some components like go-ds-measure, go-ipfs-blockstore and go-bitswap
expose metrics via ipfs/go-metrics-interface, but Lotus never injects
the Prometheus exporter (ipfs/go-metrics-prometheus). Therefore, those
metrics never surface in instrumentation.

Instead, Lotus uses OpenCensus directly.

This commit injects the Prometheus exporter for go-metrics-interface, and
instructs the OpenCensus Prometheus exporter to use the DefaultRegistry.

This has the effect of exposing blending the metrics of both metrics
libraries.

With this patch, the datastore, cache utilisation, and bitswap metrics
are now exported via the /debug/metrics endpoint.

This commit also fixes an issue where the metrics scope was empty, making
go-metrics-interface default to "<no-scope>". Angle brackets are inadmissible
characters for Prometheus, so it was refusing to export the affected metrics.
(These were the ARC cache metrics.)
 2020-11-02 17:56:49 +00:00
Łukasz Magiera
3b16faddad
Merge pull request #4652 from filecoin-project/feat/default-max-fee-cfg 
Config for default max gas fee
 2020-10-30 21:35:10 +01:00
Łukasz Magiera
c3d00b0ac6
Merge pull request #4650 from jsign/jsign/offlinemode 
make IPFS online mode configurable
 2020-10-29 22:24:58 +01:00
Łukasz Magiera
ae7889f830 Config for default max gas fee 2020-10-29 20:50:11 +01:00
Ignacio Hagopian
7d0f279353
make IPFS online mode configurable 
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
 2020-10-29 15:16:45 -03:00
Łukasz Magiera
660236b224 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-23 23:25:35 +02:00
Dirk McCormick
92942d44d1 feat: lite-mode - market storage and retrieval clients 2020-10-23 15:02:26 +02:00
Łukasz Magiera
7ac5dc55d0 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-17 13:45:11 +02:00
Ingar Shu
d0e4150cea
Use separate config for Retrieval Filter 2020-10-16 08:30:55 -07:00
Ingar Shu
0b7dc6971d
Rebasing 2020-10-15 09:33:01 -07:00
Łukasz Magiera
71b3b9075d Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-13 21:33:21 +02:00
Łukasz Magiera
6a232e7214
Basic multi-wallet support 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-10-12 00:07:41 +02:00
Łukasz Magiera
a6d9b302fa
Merge pull request #3583 from filecoin-project/feat/signing-backends 
Remote wallet backends
 2020-10-10 02:58:02 +02:00
Steven Allen
748d2e82a7 unshare the journal 
Motivation:

* Run lotus with the race detector enabled (primary motivation).
* Allow multiple lotus nodes in a process (not a high priority).

Previously, the journal was shared between all lotus instances, but it was
initialized for every new node. This caused safety problems in tests (at a
minimum).

This patch explicitly passes the journal to all services that need it.
 2020-10-09 13:23:07 -07:00
Łukasz Magiera
fdaa9c14ad Merge remote-tracking branch 'origin/master' into feat/signing-backends 2020-10-09 20:36:51 +02:00
Łukasz Magiera
ab8286fa38 Fix docsgen, lotus-soup build 2020-10-09 18:43:22 +02:00
Dirk McCormick
d69e4c7cf2 refactor: lite-mode - simplify organization of dep injection 2020-10-09 11:54:42 +02:00
Dirk McCormick
ef73b964fb feat: add end-to-end test for lite mode 2020-10-09 11:48:35 +02:00
Dirk McCormick
b32d25c562 feat: add RPC for GasEstimateMessageGas 2020-10-09 11:45:16 +02:00
Dirk McCormick
be09a8a00a feat: lite-mode - check that gateway API implements required chain, mpool, state methods 2020-10-09 11:43:03 +02:00
Dirk McCormick
2719adc1b1 feat: lite-mode - thin client for chan & state 2020-10-09 11:43:03 +02:00
Dirk McCormick
f1b1d8cec6 fix: MpoolAPI DI 2020-10-09 11:43:03 +02:00
Dirk McCormick
8fa4c0a970 feat: gateway - MpoolPush 2020-10-09 11:43:03 +02:00
Dirk McCormick
e19cd9ed01 feat: lotus-lite - replace wallet StateManager with thin client to gateway 2020-10-09 11:43:03 +02:00
Łukasz Magiera
8783c7434e Merge remote-tracking branch 'origin/master' into feat/signing-backends 2020-10-09 00:50:41 +02:00
Łukasz Magiera
0de3051821 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-08 13:10:41 +02:00
Łukasz Magiera
921d78f820 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-04 10:39:55 +02:00
Steven Allen
8292d60196 Merge branch 'master' into asr/spec-v1 2020-10-02 18:03:03 -07:00
Steven Allen
c5de617af6 decouple network version and upgrades 
Not all upgrades require network version bumps.
 2020-10-02 17:10:57 -07:00
Łukasz Magiera
8fe8a5df45 Cap market provider messages 2020-10-02 18:35:52 +02:00
Steven Allen
e865ba0c64 improve upgrade schedule configurable 
Unfortunately, it looks like functional ops cannot be passed in via lotus's
di system.
 2020-10-01 15:18:59 -07:00
Łukasz Magiera
5932f28519 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-01 02:39:48 +02:00
Łukasz Magiera
feecee310d
Merge pull request #4089 from filecoin-project/feat/update-markets-0.7.0 
Markets 0.7.0 with updated data stores
 2020-09-30 21:49:14 +02:00
Łukasz Magiera
2cfe22d4e5 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-30 20:48:16 +02:00
hannahhoward
be884e27be feat(markets): update markets 0.7.0 2020-09-30 10:26:50 -07:00
Dirk McCormick
d1c10a61dd fix: message signer - always compare with mpool nonce 2020-09-29 12:20:09 +02:00
Dirk McCormick
3c524ac0e0 refactor: move nonce generation from mpool to wallet 2020-09-24 14:09:42 +02:00
Łukasz Magiera
aa5bd7bc17 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-21 22:36:16 +02:00
zgfzgf
fa9be9a627 Replace fx.Extract with fx.Populate 2020-09-16 09:14:56 +08:00
Łukasz Magiera
03cf6cca40 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-15 17:47:03 +02:00
Łukasz Magiera
692027f625
Merge pull request #3819 from zgfzgf/zgfzgf 
optimize Settings.nodeType
 2020-09-15 10:42:23 +02:00
Łukasz Magiera
6665a9ca69
Merge pull request #3845 from filecoin-project/sync-manager 
syncer: make SyncManager an interface.
 2020-09-15 10:28:48 +02:00
Raúl Kripalani
dd841f32db syncer: make SyncManager an interface. 2020-09-14 21:58:59 +01:00
Łukasz Magiera
381a6cdfac Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-14 19:11:50 +02:00
Raúl Kripalani
05aa5f2d38 allow customizing disabled journal events + tests. 2020-09-14 13:01:03 +01:00
Raúl Kripalani
6d29d75724 Merge branch 'master' into inmem-journal 2020-09-14 12:17:45 +01:00
zgfzgf
45a5eaf913 optimize Settings.nodeType 2020-09-14 11:00:08 +08:00
Jakub Sztandera
64fa6fd9e5 Draw the rest of the owl 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-09-10 17:41:55 -04:00
Łukasz Magiera
5f08fe7ead Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-09-10 17:30:54 +02:00
Raúl Kripalani
878ffafb51 rename other Chain{Exchange=>Bitswap}; fix ChainBlock{s=>S}ervice(). 2020-09-07 20:20:23 +01:00
Raúl Kripalani
453e826a0f rename p2p protocol to /fil/chain/xchg/0.0.1 (backwards-compatible); rename more. 2020-09-07 19:45:34 +01:00
Raúl Kripalani
55b1456d45 blocksync: introduce interfaces; rename to chainexchange. 2020-09-07 19:31:43 +01:00
Łukasz Magiera
9e6f974f3c storage: Fix build 2020-09-07 16:12:55 +02:00
Łukasz Magiera
780f6dba34 Remote wallet backends 2020-09-05 21:39:09 +02:00
Łukasz Magiera
f90cfda2e6 wallet: Add interface layer 2020-09-04 22:18:03 +02:00
Raúl Kripalani
1ec534d607 Merge branch 'master' into inmem-journal 2020-09-04 10:55:40 +01:00
whyrusleeping
a59dadccc9 add apis and command to inspect bandwidth usage 2020-09-02 15:25:53 -07:00
Raúl Kripalani
0b6a182a44 move InitJournalKey to the top; add godcs on invoke order. 2020-09-02 19:15:07 +01:00
Raúl Kripalani
efdfd3ee3e Merge branch 'master' into inmem-journal 2020-08-26 16:38:23 +01:00
Raúl Kripalani
4e1ef09751 make journal a global var. 2020-08-26 16:09:37 +01:00
Steven Allen
5733c71c50 Lint everything 
We were ignoring quite a few error cases, and had one case where we weren't
actually updating state where we wanted to. Unfortunately, if the linter doesn't
pass, nobody has any reason to actually check lint failures in CI.

There are three remaining XXXs marked in the code for lint.
 2020-08-20 20:46:36 -07:00
vyzo
92faf5c5f9 don't prefer TLS over NOISE 2020-08-20 11:33:19 +03:00
Łukasz Magiera
65ffde9e4b fsm: Config for max WaitDeals sectors 2020-08-18 16:20:31 +02:00
Raúl Kripalani
862bafc63a fix lint errors. 2020-08-17 14:39:33 +01:00
Raúl Kripalani
efdc428d5d keep storage-fsm (renamed to storage-sealing) and sector-storage in extern. 2020-08-17 14:26:18 +01:00
Raúl Kripalani
3c17cd655e integrate extern/sector-storage into lotus proper. 2020-08-16 11:09:58 +01:00
Raúl Kripalani
fb0ccc9260 integrate extern/storage-fsm into lotus proper. 2020-08-16 10:42:13 +01:00
hannahhoward
6b5525b8d2 feat(markets): update markets 0.5.6 
update markets to 0.5.6, removing use of validators that moved inside markets
 2020-08-14 11:14:03 -07:00
Łukasz Magiera
6f890a3edd Miner fee config 2020-08-12 19:47:09 +02:00
Jakub Sztandera
a45febc065
Fix MpoolLocker 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-08-11 19:32:31 +02:00
Raúl Kripalani
2ea5abdfb5 wire journal into miner. 2020-08-11 13:48:32 +01:00
Raúl Kripalani
92598237c3 Merge branch 'next' into inmem-journal 2020-08-11 12:41:11 +01:00
Łukasz Magiera
e2d5451ffb
Merge pull request #2715 from filecoin-project/schomatis/blocksync/review 
Rewrite internal logic of block sync client
 2020-08-06 19:46:58 +02:00
Dirk McCormick
fdfccf0466 WIP: fix payment channel locking 2020-08-06 12:04:10 -04:00
Łukasz Magiera
c6acfa624e
Merge pull request #2834 from filecoin-project/feat/fund-manager-fix 
Improve Fund Manager
 2020-08-06 16:45:05 +02:00
Łukasz Magiera
73b8868046 gofmt 2020-08-06 03:32:59 +02:00
Łukasz Magiera
f8b8ecc0c3 Consensus filter 2020-08-06 03:16:39 +02:00
hannahhoward
b4d1b628be feat(fundmgr): add monitoring to FundMgr 2020-08-04 17:29:52 -07:00
Lucas Molas
76d40ec408 first pass, ready for review 2020-08-03 12:20:04 -03:00
hannahhoward
3cac1080cc feat(retrievalstoremgr): add retrievalstoremgr 
add manager for retrievals to handle different cases for IPFS/non-ipfs
 2020-07-31 14:16:18 -07:00
Łukasz Magiera
fc3c91b738 Support external deal decision logic 2020-07-31 20:29:16 +02:00
Łukasz Magiera
4d328cad54 Get things to mostly work 2020-07-30 14:31:50 +02:00
Łukasz Magiera
09b90773d8 Merge remote-tracking branch 'origin/next' into feat/cid-builder 2020-07-28 16:06:06 +02:00
hannahhoward
5a623cccb5 feat(markets): update markets mulitple deal stores 2020-07-27 23:13:28 -07:00
Steven Allen
b7a4dbb07f Support inline CIDs 
And use the new CidBuilder from the spec actors.

This patch does not switch over to inline CIDs by default, but paves the way.
 2020-07-23 23:12:32 -07:00
Łukasz Magiera
d70edbcb7c Set things on syscallShim 2020-07-22 13:35:10 -04:00
hannahhoward
98535d5e06 feat(settler): monitor settled channels 
Add a routine to monitor the chain for payment channels settling, and if it's a channel that we are
the recipient of payments on, submit any outstanding vouchers to the
chain
 2020-07-22 13:34:35 -04:00
Raúl Kripalani
cb8e209f78 journal: disable noisy message pool events. 2020-07-21 13:13:09 +01:00
Raúl Kripalani
7459ec6bba add mpool journal events; fix dependency injection. 2020-07-20 16:37:32 +01:00
Raúl Kripalani
226786c1da wip 2020-07-20 14:45:17 +01:00
Raúl Kripalani
695f6cfe45 wip in-memory journal. 2020-07-17 14:34:42 +01:00
Aayush Rajasekaran
a5ef629cef Add expected seal duration to custom deal logic, reject deals that start too early 2020-07-15 17:31:50 -04:00
Frank
14ec9a2068 update storage miner and seal worker document 2020-07-11 16:55:13 +08:00
Łukasz Magiera
66237415c5 Merge remote-tracking branch 'origin/next' into feat/client-multi-bstore 2020-07-08 22:48:47 +02:00
Łukasz Magiera
b049c5a7d1 Merge remote-tracking branch 'origin/master' into next 2020-07-08 14:35:40 +02:00
Aayush Rajasekaran
411c82ae8f Update storage-FSM, add API to set sector seal delay 2020-07-07 15:23:23 -04:00
Łukasz Magiera
47f0898ce9 Wire up client import manager 2020-07-07 10:52:19 +02:00
Łukasz Magiera
8942967223 Client Import manager 2020-07-07 10:52:04 +02:00
whyrusleeping
42bd4eccbe implement a persistent journal for lotus node operations 2020-07-06 09:42:00 -07:00
laser
b9180a99d4 toggle consideration of offline storage/retrieval deal proposals 2020-06-26 12:27:41 -07:00
laser
e910a045d0 big rename to allow for later introducing the "consider offline" config 2020-06-26 10:50:54 -07:00
laser
ef0abf2b98 disable/enable retrieval deal consideration via config 2020-06-24 08:07:12 -07:00
Yusef Napora
b448de422e improve DrandConfig dependency injection 2020-06-23 16:01:10 -04:00
laser
de7d6c255c blacklist -> blocklist 2020-06-18 15:51:01 -07:00
laser
0c8d648998 specify which CID is being blacklisted (it's the piece) 2020-06-18 14:02:22 -07:00
laser
b0edf924b4 add commands for manipulating storage deal CID blacklist 2020-06-18 13:15:18 -07:00
hannahhoward
9262661135 chore(deps): update to fil-markets 0.3.0 
Updates to latest fil markets with resumability and other fixes
 2020-06-15 15:43:47 -07:00
Łukasz Magiera
01f0bb3124
Merge pull request #1994 from laser/feat/1920-toggle-accepting-storage-deals 
allow miner to disable/enable consideration of received storage deal proposals
 2020-06-15 19:05:39 +02:00
laser
36b327b57b various symbol renames 2020-06-11 13:18:18 -07:00
laser
7587e6c08b get and set storage deal acceptance through CLI 2020-06-11 12:59:50 -07:00
laser
d6b2519843 config-driven IsAcceptingStorageDeals flag 
Makes incremental progress towards #1920.
 2020-06-11 11:29:59 -07:00
Jim Pick
6e11ff3796 Make libp2p swarm Announce / NoAnnounce addresses configurable 
In go-ipfs, it is possible to manually configure addresses to announce
or not announce to the swarm.

https://github.com/ipfs/go-ipfs/blob/master/docs/config.md#addressesannounce

This PR adds the same feature to Lotus. It can be configured from the [libp2p]
section of either the .lotus or .lotusstorage config.toml file. Example:

[Libp2p]
  ListenAddresses = ["/ip4/0.0.0.0/tcp/34515", "/ip6/::/tcp/34515"]
  AnnounceAddresses = ["/ip4/52.13.91.110/tcp/34515"]
 2020-06-08 17:03:11 -07:00
Jakub Sztandera
7762cd0a68
Assign positive scores to drand bootstrappers 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-06-08 11:34:46 +02:00
Jakub Sztandera
7bee5b3abc
Merge remote-tracking branch 'origin/master' into next 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-06-05 00:33:43 +02:00
Jakub Sztandera
186fd4da74
Add lotus net scores for pubsub score visibility 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-06-03 03:13:49 +02:00
laser
bea9cd2fe2 rename to better convey intent + document 2020-06-02 12:35:11 -07:00
laser
38d714a28a first take at "daemon stop" command 
Fixes #1827

TODO: plumb the daemon-stopping into lotus-storage-miner, too
 2020-06-02 12:29:58 -07:00
Jakub Sztandera
71eb41b9aa
Add pubsub support to drand beacon 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2020-05-29 17:55:17 +02:00
Ignacio Hagopian
0af7cd1448 allow using ipfs for retrieval 
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
 2020-05-28 22:11:49 +02:00
Ignacio Hagopian
de3edc2ec7 remote ipfs support & automatic env 
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
 2020-05-28 22:11:49 +02:00
Ignacio Hagopian
61daaa774b
allow using ipfs for retrieval 
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
 2020-05-26 14:52:52 -03:00
Ignacio Hagopian
d5346f8326
remote ipfs support & automatic env 
Signed-off-by: Ignacio Hagopian <jsign.uy@gmail.com>
 2020-05-26 14:52:51 -03:00
vyzo
314da26da3 unconditionally disable relay 2020-05-25 13:39:50 +03:00
hannahhoward
3264a9faad feat(deps): update to latest go-fil-markets 
update to go-fil-markets v0.2.7 with latest changes, modify constructors as needed to accomodate new
DS Prefix customization
 2020-05-20 16:01:03 -07:00
shannonwells
e68ab286f6 upgrade lotus to go-fil-markets v0.2.5 2020-05-20 15:56:58 -07:00
Jeromy
d3eb808ff4 add profile for bootstrappers 2020-05-14 09:27:07 -07:00
vyzo
112aca7896 pubsub-specific configuration 2020-05-04 18:30:54 +03:00
Łukasz Magiera
5ec76d4b7b support making deals with data directly from IPFS nodes 2020-04-30 01:56:45 +02:00