cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity
18,122 Commits 24 Branches 342 Tags 314 MiB
c6bd9bc9e0
Commit Graph

2937 Commits

Author SHA1 Message Date
Łukasz Magiera
330cfc33ee worker: Typo in resources cmd usage 
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
 2021-11-30 02:06:58 +01:00
Łukasz Magiera
cf20b0b2b8 worker: Command to print resource-table env vars 2021-11-30 02:06:58 +01:00
Clint Armstrong
93e4656a27 Use a float to represent GPU utilization 
Before this change workers can only be allocated one GPU task,
regardless of how much of the GPU resources that task uses, or how many
GPUs are in the system.

This makes GPUUtilization a float which can represent that a task needs
a portion, or multiple GPUs. GPUs are accounted for like RAM and CPUs so
that workers with more GPUs can be allocated more tasks.

A known issue is that PC2 cannot use multiple GPUs. And even if the
worker has multiple GPUs and is allocated multiple PC2 tasks, those
tasks will only run on the first GPU.

This could result in unexpected behavior when a worker with multiple
GPUs is assigned multiple PC2 tasks. But this should not suprise any
existing users who upgrade, as any existing users who run workers with
multiple GPUs should already know this and be running a worker per GPU
for PC2. But now those users have the freedom to customize the GPU
utilization of PC2 to be less than one and effectively run multiple PC2
processes in a single worker.

C2 is capable of utilizing multiple GPUs, and now workers can be
customized for C2 accordingly.
 2021-11-30 02:06:58 +01:00
Clint Armstrong
c4f46171ae Report memory used and swap used in worker res 
Attempting to report "memory used by other processes" in the MemReserved
field fails to take into account the fact that the system's memory used
includes memory used by ongoing tasks.

To properly account for this, worker should report the memory and swap
used, then the scheduler that is aware of the memory requirements for a
task can determine if there is sufficient memory available for a task.
 2021-11-30 02:06:58 +01:00
Jiaying Wang
4d2f3375e8
Merge pull request #7699 from filecoin-project/feat/listcids-verbose 
Add verbose mode to lotus-miner pieces list-cids
 2021-11-29 17:43:09 -05:00
Łukasz Magiera
f8b132890c Add verbose mode to lotus-miner pieces list-cids 2021-11-29 17:19:47 +01:00
Jakub Sztandera
4d8be81a8f
Add usage 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-11-26 15:01:05 +01:00
Jakub Sztandera
e3c7b8d006
Fix typo 
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
 2021-11-26 14:42:18 +01:00
Jakub Sztandera
af113f867d
Add dump code 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-11-25 20:53:49 +01:00
Łukasz Magiera
b522901845 update storage attach allowTo usage 2021-11-23 16:23:06 +01:00
Łukasz Magiera
e362d47903 Add optional group flags to storage attach commands 2021-11-23 16:11:04 +01:00
Łukasz Magiera
b1781c33ce Show group info in storage list 2021-11-23 16:11:04 +01:00
Łukasz Magiera
51753a1e25
Merge pull request #7414 from filecoin-project/feat/shed-wallet-balancer 
shed: simple wallet balancer util
 2021-11-22 18:11:42 +01:00
Łukasz Magiera
0262f1cd6e shed: simple wallet balancer util 2021-11-22 14:29:23 +01:00
Łukasz Magiera
81a2f2f06d
Merge pull request #7329 from filecoin-project/feat/lotus-stats-update 
Add caches to lotus-stats and splitcode
 2021-11-22 12:44:00 +01:00
Łukasz Magiera
9adaa9b967 lotus-shed msg: Decode submessages/msig proposals 2021-11-17 23:10:14 +01:00
Aayush Rajasekaran
1449644c34 Integrate v7 actors 2021-11-17 15:55:14 -05:00
Anton Evangelatov
5454aebf13
CLI tools for the DAGStore inverted index (#7361) 
* cli commands for dagstore

* address comments from Dirk

* rename inverted index to piece index
 2021-11-17 15:39:45 +02:00
Dirk McCormick
599a6bfb24 fix: err message 2021-11-17 12:37:15 +01:00
Dirk McCormick
501308239f feat: CLI command to announce deal to indexers 2021-11-17 12:16:22 +01:00
Aayush Rajasekaran
495f08473d Shed: Add a util to list miner faults 2021-11-07 23:33:37 +00:00
Aayush Rajasekaran
3be0c068b9 Shed: Add a util to create miners more easily 2021-11-03 00:20:48 +00:00
Travis Person
2d4f5958e2 Add caches to lotus-stats and splitcode 2021-11-01 09:05:14 +00:00
Jennifer Wang
dc6bd2b015 use existing util 2021-10-26 20:53:16 -04:00
Jennifer Wang
18834de7d0 parse attofil to fil 2021-10-26 19:37:29 -04:00
Aayush Rajasekaran
e1deed3b1a
Merge pull request #7544 from filecoin-project/asr/shed 
Shed: Add a util to find miner based on peerid
 2021-10-20 17:55:16 -07:00
Łukasz Magiera
f7884c4f45
Merge pull request #7527 from filecoin-project/feat/show-prepared-waitres 
Show prepared tasks in sealing jobs
 2021-10-20 15:22:35 +01:00
Aayush Rajasekaran
9ba7f372cd Shed: Add a util to find miner based on peerid 2021-10-19 18:46:09 -07:00
Łukasz Magiera
ba171954b4
Merge pull request #7479 from filecoin-project/feat/prov-check-by-storage 
Add storage-id flag to proving check
 2021-10-19 10:06:30 +01:00
Łukasz Magiera
261238e157 Show prepared tasks in sealing jobs 2021-10-18 18:44:56 +02:00
Jennifer Wang
d1e15e61d0 Merge branch 'release/v1.12.0' into jen/12-master 2021-10-11 14:37:32 -04:00
Jennifer Wang
16743a25c2 print confidence in decimal 2021-10-11 12:34:51 -04:00
dirkmc
f9433811c7
Merge pull request #7454 from filecoin-project/nonsense/retry-in-publish-deal 
itests: retry deal when control addr is out of funds
 2021-10-11 15:14:27 +02:00
Jakub Sztandera
61e1297cc7 gofmt 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-10-11 08:24:55 -04:00
Jakub Sztandera
e40c7b7a86 Fix withdrawn amount equiality check 
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
 2021-10-11 08:24:33 -04:00
Jakub Sztandera
177033d0f2 Fix withdrawn amount equiality check 
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
 2021-10-11 08:24:24 -04:00
Aayush Rajasekaran
cab724841a Update cmd/lotus-miner/actor.go 
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
 2021-10-11 08:24:00 -04:00
Aayush Rajasekaran
2241f02621 Bugfix: actor withdraw CLI should depend on network version 2021-10-11 08:23:53 -04:00
Jakub Sztandera
5abba9ca9f
gofmt 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-10-11 14:10:06 +02:00
Anton Evangelatov
1acb6b4328 update CLI command to take arg; reduce time for itest 2021-10-11 14:03:56 +02:00
Anton Evangelatov
6e5ccc87cf cli: add retry for deals stuck in Publish with no funds 2021-10-11 14:03:55 +02:00
Jakub Sztandera
8533ae0032
Fix withdrawn amount equiality check 
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
 2021-10-11 14:03:28 +02:00
Jakub Sztandera
90ff1d3ccf
Fix withdrawn amount equiality check 
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
 2021-10-11 14:03:21 +02:00
Aayush Rajasekaran
be774f55ce
Update cmd/lotus-miner/actor.go 
Co-authored-by: Jiaying Wang <42981373+jennijuju@users.noreply.github.com>
 2021-10-10 10:40:22 -04:00
Aayush Rajasekaran
e158b1d1e4 Bugfix: actor withdraw CLI should depend on network version 2021-10-09 20:17:03 -04:00
Łukasz Magiera
bfcfb199a0 Add storage-id flag to proving check 2021-10-08 14:56:08 +02:00
Peter Rabbitson
8b01725129 Dep upgrade pass 
Touching just the seemingly non-controversial ones
 2021-10-08 12:37:40 +02:00
Aayush Rajasekaran
89db4235ce Shed: Add a util to find the most recent null tipset 2021-10-06 19:37:35 -04:00
Jennifer Wang
38e01cba83 Merge branch 'release/v1.12.0' into jen/12tomaster 2021-10-04 02:38:00 -04:00
Aayush Rajasekaran
a335cb3767 Update to latest actors 2021-10-01 17:11:05 -04:00
Łukasz Magiera
a8a9818043 Expose storage states on the metrics endpoint 2021-10-01 14:45:01 +02:00
Łukasz Magiera
95e8b59367
Merge pull request #7341 from filecoin-project/mg/feat/panic-reporter 
feat: Catch panic to generate report and reraise
 2021-10-01 10:50:19 +01:00
Mike Greenberg
2f8a2fc907
fix: Update lotusminer default paths 
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
 2021-09-30 09:06:55 -04:00
Aayush Rajasekaran
2bafdf7271 Modify tooling to support new WithdrawBalance return 2021-09-29 13:01:04 -04:00
Aayush Rajasekaran
c3c46e9097 Fix Drand fetching around null tipsets 2021-09-29 11:57:59 -04:00
Łukasz Magiera
83c8e981b4 lotus shed: fr32 utils 2021-09-27 18:31:36 +02:00
Łukasz Magiera
53a9dd5da4 Add more info to miner allinfo command 2021-09-24 12:02:06 +02:00
Łukasz Magiera
0e7e665e6d
Merge pull request #7312 from filecoin-project/nonsense/crossref-datatransfer-storagedeal 
add `lotus-miner storage-deals list --format=json` with transfers
 2021-09-24 10:57:59 +01:00
Aarsh Shah
a80bc34d59 fix build 2021-09-21 15:23:01 +04:00
Mike Greenberg
3508a5b664 chore: Use After hooks and proper args 2021-09-17 18:01:54 -04:00
Aayush Rajasekaran
888b2040de Miner CLI: Allow trying to change owners of any miner actor 2021-09-14 11:50:30 +02:00
Anton Evangelatov
3e10f2e250 make gen 2021-09-10 15:18:32 +02:00
Anton Evangelatov
67dc7523d5 fixup 2021-09-10 15:09:37 +02:00
Anton Evangelatov
caaff9b7af remove flag 2021-09-10 15:06:51 +02:00
Anton Evangelatov
fe602ab3bb refactor 2021-09-10 15:05:51 +02:00
Łukasz Magiera
b4a3eb55e1 Show more deal states in miner info 2021-09-10 12:21:33 +02:00
Anton Evangelatov
027c1b5ab8 add lotus-miner storage-deals lwt 2021-09-10 12:21:02 +02:00
Łukasz Magiera
8612d1e824 Add --unproven flag to the sectors list command 2021-09-09 23:29:52 +02:00
Aayush Rajasekaran
16c60914eb Seed: improve helptext 2021-09-08 15:26:37 -04:00
Łukasz Magiera
dfc039276d address review 2021-09-07 19:42:52 +02:00
Łukasz Magiera
186c4990dd Reduce nesting in sectors list command 2021-09-07 18:49:53 +02:00
Łukasz Magiera
e4044151f0 Show deal sizes is sealing sectors 2021-09-07 18:49:53 +02:00
Łukasz Magiera
3c2d854a9e
Merge pull request #7246 from filecoin-project/feat/sectorinfo-partition 
Add partition info to the 'sectors status' command
 2021-09-02 20:22:11 +02:00
Łukasz Magiera
fea430a553 Add partition info to the 'sectors status' command 2021-09-02 20:01:15 +02:00
Łukasz Magiera
727bb84592 stmgr: drop MethdosMap 2021-09-02 18:58:34 +02:00
Łukasz Magiera
30fccaa0bd fix lint 2021-09-02 18:45:18 +02:00
Łukasz Magiera
95b128b7bc chain: Cleanup consensus logic 2021-09-02 18:09:37 +02:00
Łukasz Magiera
daaa725e3b sectors expired: Handle precomitted and unproven sectors correctly 2021-08-31 16:24:29 +02:00
Steven Allen
1cf556c3a2 feat: expose ChainGetPath on the gateway 2021-08-30 16:43:21 -07:00
Łukasz Magiera
22d75f4843 Tweak miner info --blocks output 2021-08-27 13:41:41 +02:00
Łukasz Magiera
4630216c38 Merge branch 'info-blocks' of github.com:filcloud/lotus-mirror into filcloud-info-blocks 2021-08-27 13:20:05 +02:00
Łukasz Magiera
a63f5c8bf2
Merge pull request #7025 from lanzafame/fix/need-empty-storage-json 
fix: init restore adds empty storage.json
 2021-08-27 12:29:27 +02:00
Łukasz Magiera
c19ee58f46
Merge pull request #7194 from filecoin-project/feat/big-data 
Insert miner and network power data as gibibytes to avoid int64 overflows
 2021-08-27 11:19:54 +02:00
Travis Person
0e8abc1973 Insert miner and network power data as gibibytes to avoid int64 overflows. 2021-08-27 00:01:44 +00:00
Łukasz Magiera
d6e309aa01 Show alert count in lotus-miner info 2021-08-26 15:45:19 +02:00
Łukasz Magiera
b094e0913d untangle fsjournal dependencies 2021-08-26 15:44:45 +02:00
Anton Evangelatov
548865e8dd simplify assignments to vmemBar and ramBar 2021-08-25 11:48:15 +03:00
Anton Evangelatov
0eacb9b9c8 call string.Repeat always with positive int 2021-08-25 11:48:15 +03:00
Aayush Rajasekaran
2495c52b05
Merge pull request #7099 from filecoin-project/asr/verifreg-nonmsig 
Shed: Create a verifreg command for when VRK isn't a multisig
 2021-08-24 21:03:38 -04:00
Łukasz Magiera
5a23c2bb90 sectors expired: Address review 2021-08-24 11:29:25 +02:00
Łukasz Magiera
a9bf24695d Fix lint 2021-08-23 11:27:34 -07:00
Łukasz Magiera
ccf8844689 lotus-miner sectors expired --remove-expired 2021-08-23 11:27:34 -07:00
Łukasz Magiera
d1759a4335 Show more info in sectors expired cmd 2021-08-23 11:27:34 -07:00
Łukasz Magiera
927ef041f8 miner: Command to list expired sectors 2021-08-23 11:27:34 -07:00
Łukasz Magiera
69204eea04 shed: Support raw encoding in cid id 2021-08-20 17:12:26 +02:00
Frrist
518b6f1d41
fix(miner): always create miner deal staging directory (#7098) 
- fixes #7097

Co-authored-by: Raúl Kripalani <raul@protocol.ai>

Co-authored-by: Raúl Kripalani <raul@protocol.ai>
 2021-08-17 00:16:06 +01:00
Aarsh Shah
d7076778e2
integrate DAG store and CARv2 in deal-making (#6671) 
This commit removes badger from the deal-making processes, and
moves to a new architecture with the dagstore as the cental
component on the miner-side, and CARv2s on the client-side.

Every deal that has been handed off to the sealing subsystem becomes
a shard in the dagstore. Shards are mounted via the LotusMount, which
teaches the dagstore how to load the related piece when serving
retrievals.

When the miner starts the Lotus for the first time with this patch,
we will perform a one-time migration of all active deals into the
dagstore. This is a lightweight process, and it consists simply
of registering the shards in the dagstore.

Shards are backed by the unsealed copy of the piece. This is currently
a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so
when it's time to acquire a shard to serve a retrieval, the unsealed
CARv1 is joined with its index (safeguarded by the dagstore), to form
a read-only blockstore, thus taking the place of the monolithic
badger.

Data transfers have been adjusted to interface directly with CARv2 files.
On inbound transfers (client retrievals, miner storage deals), we stream
the received data into a CARv2 ReadWrite blockstore. On outbound transfers
(client storage deals, miner retrievals), we serve the data off a CARv2
ReadOnly blockstore.

Client-side imports are managed by the refactored *imports.Manager
component (when not using IPFS integration). Just like it before, we use
the go-filestore library to avoid duplicating the data from the original
file in the resulting UnixFS DAG (concretely the leaves). However, the
target of those imports are what we call "ref-CARv2s": CARv2 files placed
under the `$LOTUS_PATH/imports` directory, containing the intermediate
nodes in full, and the leaves as positional references to the original file
on disk.

Client-side retrievals are placed into CARv2 files in the location:
`$LOTUS_PATH/retrievals`.

A new set of `Dagstore*` JSON-RPC operations and `lotus-miner dagstore`
subcommands have been introduced on the miner-side to inspect and manage
the dagstore.

Despite moving to a CARv2-backed system, the IPFS integration has been
respected, and it continues to be possible to make storage deals with data
held in an IPFS node, and to perform retrievals directly into an IPFS node.

NOTE: because the "staging" and "client" Badger blockstores are no longer
used, existing imports on the client will be rendered useless. On startup,
Lotus will enumerate all imports and print WARN statements on the log for
each import that needs to be reimported. These log lines contain these
messages:

- import lacks carv2 path; import will not work; please reimport
- import has missing/broken carv2; please reimport

At the end, we will print a "sanity check completed" message indicating
the count of imports found, and how many were deemed broken.

Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>

Co-authored-by: Raúl Kripalani <raul@protocol.ai>
Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
 2021-08-16 23:34:32 +01:00
raulk
473a192b85
fix build after merging #6097. (#7096) 
fix build after merging #6097.
 2021-08-16 23:18:32 +01:00
ZenGround0
10615b53ad
Merge pull request #6853 from filecoin-project/frrist/miner-temp-file-path 
feat(miner deals): create subdir to miner repo for staged deals
 2021-08-16 16:19:05 -04:00
ZenGround0
2e5b492edd
Merge pull request #6097 from filcloud/sectors-renew 
Extending sectors: more practical and flexible tools
 2021-08-16 13:50:38 -04:00
Aayush Rajasekaran
90e60f7a98 Shed: Create a verifreg command for when VRK isn't a multisig 2021-08-16 12:56:27 -04:00
He Weidong
b50b1e9ac4 Optimize for ci 2021-08-14 10:41:01 +08:00
He Weidong
c5f81d35f5 minor modification 2021-08-13 13:47:48 +08:00
Raúl Kripalani
5f6166435c make lotus-miner net commands hit markets subsystem. 2021-08-12 11:15:00 +01:00
Steven Allen
dcff06bd91 test: genesis actors version 
Properly handle genesis in actors tests. Fast-forward upgrading to
actors v13 doesn't work because there needs to be at least a day between
v0 genesis and v13 (due to differences in miner cron).
 2021-08-11 13:30:15 -07:00
swift-mx
d6562bccfc
Merge branch 'filecoin-project:master' into master 2021-08-11 19:25:27 +08:00
mx
46a93bb78a fix: get other actor controlList 2021-08-11 19:23:37 +08:00
lanzafame
0b8a211e70 fix: init restore adds empty storage.json 2021-08-11 13:36:01 +10:00
Steven Allen
130fc6fce1
Merge pull request #7019 from filecoin-project/asr/genesis-shed 
Shed: Include network name in genesis-verify
 2021-08-10 16:25:02 -07:00
Aayush Rajasekaran
feb9d3c557 Shed: Include network name in genesis-verify 2021-08-10 14:46:24 -04:00
ZenGround0
6da26dcefc Remove network version panic 2021-08-10 13:07:30 -04:00
Frank
2744bb4e5a update 2021-08-04 15:49:35 +08:00
Frank
2fec0c2440 add produced blocks info 2021-08-04 15:41:06 +08:00
vyzo
c9124bd1a9 clear splitstore dir before deleting, only warn if the latter fails 
to address situations with symlinks
 2021-08-03 15:39:48 +03:00
vyzo
db6b7b91d8 add option to only clear splistore keys 2021-08-03 15:32:53 +03:00
He Weidong
94c5076710 Make flags more intuitive 2021-08-02 16:44:22 +08:00
He Weidong
66e8418125 Enable exclude 2021-08-02 16:39:16 +08:00
He Weidong
677afbcb91 update to miner5 2021-08-02 16:26:29 +08:00
He Weidong
7f6c942b81 Use max-fee instead of gas-feecap 2021-08-02 16:09:00 +08:00
He Weidong
e09cdc45a7 Add gas-feecap flag for cmd sectors renew 2021-08-02 16:08:57 +08:00
He Weidong
9ee80fd1db Prettify json output of extending params 2021-08-02 16:08:55 +08:00
He Weidong
77166f88ac Optimize search of sectors location 2021-08-02 16:08:52 +08:00
He Weidong
e0777fed15 Add a more flexible cli tool: sectors renew 2021-08-02 16:08:50 +08:00
He Weidong
075bb3b314 Add a cli tool: sectors check-expire 2021-08-02 16:08:47 +08:00
vyzo
4a1655b225 add lotus-shed splitstore clear command 2021-08-01 19:18:29 +03:00
frrist
ca697c9aa2 feat(miner deals): create subdir to miner repo for staged deals 2021-07-29 21:08:38 -07:00
Aayush Rajasekaran
e5fb4a6c04
Merge pull request #6942 from filecoin-project/asr/nit 
Improve formatting of workers CLI
 2021-07-29 15:09:06 -04:00
Aayush Rajasekaran
1f7d584517
Update cmd/lotus-miner/sealing.go 2021-07-29 14:46:19 -04:00
Aayush Rajasekaran
28e3c8bd3f Improve formatting of workers CLI 2021-07-29 11:46:42 -04:00
Raúl Kripalani
8433aae1af use fallback api infos last; init service with markets-path. 2021-07-29 16:10:04 +01:00
Raúl Kripalani
b17fcfc4d1 bugfix. 2021-07-29 14:36:04 +01:00
Raúl Kripalani
90427bc3af fix docs, add flag. 2021-07-29 13:39:03 +01:00
Raúl Kripalani
3b2c3a3f6b polish. 2021-07-29 12:56:08 +01:00
Raúl Kripalani
d9c6a32b8e refactor miner info command. 2021-07-29 12:54:08 +01:00
Raúl Kripalani
22c0884f5f target markets API for markets commands. 2021-07-29 12:37:29 +01:00
raulk
80aedcd276
Merge branch 'master' into nonsense/get-enabled-subsystems-api 2021-07-28 20:09:43 +01:00
Raúl Kripalani
07f40b9488 fix docs and nits. 2021-07-28 20:03:25 +01:00
Łukasz Magiera
25052fe48f Remove chainwatch (now filecoin-project/sentinel) 2021-07-28 17:59:01 +02:00
Łukasz Magiera
389f71251c Remove townhall 2021-07-28 17:58:46 +02:00
Anton Evangelatov
6969202a03 make linter happy 2021-07-28 17:47:01 +03:00
Anton Evangelatov
54c6a3a454 extract outputs for mining node and markets node in separate functions 2021-07-28 17:31:48 +03:00
Anton Evangelatov
de4a847078 add RuntimeSubsystems API method; use it in lotus-miner info 2021-07-28 16:02:05 +03:00
Jiaying Wang
29b2d27f00
Merge branch 'master' into chore/cleanup-chain 2021-07-27 18:04:17 -04:00
raulk
35c90349ce
Merge pull request #6840 from filecoin-project/nonsense/lotus-shed-support-rollback-for-mra 
lotus-shed: initial export cmd for markets related metadata
 2021-07-27 22:13:29 +01:00
Jiaying Wang
d45b4a7a1f
Merge branch 'master' into chore/cleanup-chain 2021-07-27 17:11:36 -04:00
Jiaying Wang
4918a7c16f
Merge branch 'master' into feat/cli-super-verbose 2021-07-27 17:07:16 -04:00
raulk
e4b225f4ba
Merge branch 'master' into feat/cli-super-verbose 2021-07-27 20:49:44 +01:00
Raúl Kripalani
b04fb75a92 rename flag to very verbose. 2021-07-27 20:49:30 +01:00
vyzo
7c28ff3826
Merge branch 'master' into feat/moving-gc 2021-07-27 21:48:51 +03:00
Łukasz Magiera
e163d556fb
Merge branch 'master' into chore/cleanup-chain 2021-07-27 17:49:42 +02:00
Jiaying Wang
e46f02f59d
Merge branch 'master' into feat/vis-sector-alloc 2021-07-27 09:49:33 -04:00
Łukasz Magiera
ae63a4b33b fix lotus-sim build 2021-07-27 15:49:01 +02:00
Łukasz Magiera
9bd312881d VMSys doesn't belong in chainstore 2021-07-27 15:30:23 +02:00
Raúl Kripalani
48cddd3644 add a super verbose -vv flag to lotus and lotus-miner. 2021-07-27 14:25:28 +01:00
Raúl Kripalani
e0522a6b1f rename cmd/lotus{-storage=>}-miner to match binary. 2021-07-27 11:54:46 +01:00
Jiaying Wang
790f27d6a2
Merge branch 'master' into feat/vis-sector-alloc 2021-07-26 16:15:15 -04:00
Anton Evangelatov
9d61f91cdf
Update cmd/lotus-shed/market.go 
Co-authored-by: Aayush Rajasekaran <arajasek94@gmail.com>
 2021-07-26 16:41:22 +02:00
Jakub Sztandera
7de866d881
lint = happy 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-07-26 16:39:12 +02:00
vyzo
4715b1f436 fix lotus-shed 2021-07-26 15:04:46 +03:00
vyzo
221dc7024f add splitstore info command 2021-07-26 08:47:02 +03:00
vyzo
c99dc3e811 add splitstore check command 2021-07-25 11:24:06 +03:00
vyzo
e317c831a8 quiet excessive badger logs 2021-07-25 09:07:27 +03:00
vyzo
ce6f410f2a add options to control compaction/gc of the coldstore and config rewrite during rollback 2021-07-25 09:01:34 +03:00
vyzo
254c489164 fix typo 
Co-authored-by: raulk <raul@protocol.ai>
 2021-07-25 09:01:34 +03:00
vyzo
33cdc90f13 fix typo 
Co-authored-by: raulk <raul@protocol.ai>
 2021-07-25 09:01:34 +03:00
vyzo
da66e7a878 fix typo 2021-07-25 09:01:34 +03:00
vyzo
e696a2c0cc fix newline in progres message 2021-07-25 09:01:34 +03:00
vyzo
36b209ca5c compact and gc coldstore after copying 2021-07-25 09:01:34 +03:00
vyzo
92b9d8c895 quiet linter 2021-07-25 09:01:34 +03:00
vyzo
0c68bccbda add splitstore rollback lotus-shed command 2021-07-25 09:01:34 +03:00
Jakub Sztandera
e615eecbea Skip also set sectors 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-07-23 19:01:50 -04:00
Jakub Sztandera
39cc536115 Add sectors vis 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-07-23 19:01:48 -04:00
Łukasz Magiera
ac2a1bfeab fix lint 2021-07-23 15:53:31 +02:00
Łukasz Magiera
4757b014bc Nicer lotus-miner config commands 2021-07-23 14:57:42 +02:00
Łukasz Magiera
8b1f19e94c Move doc-comment logic to the config pkg 2021-07-23 14:55:19 +02:00
Łukasz Magiera
97f5bb66c7 config: doc struct codegen 2021-07-23 13:55:50 +02:00
Łukasz Magiera
cf65804883 lotus daemon config utils 2021-07-23 13:09:03 +02:00
Dirk McCormick
52498f78a1 feat: add import market datastore cmd 2021-07-23 10:30:58 +02:00
Dirk McCormick
7ef167b04f refactor: simplify market datastore backup 2021-07-23 09:33:26 +02:00
Anton Evangelatov
adba595350 commit from @dirkmc - initial export cmd for martkers related metadata 2021-07-22 18:41:57 +02:00
Łukasz Magiera
15a6995cda fix lint 2021-07-22 16:31:04 +02:00
Łukasz Magiera
0656b62176 Merge remote-tracking branch 'origin/master' into feat/compact-sectors-numbers-cmd 2021-07-22 16:04:25 +02:00
Steven Allen
f9c759fcb2 fix: always check if StateSearchMessage returns nil 
It returns nil on "not found".

Fixes the cases not covered in:
https://github.com/filecoin-project/lotus/pull/6787
 2021-07-20 11:28:27 -07:00
Peter Rabbitson
a37ababc20 Minor rearrange 2021-07-14 08:32:07 +02:00
Peter Rabbitson
677c193e6c Add current power to the stats 2021-07-14 08:11:23 +02:00
Łukasz Magiera
dd09666399
Merge pull request #6743 from filecoin-project/chore/centralize_all_cli_color_handling 
Handle the --color flag via proper global state
 2021-07-13 17:34:25 +02:00
Łukasz Magiera
837322ea59
Merge pull request #6356 from filecoin-project/nonsense/split-market-miner-processes 
Support standalone miner-market process
 2021-07-13 17:16:07 +02:00
Anton Evangelatov
2af02af1c1 rename --name to --type; provide empty stores.StorageConfig when init new service 2021-07-13 12:57:43 +02:00
Peter Rabbitson
6efd2e7335 Handle the --color flag via proper global state 
Use `color.NoColor` as a global progressively modified as one descends
down command flags. Allows non-surprising operation when --color is
accepted at multiple layers, and even makes this behave predictably:

    lotus-miner --color actor control list --color=false
 2021-07-13 12:22:29 +02:00
Łukasz Magiera
4433c3cd7c
Merge pull request #6708 from filecoin-project/feat/miner-info-deals 
Add more deal details to lotus-miner info
 2021-07-12 22:18:56 +02:00
Anton Evangelatov
7728d6bfc3 remove --enable-markets flag; add --name string slice flag 2021-07-12 14:38:29 +02:00
Anton Evangelatov
4bc9fa04ba rename cfg.Subsystems.EnableStorageMarket to EnableMarkets 2021-07-12 12:12:29 +02:00
Anton Evangelatov
d89ddb9315 resolve conflicts 2021-07-12 11:34:37 +02:00
Anton Evangelatov
16784aa2cc remove pieceProvider from DI; small refactors 2021-07-12 11:30:26 +02:00
Aayush Rajasekaran
20582037c5 Fix Lotus shed 2021-07-11 14:03:22 -04:00
Aayush Rajasekaran
e983d90a28
Merge pull request #6066 from chadwick2143/extend-v1-sectors 
Fix bugs in sectors extend --v1-sectors
 2021-07-11 13:19:35 -04:00
Aayush Rajasekaran
ea9ad8c5a8
Update cmd/lotus-storage-miner/sectors.go 2021-07-11 13:01:19 -04:00
Aayush Rajasekaran
93d929a10a
Merge pull request #5950 from zgfzgf/feat-shed-election-backtest 
add election backtest
 2021-07-11 12:08:21 -04:00
Łukasz Magiera
44e81424f2 Print completed deal stats in miner info 2021-07-08 19:40:05 +02:00
Łukasz Magiera
ed0cc0b56d Handle bad peerIDs in MarketListRetrievalDeals 2021-07-08 19:27:03 +02:00
Łukasz Magiera
10825e3a95 Add more info about deals to miner info 2021-07-08 19:02:18 +02:00
Łukasz Magiera
8f7bf1df86
Merge pull request #6697 from filecoin-project/feat/dealpublish-ctl-config 
Config for deal publishing control addresses
 2021-07-08 19:00:29 +02:00
Peter Rabbitson
140a2e0200 Stabilize default text and make gen docsgen-cli 2021-07-08 10:44:13 +02:00
Peter Rabbitson
c7bb326c78 Enable color by default only if os.Stdout is a TTY 2021-07-07 18:12:24 +02:00
Łukasz Magiera
8f5c0c60f5 Config for deal publishing control addresses 2021-07-07 18:00:54 +02:00
Anton Evangelatov
6b014f57e5 pass Subsystems to StorageMiner option; add enableLibp2p bool in Settings 2021-07-07 13:56:37 +02:00
Anton Evangelatov
7a78527fde remove not used sa; cleanup 2021-07-06 17:46:21 +02:00
turuslan
8697edc8b7 fix "lotus-seed car" error "merkledag: not found" 2021-07-06 12:54:09 +03:00
Anton Evangelatov
4be0a7a215 resolve conflicts with master 2021-07-05 13:13:32 +02:00
Łukasz Magiera
8a94ab676e storage: Fix FinalizeSector with sectors in stoage paths 2021-07-02 19:54:45 +02:00
Anton Evangelatov
adba1a792a add missing err check 2021-07-02 15:15:16 +02:00
Anton Evangelatov
ab6e01e379 remove --enable-markets flag, read from repo 2021-07-02 15:08:25 +02:00
Anton Evangelatov
3aa6d03cc1 connect miner to full node , only when running markets 2021-06-30 16:45:11 +02:00
Anton Evangelatov
4f0a96c9c7 resolved conflicts 2021-06-30 13:16:52 +02:00
Anton Evangelatov
9ec7cd7239 enable libp2p options 2021-06-30 12:55:43 +02:00
Łukasz Magiera
022d4b548a shed tool to estimate aggregate network fees 2021-06-29 20:42:26 +02:00
Łukasz Magiera
cefd140e45
Merge pull request #6329 from filecoin-project/raulk/itests-refactor-kit 
revamped integration test kit (aka. Operation Sparks Joy)
 2021-06-28 13:20:49 +02:00
Anton Evangelatov
7fb93d4585 refactor kit.NewDealHarness to include main miner and market node 2021-06-23 11:24:55 +02:00
Steven Allen
63e2caae81 lint(lotus-sim): handle error 2021-06-22 15:06:44 -07:00
Steven Allen
87c306fd47 feat(lotus-sim): use current power instead of lookback 
I'd _really_ like to use lookback, but don't have that when starting
from a snapshot.
 2021-06-22 15:06:30 -07:00
Steven Allen
0b06de2bd3 fix(lotus-sim): unembed Node from Simulation 
I wanted to expose the node's _fields_, but this also exposed the
methods. That got rather confusing.

(probably could have used a new type, but eh)

foo
 2021-06-22 14:53:06 -07:00
Anton Evangelatov
6720463799 resolve merge conflicts 2021-06-22 11:28:23 +02:00
Steven Allen
b7c36bc02c fix(lotus-sim): make NewNode take a repo 
This is primarily for testing, so we can just pass an in-memory repo.
 2021-06-21 11:32:19 -07:00
Steven Allen
c532b1b819 fix(lotus-sim): remove unused field and function 2021-06-21 11:25:40 -07:00
Steven Allen
80eba1069a feat(lotus-sim): NewNode to construct a node without a repo 2021-06-21 11:25:40 -07:00
Steven Allen
ffb63a93ff fix(lotus-sim): make 'fund' easier to understand 2021-06-21 11:25:40 -07:00
Raúl Kripalani
83ee345b1c Merge branch 'master' into raulk/itests-refactor-kit 2021-06-21 18:19:26 +01:00
Steven Allen
d6abcff63c
fix(lotus-sim): apply code review from magik6k 
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
 2021-06-21 09:05:48 -07:00
Raúl Kripalani
7ee184e965 fix lotus-storage-miner tests. 2021-06-21 11:56:10 +01:00
Steven Allen
69a8a6bc0e fix(lotus-sim): lint 2021-06-18 15:44:37 -07:00
Jakub Sztandera
2fdf49e7da Add histogram and quantiles for message sizes 
Resolves https://github.com/filecoin-project/lotus/issues/6513

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-06-18 15:44:37 -07:00
Steven Allen
eb2b706156 chore: fix lint errors in simulation 2021-06-18 15:44:37 -07:00
Steven Allen
2aedd82c72 fix(lotus-sim): correct window post batch sizes 2021-06-18 15:44:37 -07:00
Steven Allen
bc2698a988 fix(lotus-sim): simulate using realistic gas numbers 
Previously, we assumed the network was "optimal". Now, we're using real
numbers.
 2021-06-18 15:44:37 -07:00
Steven Allen
a26cd5a809 fix(lotus-sim): mark provecommit stage initialized 2021-06-18 15:44:37 -07:00
Steven Allen
f0d0b40bd3 fix(lotus-sim): debug log mock 2021-06-18 15:44:37 -07:00
Steven Allen
ce29a0ac17 fix(lotus-sim): initialize commit queue 2021-06-18 15:44:37 -07:00
Steven Allen
28e6fa5923 fix(lotus-sim): remove duplicate error handling 2021-06-18 15:44:37 -07:00
Steven Allen
ec5fab09a1 feat(lotus-sim): log failing proofs 2021-06-18 15:44:37 -07:00
Steven Allen
e41f0842b0 fix(lotus-sim): load prove-commits 
(regression from refactor)
 2021-06-18 15:44:37 -07:00
Steven Allen
af33d69357 fix(lotus-sim): don't close node on list 2021-06-18 15:44:37 -07:00
Steven Allen
885062f712 fix(lotus-sim): fix info state imports 2021-06-18 15:44:37 -07:00
Steven Allen
73ae1924bc feat(lotus-sim): state size command 2021-06-18 15:44:37 -07:00
Steven Allen
22267eb45d feat(lotus-sim): split info command file 2021-06-18 15:44:36 -07:00
Steven Allen
5d7b7ce5c1 feat(lotus-sim): allow profile info 2021-06-18 15:44:36 -07:00
Steven Allen
95cf577447 fix(lotus-sim): really cancel walk immediately 2021-06-18 15:44:36 -07:00
Steven Allen
8fffaa5c47 fix(lotus-sim): average over 2 days 
There's too much noise per day.
 2021-06-18 15:44:36 -07:00
Steven Allen
63178ce982 feat(lotus-sim): daily capacity growth 2021-06-18 15:44:36 -07:00
Steven Allen
0af7dcdedb fix(lotus-sim): rename power to capacity 2021-06-18 15:44:36 -07:00
Steven Allen
f9d2a23132 fix(lotus-sim): correctly handle cancellation in walk 
1. Select order is not guaranteed, always check if the context has been
canceled explicitly.
2. Never close a work channel unless we're actually done. This can yield
out-of-order results due to buffering.
 2021-06-18 15:44:36 -07:00
Steven Allen
ec3f969e9a feat(lotus-sim): allow walking back past the start 2021-06-18 15:44:36 -07:00
Steven Allen
f6043a0250 feat(lotus-sim): measure daily power growth 2021-06-18 15:44:36 -07:00
Steven Allen
52261fb814 refactor(lotus-sim): enterprise grade 
While the previous version "worked", this version nicely separates out
the state for the separate stages. Hopefully, we'll be able to use this
to build different pipelines with different configs.
 2021-06-18 15:44:36 -07:00
Jakub Sztandera
7dd58efb84 Add quantiles and histogram 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-06-18 15:44:36 -07:00
Steven Allen
2721279e87 fix(lotus-sim): describe info commands 2021-06-18 15:44:36 -07:00
Steven Allen
985994cc0f feat(lotus-sim): add command for analyzing post stats over time 2021-06-18 15:44:36 -07:00
Steven Allen
707b3bf08a fix(lotus-sim): refuse to start simulation with no miners 2021-06-18 15:44:36 -07:00
Steven Allen
d551f2b4bd feat(lotus-sim): print duration info in days 2021-06-18 15:44:36 -07:00
Steven Allen
1df5445ed2 feat(lotus-sim): make walk parallel 2021-06-18 15:44:36 -07:00
Steven Allen
fbaffe86da fix(lotus-sim): return error from walk 2021-06-18 15:44:36 -07:00
Steven Allen
500fae6a52 fix(lotus-sim): less indentation in info 2021-06-18 15:44:36 -07:00
Steven Allen
ab59474c4c fix(lotus-sim): count single prove-commits when computing efficiency 2021-06-18 15:44:36 -07:00
Jakub Sztandera
3d3c26fa0c Add lookback limit 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-06-18 15:44:36 -07:00
Steven Allen
68593ce995 fix(lotus-sim): obey context in walk 2021-06-18 15:44:36 -07:00
Jakub Sztandera
ca9eadd7c7 Add gas info command 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-06-18 15:44:36 -07:00
Steven Allen
16449007ab fix(lotus-sim): fix funding error check 2021-06-18 15:44:35 -07:00
Steven Allen
8d734d81d9 fix(lotus-sim): log failed pre-commits and continue 2021-06-18 15:44:35 -07:00
Steven Allen
783dc5a33d fix(lotus-sim): fund multiple times 
Sometimes, a miner is deep in the red.
 2021-06-18 15:44:35 -07:00
Steven Allen
be713ec04a fix(lotus-sim): we always fill the block with pre-commits 2021-06-18 15:44:35 -07:00
Steven Allen
c18ca60d28 fix(lotus-sim): specify ErrWriter 
Apparently, it defaults to nil...
 2021-06-18 15:44:35 -07:00
Steven Allen
977bf1cad9 fix(lotus-sim): write pprof profiles to a directory 
We need to know the sizes up-front for tar, and that's not happening.
 2021-06-18 15:44:35 -07:00
Steven Allen
7a8bfd8725 doc(lotus-sim): document signals 2021-06-18 15:44:35 -07:00
Steven Allen
a3f64e0768 fix(lotus-sim): profile signal handling 2021-06-18 15:44:35 -07:00
Steven Allen
5766002370 fix(lotus-sim): guard info with dashes 2021-06-18 15:44:35 -07:00
Steven Allen
4a80c83533 fix(lotus-sim): fix spelling 2021-06-18 15:44:35 -07:00
Steven Allen
936659d087 feat(lotus-sim): print info on SIGUSR1 2021-06-18 15:44:35 -07:00
Steven Allen
dcdb0abe27 feat(lotus-sim): profile on SIGUSR2 2021-06-18 15:44:35 -07:00
Steven Allen
2b77c17546 chore(lotus-sim): fix import grouping 
I got a bit lazy when developing this.
 2021-06-18 15:44:35 -07:00
Steven Allen
1802ae31a6 feat(lotus-sim): record timing information for pre/prove-commit packing 2021-06-18 15:44:35 -07:00
Steven Allen
868231adc7 fix(lotus-sim): don't take from the fund account when funding 2021-06-18 15:44:35 -07:00
Steven Allen
2b4f865665 feat(lotus-sim): fund from a funding account 
Instead of funding the simulation from the burnt-funds actor, fund from
a custom account and take a "tax" to fund that account.

Otherwise, we run out of funds...
 2021-06-18 15:44:35 -07:00
Steven Allen
86e459f585 feat(lotus-sim): return receipt 
Instead of returning a "full" boolean and an error, return a receipt and
an error. We now use the error to indicate if the block is "full".
 2021-06-18 15:44:35 -07:00
Steven Allen
88af350774 fix(lotus-sim): use global base-fee value 2021-06-18 15:44:35 -07:00
Steven Allen
747b3d3e57 fix(lotus-sim): skip miners without power when loading 2021-06-18 15:44:35 -07:00
Steven Allen
ba65a1ba9b chore(lotus-sim): rename stat to info 2021-06-18 15:44:35 -07:00
Steven Allen
e097ba8640 feat(lotus-sim): wire up signal handler 2021-06-18 15:44:35 -07:00
Steven Allen
4578e0dd8d chore(lotus-sim): remove dead code 2021-06-18 15:44:35 -07:00
Steven Allen
77f0fee58e chore(lotus-sim): fix comment about simulation block miner 2021-06-18 15:44:35 -07:00
Steven Allen
bb4753ffbf feat(lotus-sim): add commands to rename and copy 2021-06-18 15:44:35 -07:00
Steven Allen
4f0b9eefc1 fix(lotus-sim): check for slash in names on copy 2021-06-18 15:44:35 -07:00
Steven Allen
8410b0f79f feat(lotus-sim): add a feature to copy/rename simulation. 
Useful to backup old simulations before creating a new one.
 2021-06-18 15:44:35 -07:00
Steven Allen
c5dc67ccd8 feat(lotus-sim): add a walk function 
This way, we can easily walk the chain and:

1. Inspect messages and their results.
2. Inspect tipset state.
 2021-06-18 15:44:35 -07:00
Steven Allen
ccdd660f0d feat(lotus-sim): more stats 2021-06-18 15:44:35 -07:00
Steven Allen
5f6733fe44 feat(lotus-sim): expose StateManager from Simulation 2021-06-18 15:44:35 -07:00
Steven Allen
0faacbe154 feat(lotus-sim): store start tipset 2021-06-18 15:44:35 -07:00
Steven Allen
a57c509e1e fix(lotus-sim): cleanup and document pre-commit filtering 2021-06-18 15:44:35 -07:00
Steven Allen
f9ebe3017d test(lotus-sim): test commit-queue rollover 2021-06-18 15:44:35 -07:00
Steven Allen
0725019bdb feat(lotus-sim): completely pack block 
Instead of packing till we see "full". Prove-commits are large, we may
have room for some more pre-commits.
 2021-06-18 15:44:34 -07:00
Steven Allen
dfdafa3c15 fix(lotus-sim): pretend all messages are BLS 
It doesn't really matter, and it ensures they all get executed in-order.
 2021-06-18 15:44:34 -07:00
Steven Allen
5b31ae39ea fix: test commit queue 2021-06-18 15:44:34 -07:00
Steven Allen
82019ce474 fix(lotus-sim): correctly merge forward commit queue. 2021-06-18 15:44:34 -07:00
Steven Allen
2e4f526375 fix(lotus-sim): skip (and log) missing/expired pre-commits 2021-06-18 15:44:34 -07:00
Steven Allen
0ccf716989 fix(lotus-sim): refactor miner state loading 
Add a helper function so we don't need to constantly repeat ourselves.
 2021-06-18 15:44:34 -07:00
Steven Allen
7925b69573 doc(lotus-sim): document block generation logic 2021-06-18 15:44:34 -07:00
Steven Allen
2f7d7aed31 feat(lotus-sim): refactor and document 
Hopefully, this'll make this code a bit easier to approach.
 2021-06-18 15:44:34 -07:00
Steven Allen
b7bfc06ebe feat(lotus-sim): add an info command 2021-06-18 15:44:34 -07:00
Steven Allen
be9e30e39d fix(lotus-sim): rename step to run 
And make it run forever by default.
 2021-06-18 15:44:34 -07:00
Steven Allen
8000decac6 feat(lotus-sim): add command to list pending upgrades 2021-06-18 15:44:34 -07:00
Steven Allen
e2f5c494b0 feat: implement lotus-sim 2021-06-18 15:44:34 -07:00
Łukasz Magiera
44de67cf7d
Merge pull request #6175 from filecoin-project/feat/dynamic-retreival-pricing 
Dynamic Retrieval pricing
 2021-06-17 10:25:48 +02:00
Aayush Rajasekaran
35f76f58df
Merge pull request #6309 from filecoin-project/feat/better_mining_projection_and_accuracy 
Adjust various CLI display ratios to arbitrary precision
 2021-06-16 17:48:23 -04:00
Anton Evangelatov
d45bb14015 Merge branch 'master' into nonsense/split-market-miner-processes 2021-06-16 18:49:15 +02:00
Aayush Rajasekaran
bee548face Add utils to use multisigs as miner owners 2021-06-15 16:57:52 -04:00
Anton Evangelatov
323c78ef44 comment for checkV1ApiSupport 2021-06-15 14:47:11 +02:00
Peter Rabbitson
ae2b089a5c Be even more precise - properly account for multi-win Poisson 2021-06-14 23:57:17 +02:00
Peter Rabbitson
4d1fccd290
Better wolfram link 2021-06-14 17:54:38 +02:00
Łukasz Magiera
0514f38dc4 Merge remote-tracking branch 'origin/master' into chore/merge-1.10 2021-06-14 13:27:53 +02:00
Anton Evangelatov
31e6fb154e resolve conflicts 2021-06-14 11:27:54 +02:00
aarshkshah1992
3d086dfb43 changes as per review 2021-06-14 09:40:34 +05:30
Łukasz Magiera
ec06f086ef sealing: Early finalization option 2021-06-11 11:41:28 +02:00
aarshkshah1992
fed5afa704 merge master 2021-06-11 09:35:20 +05:30
Łukasz Magiera
6f80d9f3c2
Merge pull request #6311 from filecoin-project/raulk/itests 
consolidate integration tests into `itests` package; create test kit; cleanup
 2021-06-10 12:33:29 +02:00
Raúl Kripalani
6d46be53bd make tests no longer create auth tokens. 2021-06-10 00:15:29 +01:00
Steven Allen
a64a059780 implement a command to export a car 2021-06-09 15:52:32 -07:00
Anton Evangelatov
beb5bffaf6 fix log.Error; rename RemoteStore 2021-06-09 13:05:54 +02:00
Anton Evangelatov
9f3ec82cc1 initial resolution of conflicts 2021-06-08 13:01:22 +02:00
aarshkshah1992
16166504f4 merged master 2021-06-08 09:17:40 +05:30
Raúl Kripalani
40ea660cdd fix TestWorkerKeyChange. 2021-06-07 23:56:59 +01:00
Raúl Kripalani
a274fcc533 Merge branch 'master' into raulk/itests 2021-06-07 23:44:22 +01:00
Raúl Kripalani
db1a61852d Merge branch 'master' into raulk/itests 2021-06-07 23:17:44 +01:00
aarshkshah1992
4efaa8d688 unit tests for the remote store Reader 2021-06-07 15:02:52 +05:30
aarshkshah1992
759d8f090b test http handler 2021-06-07 15:02:52 +05:30
aarshkshah1992
65eb610ec3 docs, logs and green ci 2021-06-07 15:02:49 +05:30
Anton Evangelatov
8bd09e39ca resolve merge conflicts 2021-06-04 16:17:00 +02:00
Łukasz Magiera
6d9779ecae Merge remote-tracking branch 'origin/master' into feat/nv13-1.11 2021-06-02 15:07:13 +02:00
Łukasz Magiera
ae90d7c3b5 Fix lint 2021-06-02 12:00:39 +02:00
Travis Person
07102ec686 lotus-gateway: add check command 
The check command provides a quick way to try out different transports
and verify that a connection can be made to a running gateway.
 2021-06-02 11:59:11 +02:00
Łukasz Magiera
dc642d0b7b Merge remote-tracking branch 'origin/feat/nv13' into feat/nv13-1.11 2021-06-01 21:06:58 +02:00
Łukasz Magiera
482e1110c2 precommit batcher: Improve error propagation 2021-06-01 14:35:30 +02:00
Łukasz Magiera
5c3ebd424c miner: Improve batching CLI UX 2021-06-01 12:33:24 +02:00
Łukasz Magiera
cd0a1c97fa Improve cli for flushing commit batches 2021-06-01 12:25:19 +02:00
Łukasz Magiera
3671f2a6ff fix 2k build 2021-05-31 21:52:23 +02:00
Łukasz Magiera
c3e8eddb9b Merge remote-tracking branch 'origin/master' into feat/nv13-1.11 2021-05-31 21:24:56 +02:00
Aayush Rajasekaran
92f544d96c Add verifreg utils to CLI 2021-05-31 15:01:55 -04:00
Łukasz Magiera
937366f04a lotus-wallet: Describe setup with auth 2021-05-31 14:33:54 +02:00
Łukasz Magiera
7d1ae7daf5 lotus-wallet: Support permissioned api 2021-05-31 13:56:35 +02:00
Łukasz Magiera
47608c1937 wallet: Handle jwt headers 2021-05-31 13:38:14 +02:00
Anton Evangelatov
d195a12565 Merge branch 'master' into nonsense/split-market-miner-processes 2021-05-31 11:26:11 +02:00
Anton Evangelatov
46a368d2b9 bump api ver 2021-05-27 13:35:54 +02:00
Łukasz Magiera
1e4456138e Merge master into feat/nv13 2021-05-27 12:28:20 +02:00
Aayush Rajasekaran
1ad2c4dab0 Make devnets work again 2021-05-26 12:48:19 -04:00
Aayush Rajasekaran
7714537239 Allow starting networks from arbitrary actor versions 2021-05-26 12:48:14 -04:00
Łukasz Magiera
43c62f4406
Revert "Allow starting networks from arbitrary actor versions" 2021-05-26 12:33:08 +02:00
Aayush Rajasekaran
cf574ca9a1 Allow starting networks from arbitrary actor versions 2021-05-25 19:30:20 -04:00
Łukasz Magiera
f5409845b5 Some review addressing 2021-05-25 16:07:45 +02:00
Peter Rabbitson
c2e5a837e6 Adjust various CLI display ratios to arbitrary precision 
Originally the deviations from using float64 were insignificant, but at
exabyte scale they start to show up. Cleanup all displays, and clarify
the expectation text, adding an extra 99.9% probability calculator to
`lotus-miner info`
 2021-05-25 14:09:01 +02:00
Łukasz Magiera
05082e262a
Merge pull request #6316 from Aloxaf/fix_completion 
Fix shell completions
 2021-05-24 13:44:02 +02:00
Aloxaf
559c76a214
Fix shell completions 2021-05-24 15:00:47 +08:00
Raúl Kripalani
f6a7f21def Merge branch 'raulk/refactor-rpc' into raulk/itests 2021-05-23 22:00:49 +01:00
Raúl Kripalani
57dab5b71c remove previously redundant context. 2021-05-23 19:12:13 +01:00
Raúl Kripalani
d8d48953d9 Merge branch 'raulk/refactor-rpc' into raulk/itests 2021-05-23 18:47:55 +01:00
Raúl Kripalani
188688c9ce refactor gateway rpc. 2021-05-23 18:37:53 +01:00
Raúl Kripalani
c3f45cc836 Merge branch 'raulk/refactor-rpc' into raulk/itests 2021-05-23 14:56:54 +01:00
Raúl Kripalani
2337248aa5 rpc: separate handlers/server; refactor miner rpc. 2021-05-23 14:48:42 +01:00
Raúl Kripalani
627248d610 Merge branch 'raulk/refactor-rpc' into raulk/itests 2021-05-23 12:41:25 +01:00
aarshkshah1992
dc6dbc9a11 dpr changes and test based on new unsealing PR 2021-05-22 22:40:21 +05:30
Raúl Kripalani
5b77dbd06f decouple rpc server and shutdown logic from cmd. 2021-05-21 21:34:17 +01:00
Raúl Kripalani
2e9b0f0895 adapt cmd/lotus-storage-miner tests. 2021-05-21 14:52:30 +01:00
Łukasz Magiera
807c670184
Merge pull request #6135 from filecoin-project/asr/verifreg 
Move verifreg shed utils to CLI
 2021-05-21 11:21:38 +02:00
aarshkshah1992
bd9959070e unit tests for the remote store Reader 2021-05-20 15:25:46 -06:00
aarshkshah1992
dd9c9fd4fd test http handler 2021-05-20 15:25:46 -06:00
aarshkshah1992
73613ee883 docs, logs and green ci 2021-05-20 15:25:46 -06:00
Anton Evangelatov
8d3d3c88d4 introduce init service, refactor restore service 2021-05-20 13:25:54 +02:00
Raúl Kripalani
06b8ee5b95 Merge branch 'master' into raulk/itests 2021-05-20 11:56:43 +01:00
Anton Evangelatov
2562f2e9a6 add ReaderParamDecoder to json rpc server to lotus-miner run 2021-05-20 12:36:24 +02:00
Anton Evangelatov
2aad7b6979 update sectorstorage.New (Manager) interface 2021-05-20 12:32:29 +02:00
Łukasz Magiera
678812f35b gofmt, lint 2021-05-20 12:05:04 +02:00
Łukasz Magiera
e0a520cb65
Merge pull request #6294 from filecoin-project/raulk/refactor-gateway 
gateway: spin off from cmd to package
 2021-05-19 20:59:37 +02:00
Raúl Kripalani
ac4f3ab684 remove duplicated vars. 2021-05-19 18:17:38 +01:00
Raúl Kripalani
2a70ff96bf move the gateway test to itests package. 2021-05-19 17:34:48 +01:00
Raúl Kripalani
625b18771d remove duplicated vars. 2021-05-19 17:30:06 +01:00
Łukasz Magiera
a5677d1b7a ffiwrapper: Separate Prover interface 2021-05-19 15:20:23 +02:00
Raúl Kripalani
88c6642330 restructure gateway. 
Move the gateway implementation to the `gateway` top-level
package. cmd/lotus-gateway now contains only the entrypoint.

This separation is important for the testing refactors.
 2021-05-19 12:45:55 +01:00
Anton Evangelatov
a989f60e27 add SectorAddPieceToAny and SectorUnsealPiece to StorageMiner iface; model moved to api package - PieceDealInfo, DealSchedule 2021-05-19 13:05:07 +02:00
Aayush Rajasekaran
fc7ea8dfa3 Useful tool 2021-05-18 21:13:12 -04:00
Łukasz Magiera
0419c64a06 CLI for precommit batching 2021-05-18 20:35:35 +02:00
Łukasz Magiera
f66b9c5663 Maybe working precommit batching 2021-05-18 17:21:16 +02:00
Łukasz Magiera
6278bdc69a Make things build 2021-05-17 20:47:41 +02:00
whyrusleeping
b4b38788ef add flags to control gateway lookback parameters 2021-05-13 11:00:42 -07:00
chenliang
3b7a5c70c3
Merge branch 'master' into lotus-shed-actor 2021-05-12 11:03:49 +08:00
Łukasz Magiera
506f39b294 WIP: Integrate FIP0013 2021-05-11 22:10:29 -04:00
Aayush Rajasekaran
bf10b051ff
Merge pull request #6108 from filecoin-project/feat/actors-codegen 
wip actor wrapper codegen
 2021-05-10 20:12:57 -04:00
Aayush Rajasekaran
8465a97c4b
Merge pull request #5822 from filecoin-project/epic/robust-message-management 
Robust message management
 2021-05-10 19:58:33 -04:00
Łukasz Magiera
a80259d986 Actor upgrade checklist 2021-05-10 19:46:44 -04:00
Aayush Rajasekaran
9977cb2604
Merge pull request #6169 from filecoin-project/asr/shed-minertypes 
Add a shed util to count miners by post type
 2021-05-10 19:45:52 -04:00
Aayush Rajasekaran
b5da2655dc Introduce v5 actors 2021-05-10 19:44:28 -04:00
Łukasz Magiera
63189cd81d Generate wrappers for new actor versions 2021-05-09 21:04:50 -04:00
Jakub Sztandera
18cbdcfc81
Disable checks API on Lotus Lite 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 17:19:17 +02:00
Jakub Sztandera
61dbd443b8
Fix tests and verifreg 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 15:30:05 +02:00
Jakub Sztandera
7535c5bb53
Add mpool manage command 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 15:30:05 +02:00
vyzo
e2d0047a2a
introduce message prototypes 
This introduces message prototypes to applicable API endpoints, which
allows us to invert control of message sending and give the user a
chance to intervene with an interactive ui.

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-05-07 15:30:05 +02:00
ZenGround0
5351db9da7 Lint 2021-05-06 10:17:25 -04:00
ZenGround0
e1185dd5b5 cron-wc 2021-05-06 09:53:29 -04:00
Aayush Rajasekaran
f6360c34dd Add verifreg utils to CLI 2021-05-03 22:16:12 -04:00
Aayush Rajasekaran
edc6a63e93 Add a shed util to count miners by post type 2021-05-01 00:24:14 -04:00
Aayush Rajasekaran
e6779b0b6f Add a command to get the fees of a deal 2021-04-29 13:29:17 -04:00
Steven Allen
f983fa8cd5
Merge pull request #6134 from filecoin-project/asr/emptytsk 
Use EmptyTSK where appropriate
 2021-04-29 09:35:43 -07:00
Steven Allen
2857f6c0ed fix: use a consistent tipset in commands 
It's very easy to write an incorrect command that operates over
different heads by using the "empty" tipset. This change makes the
`LoadTipSet` command helper get the latest head from the lotus daemon if
its unset.

The cost is an extra call to get the head. That should be trivial in
most cases.
 2021-04-29 08:50:08 -07:00
Aayush Rajasekaran
d794b49df3 Use EmptyTSK where appropriate 2021-04-29 01:00:03 -04:00
Steven Allen
bcfad6b2bb fix(lotus-shed): sanity check start height for dup check 2021-04-28 17:51:25 -07:00
Steven Allen
701682c98a feat(lotus-shed): make it possible to filter by to/from when checking dups 2021-04-28 17:43:07 -07:00
lotus
77eefcd6d8 feat(lotus-shed): improve duplicate-messages audit command 
- Allow a start/end epoch.
- Print one line per duplicate set.
- Allow filtering duplicate messages by method number.
- Make the number of threads configurable.
 2021-04-28 17:03:37 -07:00
Łukasz Magiera
c8fcab5d22 shed: Command to list duplicate messages in tipsets 2021-04-28 16:23:35 -07:00
Łukasz Magiera
a24a73bb52 Merge tag 'v1.8.0' into release/v1.9.0 2021-04-27 10:30:33 +02:00
Łukasz Magiera
c4137a438d Merge commit 'f1ded63d608e1b191d79c780b8065f8cff367c98' into feat/nv12-1.11 2021-04-27 08:22:55 +02:00
Aayush Rajasekaran
f1ded63d60 v4 specs-actors integration, nv12 migration 2021-04-27 02:02:44 -04:00
Łukasz Magiera
86d4f5f738
Merge pull request #6075 from filecoin-project/feat/gateway-walletbalance 
implement WalletBalance on gateway
 2021-04-23 16:00:54 +02:00
Aayush Rajasekaran
448b5cb2a3 Shed util to sanity-check total balance is FilBase 2021-04-22 19:45:35 -04:00
Cory Schwartz
39e49f1700 implement WalletBalance on gateway 2021-04-20 21:55:47 -07:00
chadwick2143
2cb3c89570 Fix bugs in sectors extend --v1-sectors 2021-04-20 14:00:01 +08:00
Łukasz Magiera
4436c184ed Fix v0/v1 API versions 2021-04-16 00:20:13 +02:00
chadwick2143
cba911ab5e Add actorControlList command 2021-04-14 16:37:59 +08:00
chadwick2143
8972152eb1 Set MaxFee for lotus-shed actor withdraw to 0.1FIL 2021-04-14 16:34:57 +08:00
Łukasz Magiera
e76935147f Merge remote-tracking branch 'origin/master' into feat/v1-api 2021-04-13 14:24:22 +02:00
Aayush Rajasekaran
c1e4eb3b20 Add a whitelisted block 2021-04-10 02:12:38 -04:00
Aayush Rajasekaran
470bd307c3 add a shed util to verify BLS signatures 2021-04-10 01:40:24 -04:00
Łukasz Magiera
1e6bba5388
Merge pull request #5943 from filcloud/4857-set-addrs 
fix 4857: show help for set-addrs
 2021-04-10 02:00:34 +02:00
Frank
2dbd597f82
Update cmd/lotus-storage-miner/actor.go 
Co-authored-by: Łukasz Magiera <magik6k@users.noreply.github.com>
 2021-04-08 11:14:53 +08:00
zgfzgf
640d6fa474 optimize code 2021-04-06 22:40:05 +08:00
Peter Rabbitson
25a77d905d Centralize everything on ipfs/go-log/v2 
I am not entirely sure this is right, but everything seems to build...
 2021-04-06 15:04:32 +02:00
Łukasz Magiera
cf96ad4fdb fix lint 2021-04-06 14:30:49 +02:00
Łukasz Magiera
c3736f40fe gateway: use correct path in serveRpc 2021-04-06 12:29:11 +02:00
Łukasz Magiera
e8f28d7b9f Fix tests 2021-04-06 12:24:58 +02:00
Frank
4403158680 add unset for set-addrs 2021-04-06 11:24:32 +08:00
Aayush Rajasekaran
37ca23d9e2 Merge branch 'releases' into asr/release-master 2021-04-05 17:29:19 -04:00
Łukasz Magiera
eee50caaf1 Fix buildall 2021-04-05 20:12:47 +02:00
Łukasz Magiera
81bd27911f Propagate StateMsg api changes 2021-04-05 19:56:53 +02:00
Łukasz Magiera
49f8b8df00
Merge pull request #5900 from filecoin-project/mg/chore/move-mpool-clear 
chore: Move lotus mpool clear to lotus-shed
 2021-04-05 18:43:57 +02:00
Łukasz Magiera
d8bff4d19f Make gateway work with v1 api 2021-04-05 13:47:10 +02:00
Łukasz Magiera
deb2b90b6a Fix lotus/miner build 2021-04-05 13:23:46 +02:00
Aayush Rajasekaran
72c410cd8f Address review on v1 sector extension tool 2021-04-03 22:19:36 -04:00
Aayush Rajasekaran
01bf3a89f4 Add checks to v1 extension util 2021-04-03 16:54:16 -04:00
Aayush Rajasekaran
b734e5a232 Miner CLI: If performing read-only operations on a foreign miner, miner API isn't needed 2021-04-03 16:54:16 -04:00
Aayush Rajasekaran
ded4ff43f4 Allow miners to extend all v1 sectors 2021-04-03 16:54:16 -04:00
zgfzgf
ef08bd48b5 add election backtest 2021-04-03 11:52:30 +08:00
Frank
0fe705605e fix 4857: show help for set-addrs 2021-04-02 19:25:36 +08:00
Łukasz Magiera
c9866da688 Merge remote-tracking branch 'origin/master' into feat/v1-api 2021-04-02 13:15:33 +02:00
Łukasz Magiera
0cfc76c078
Merge pull request #5891 from filecoin-project/feat/backupds-trunc-log-handling 
backupds: Improve truncated log handling
 2021-04-02 13:04:11 +02:00
Łukasz Magiera
cf4128fc78
Merge pull request #5092 from zgfzgf/feat-shed-election 
add shed election estimate command
 2021-04-01 17:05:09 +02:00
Mike Greenberg
c800211fbb chore: Move lotus mpool clear to lotus-shed 2021-03-29 12:54:08 -04:00
chadwick2143
4395f27143 Transplant some useful commands to lotus-shed actor 
Transplant some useful commands from lotus-miner actor to lotus-shed actor,
so that you can excute them without miner api.
 2021-03-29 12:27:08 +08:00
Łukasz Magiera
411f561db6 backupds: Improve truncated log handling 2021-03-27 10:05:12 +01:00
Łukasz Magiera
e003977559 apiclient: Version client funcs 2021-03-25 15:39:48 +01:00
Łukasz Magiera
7a23b411ae Merge apistruct with the api package 2021-03-25 15:10:08 +01:00
Łukasz Magiera
759ea5b14f Merge remote-tracking branch 'origin/master' into feat/v1-api 2021-03-25 14:13:21 +01:00
Łukasz Magiera
6d9ae1e28e
Merge pull request #5819 from chadwick2143/lotus-shed 
Add --actor flag in lotus-shed sectors terminate
 2021-03-25 14:02:37 +01:00
Łukasz Magiera
8f78066d4f
Merge pull request #5875 from filecoin-project/feat/log-compact-restart 
backupds: Compact log on restart
 2021-03-25 10:42:39 +01:00
Łukasz Magiera
924c419e61 backupds: handle log bool in shed 2021-03-24 21:26:24 +01:00
Aayush Rajasekaran
dac171a99b Add a util to validate FILPoll votes 2021-03-24 00:00:11 -04:00
Aayush Rajasekaran
fdb69fdd71 Add a util to decode base64 inputs 2021-03-23 23:32:56 -04:00
Łukasz Magiera
eed5840de3 fix lotus-shed 2021-03-23 19:37:30 +01:00
Łukasz Magiera
65dcec0ebc api client: Use versioned api packages 2021-03-23 19:15:53 +01:00
Łukasz Magiera
d198cf456e make docsgen work with versioned api 2021-03-23 17:20:56 +01:00
Łukasz Magiera
571114bc44 Scaffolding for API versioning 2021-03-23 16:40:22 +01:00
Łukasz Magiera
c41777dcd2
API proxy struct codegen (#5854) 
* mostly working api proxy gen

* api: Consistent api names

* fix docsgen

* regenerate api struct

* api: expand external interfaces

* Add missing gen files

* apigen: fix perm detection

* api: Move perm tags to the interface

* gofmt

* worker perms

* docsgen

* docsgen: ignore tag comments

* apigen: add codegen warning

* gofmt

* missing actor type

* docsgen

* make linter happy

* fix lint

* apigen: use directives for tags

* docsgen

* regen openrpc docs
 2021-03-23 13:42:56 +01:00
Łukasz Magiera
43d9cc36a4
OpenRPC Support (#5843) 
* main: init implement rpc.Discover RPC method

This implement the basic functionality for the method
over HTTP RPC.

Signed-off-by: meows <b5c6@protonmail.com>

* main,go.mod,go.sum: init example with go-openrpc-reflect lib

Signed-off-by: meows <b5c6@protonmail.com>

 Conflicts:
	go.mod
	go.sum

* main: make variable name human-friendly

Signed-off-by: meows <b5c6@protonmail.com>

* main,go.mod,go.sum: init impl of go-openrp-reflect printing document

Signed-off-by: meows <b5c6@protonmail.com>

 Conflicts:
	go.mod
	go.sum

* go.mod,go.sum: use go-openrpc-reflect and open-rpc/meta-schema hackforks

This is for development only.
Versions need to be bumped when they're ready for use
as canonical remotes.

Signed-off-by: meows <b5c6@protonmail.com>

* main,openrpc,main: refactor openrpc supporting code to own package

This eliminates code duplication.

Signed-off-by: meows <b5c6@protonmail.com>

* main: add rpc.Discover to openrpc document

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: fix rpc.discover method name casing

Also fixes casing stuff for the rest of Filecoin.
methods.

Signed-off-by: meows <b5c6@protonmail.com>

* Revert "main: add rpc.Discover to openrpc document"

This reverts commit 116898efb10f33e405ac74acb1aa6daefcd46a62.

* main: fix document creation method name

This fixes an issue caused with the latest reverting
commit.

Signed-off-by: meows <b5c6@protonmail.com>

* main,docgen,openrpc: refactor to share api parsing, etc as docgen exported stuff

Signed-off-by: meows <b5c6@protonmail.com>

Makefile: fix docgen refactoring for makefile use of command

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: add schema.examples to app reflector

There are quite of few of these already registered
for the docgen command, so it makes sense to use
those!

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: init method pairing examples

Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: bump go.mod to use latest meta-schema and openrpc-reflect versions

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: init SchemaType mapper function

This function will handle the manual configurations
for app-specific data types w/r/t their json schema
representation.

This is useful for cases where the reflect library
is unable to provide a sufficient representation
automatically.

Provided in this commit is an initial implementation
for the integerD type (assuming number are represented
in the API as hexs), and a commonly used cid.Cid type.

Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: tame dependencies by bumping etclabscore/go-openrpc-reflect

This removes a problematic dependency
on github.com/ethereum/go-ethereum, which was
imported as a dependency for a couple github.com/etclabscore/go-openrpc-reflect
tests.

etclabscore/go-openrpc-reflect v0.0.36 has removed this
dependency, so this commit is the result of bumping
that version and then running 'go mod tidy'

This is in response to a review at
https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205

Date: 2020-11-21 06:52:48-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main: add 'miner' arg to openrpc gen cmd

This allows the command to EITHER
generate the doc for Full or Miner APIs.

See comment for usage.

Date: 2020-11-21 07:48:05-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgen: add missing examples for Miner API

Generating the Miner API OpenRPC doc
(via 'go run ./api/openrpc/cmd miner') caused
the example logic to panic because some types
were missing.

This commit adds those missing types, although
I'm not an expert in the API so I can't
suggest that the example values provided are
ideal or well representative.

Date: 2020-11-21 07:50:21-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build/openrpc/full.json,build/openrpc/miner.json: add build/openrpc/[full/miner].json docs

These will be used as static documents
provided by the rpc.discover method.

Date: 2020-11-21 07:51:39-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: init go-rice openrpc static assets

Date: 2020-11-21 08:23:06-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main: remove rpc.discover implementation from runtime plugin

Instead of generating the doc on the fly,
we're going to serve a static asset.
Rel https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205
This removes the runtime implementation from the
RPC server construction.

Date: 2020-11-21 08:41:20-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api,apistruct,common: add Discover(ctx) method to CommonAPI interface and structs

Date: 2020-11-21 08:41:56-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main: use rpc server method aliasing for rpc.discover

This depends on a currently-forked change at
filecoin-project/go-jsonrpc 8350f9463ee451b187d35c492e32f1b999e80210
which establishes this new method RPCServer.AliasMethod.

This solves the problem that the OpenRPC
spec says that the document should be served
at the system extension-prefixed endpoing
rpc.discover (not Filecoin.Discover).

In fact, the document will be available at BOTH
endpoints, but that duplicity is harmless.

Date: 2020-11-21 09:18:26-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api,apistruct,build,common: rpc.discover: return json object instead of string

Instead of casting the JSON asset from bytes to string,
unmarshal it to a map[string]interface{} so the
server will provide it as a JSON object.

Date: 2020-11-21 09:27:11-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* Makefile: merge resolve: docsgen command path

Date: 2020-11-22 07:19:36-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* apistruct,main,docgen,openrpc: merge resolve: fix func exporteds, signatures

Date: 2020-11-22 07:31:03-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: 'get get' auto-bumps version

Date: 2020-11-22 07:31:44-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* Makefile,docgen,main,build/openrpc: refactor openrpc documentation generation

This creates Makefile command docsgen-openrpc-json,
and refactors the docsgen command to generate both
the markdown and openrpc json documents, redirecting
the output of the openrpc json documentation to
the build/openrpc/ directory, where those json
files will be compiled as static assets via go-rice
boxes.

The api/openrpc/cmd now uses usage argumentation
congruent to that of the docgen command (switching
on API context).

Date: 2020-11-22 08:01:18-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main,docgen_openrpc: rename api/openrpc -> api/docgen-openrpc

Renames the package as well.

This is intended to parallel the
existing docgen package and command
namespacing.

Date: 2020-11-22 10:34:46-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api,apistruct,docgen,build,build/openrpc: use typed Discover response

Instead of using a map[string]interface{}, use
a typed response for the Discover method implementation.

This avoids having to set a docgen Example for
the generic map[string]interface{} (as an openrpc document)
which both pollutes the generic type and lacks
useful information for the Discover method example.

Date: 2020-11-22 08:31:16-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* apistruct,build,main,impl: implement Discover method for Worker and StorageMiner APIs

Methods return static compiled assets respective
to the APIs.

Date: 2020-11-22 08:57:18-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgen_openrpc,build/openrpc: remove timestamping from openrpc doc info

This should allow openrpc docs generated at different
times to be equal. This is important because the CI
(Circle) runs the docgen command and tests that
the output and the source are unchanged (via git diff).

Date: 2020-11-22 10:47:07-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main,docgen_openrpc,main,build: fix lint issues

Fixes goimports, staticcheck, golint issues.

Date: 2020-11-22 11:06:46-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgenopenrpc: fix: don't use an underscore in package name (golint)

Date: 2020-11-22 11:07:53-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.sum: fix: mod-tidy-check (run 'go mod tidy')

Date: 2020-11-22 11:09:48-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: bump filecoin-project/go-jsonrpc dep to latest

This version includes the necessary RPCServer.AliasMethod
method.

Date: 2020-11-23 12:16:15-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* Makefile,main,build,build/openrpc: init gzipped openrpc static docs

Date: 2020-11-24 06:15:06-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: refactor gzip reading

Date: 2020-11-24 06:18:34-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: add basic test for openrpc doc from static assets

Date: 2020-11-24 06:30:23-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: handle reader Close error

This keeps the errcheck linter happy.

Date: 2020-11-24 06:33:14-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.sum: run 'go mod tidy'

Date: 2020-11-24 06:36:07-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: go mod tidy

Tidying up after resolving the merge conflicts
with master at go.mod

Date: 2020-11-24 06:40:45-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: bump filecoin-project/go-jsonrpc to latest

This is a repeat of 76e6fd2, since the latest merge
to master seems to have reverted this.

Date: 2020-11-24 06:42:30-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgenopenrpc,build/openrpc: remove method example pairings, improve schema examples

Removing method example pairings since they were
redundant to schema examples and were not
implemented well.

Improved schema examples by using the ExampleValue
method instead of the map lookup.
Made a note in the comment here that this is
not ideal, since we have to make a shortcut assumption
/workaround by using 'unknown' as the method name
and the typea as its own parent.

Luckily these values aren't heavily used by the
method logic.

Date: 2020-11-27 12:57:36-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgenopenrpc: use generic number jsonschema for number types

Previously used an integer schema assuming
hex encoding. It appears, based on review some
of the examples, that this may not be the case.

Obvioussly this schema could be more descriptive,
but just shooting for mostly likely to be
not wrong at this point.

Date: 2020-12-15 14:44:37-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* cmd/lotus,go.mod,go.sum: maybe fix straggling merge resolution conflicts

Date: 2021-01-19 12:30:42-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build/openrpc/full.json.gz,build/openrpc/miner.json.gz,build/openrpc/worker.json.gz: run 'make docsgen'

Date: 2021-01-19 12:33:55-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/apistruct,node/impl: (lint) gofmt

Date: 2021-01-19 12:39:48-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/docgen: maybe fix parse error:  open ./api: no such file or directory

Date: 2021-01-19 12:52:04-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/docgen,build/openrpc: maybe fix no such file error and run 'make docsgen'

Date: 2021-01-19 12:55:52-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/docgen: return if AST comment/groupdoc parsing encounters any error

This will returns empty comments/docs maps.
This should fix issues like:
https://app.circleci.com/pipelines/github/filecoin-project/lotus/12445/workflows/4ebadce9-a298-4ad1-939b-f19ef4c0a5bf/jobs/107218

where the environment makes file lookups hard or
impossible.

Date: 2021-01-19 13:04:58-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api: Don't depend on build/

* make: support parallel docsgen

* openrpc gen: Use simple build version

* methodgen

* goimports

Co-authored-by: meows <b5c6@protonmail.com>
 2021-03-19 19:22:46 +01:00
Peter Rabbitson
c340750672 Nerpa reset backport/cherry-pick from 73eabc310f 2021-03-18 20:55:15 +01:00
Aayush Rajasekaran
80ff0b0564 Improve setting of VRK multisigs 2021-03-17 00:18:50 -04:00
chadwick2143
7417a13be3 Add --actor flag in lotus-shed sectors terminate 
Can specify miner actor address when terminate sectors,
so that you can terminate sectors without miner api.
 2021-03-16 14:09:46 +08:00
Łukasz Magiera
58213760b7 gofmt 2021-03-12 15:56:10 +01:00
Łukasz Magiera
88b0d551b9 shed command to unpack miner info dumps 2021-03-12 12:24:09 +01:00
Łukasz Magiera
a373fc4c4c make lotus-miner info all more robust 2021-03-12 12:23:43 +01:00
Łukasz Magiera
504affd266
Merge pull request #5757 from filecoin-project/chore/sr2_stats_factored_out 
Remove the SR2 stats, leave just the network totals
 2021-03-10 14:15:35 +01:00
Łukasz Magiera
05274943c9
Merge pull request #5620 from filecoin-project/feat/disable-owner-worker-fallback 
miner: Config to disable owner/worker address fallback
 2021-03-10 11:21:39 +01:00
Łukasz Magiera
40fdf6c180
Merge pull request #5624 from filecoin-project/feat/miner-storage-limit 
Configurable storage path storage limit
 2021-03-10 11:15:28 +01:00
Peter Rabbitson
bdafdf8060 Remove the SR2 stats, leave just the network totals 
The SR2 aggregator now lives at https://github.com/filecoin-project/slingshot-stats
 2021-03-09 23:36:03 +00:00
Łukasz Magiera
58cacda733 Fix bootstrapper profile setting 2021-03-09 23:46:13 +01:00
Łukasz Magiera
1c62d7a40f
Merge pull request #5729 from filecoin-project/feat/api-no-dep-build 
Feat/api no dep build
 2021-03-09 16:01:53 +01:00
Steven Allen
5304df52ca fix lotus-shed compile 2021-03-08 17:19:31 -08:00
Steven Allen
6addd9afcc remove unused imports 2021-03-08 16:43:21 -08:00
Łukasz Magiera
58d7627168
Merge pull request #5730 from filecoin-project/asr/genesis-rootkey 
Customize verifreg root key and remainder account when making genesis
 2021-03-08 22:42:09 +01:00
Łukasz Magiera
305c2ec77d miner: Config to disable owner/worker addcess fallback 2021-03-08 21:33:46 +01:00
Łukasz Magiera
7c7b107e97 Add max-storage flags to storage attach commands 2021-03-08 20:59:59 +01:00
Łukasz Magiera
8a5cd21034
Merge pull request #5666 from filecoin-project/feat/extend-sectors-cmd 
Command to extend sector expiration
 2021-03-08 17:57:35 +01:00
Aayush Rajasekaran
6534280673 lotus-seed: add a command to set the remainder 2021-03-06 01:43:48 -05:00
Aayush Rajasekaran
dbbcb1423f lotus-seed: add a command to set the verified registry root key 2021-03-06 01:43:48 -05:00
whyrusleeping
e7a1d72ba8 extract build/version to api/version, remove api package dep on build 2021-03-05 14:28:13 -08:00
Łukasz Magiera
58e49e3259 Move api client builders to a cliutil package 2021-03-05 21:08:41 +01:00
Łukasz Magiera
fe230f901e Collect worker task metrics 2021-03-05 12:48:17 +01:00
Łukasz Magiera
448813d2fe
Merge pull request #5695 from filecoin-project/feat/segregate-blockstores 
segregate chain and state blockstores
 2021-03-03 10:53:46 +01:00
Łukasz Magiera
8bd5173a54
Merge branch 'next' into refactor/lib/blockstore 2021-03-01 19:04:40 +01:00
Łukasz Magiera
5fe37404bc Merge remote-tracking branch 'origin/master' into next 2021-03-01 19:03:01 +01:00
Raúl Kripalani
3795cc2bd2 segregate chain and state blockstores. 
This paves the way for better object lifetime management.

Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
  sizing caches differently.
- specifying different retention policies for chain and state data.

This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
  store, and only retain state objects reachable from the last finality
  in the state store.
 2021-02-28 22:49:44 +00:00
Raúl Kripalani
7f0f7d0b36 Merge branch 'master' into refactor/lib/blockstore 2021-02-28 19:55:23 +00:00
Aayush Rajasekaran
191a05da48 Fixup get-cc-collateral command 2021-02-28 02:10:22 -05:00
Łukasz Magiera
29c9fa3137
Merge pull request #5612 from filecoin-project/feat/cc-fsm-cleanup 
storagefsm: Cleanup CC sector creation
 2021-02-26 11:39:28 +01:00
Łukasz Magiera
f5d819ba53
Merge pull request #5669 from ghoneycutt/import_error 
Improve error message with importing a chain
 2021-02-25 14:31:33 +01:00
Garrett Honeycutt
92b605466f
Improve error message with importing a chain 
Without this commit the error message does not provide much value as it
lacks context.
 2021-02-24 19:29:29 -05:00
Łukasz Magiera
a3a78fc705 Command to extend sector expiration 2021-02-23 09:27:14 +01:00
Łukasz Magiera
37b1fb97fb Configurable storage path storage limit 2021-02-18 16:44:34 +01:00
Łukasz Magiera
d8c9712383 Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input 2021-02-17 21:24:48 +01:00
Łukasz Magiera
d12545af6f
Merge pull request #5469 from filecoin-project/feat/faucet-captcha 
Add spam protection to fountain
 2021-02-17 20:17:28 +01:00
Aayush Rajasekaran
f358af6f56 Fix typo 2021-02-16 19:35:35 -05:00
Łukasz Magiera
fd90c03018 Roturn SectorID from PledgeSector 2021-02-16 19:16:35 +01:00
Łukasz Magiera
7111e924cc Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input 2021-02-11 13:30:22 +01:00
Łukasz Magiera
0c6aef221d Fix error logging format strings 2021-02-11 12:00:26 +01:00
Cory Schwartz
039ca9f3b4 return the err 2021-02-09 17:05:56 -08:00
Cory Schwartz
6d4caa0163 lotus-seed command to generate lotus block 
Currently devnet deployment requires spinning up a full daemon.
This would move the daemon `lotus-make-genesis` logic to lotus-seed
so it can be more easily executed in scripts, etc.
 2021-02-09 16:54:32 -08:00
Łukasz Magiera
5a3b983954
Merge pull request #5529 from filecoin-project/feat/shed-rpc-miner 
shed: Support miner nodes in rpc util
 2021-02-09 17:23:08 +01:00
Łukasz Magiera
e92b8b24b1 Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input 2021-02-09 14:04:21 +01:00
Dirk McCormick
eb03422c93 feat: add ends-in to pending publish deals output 2021-02-08 18:06:50 +01:00
Dirk McCormick
614844dc94 feat: output more info in pending-publish CLI 2021-02-08 12:08:42 +01:00
Łukasz Magiera
3ff6a6f59f address review; flush tablewriter 2021-02-05 22:33:53 +01:00
Łukasz Magiera
cfa73f34e4 market: miner CLI for managing pending deals 2021-02-05 21:55:43 +01:00
Łukasz Magiera
dc6392db49
Merge pull request #5219 from filecoin-project/feat/interactive-wallet 
lotus-wallet: Add interactive mode
 2021-02-05 13:07:25 +01:00
Łukasz Magiera
641eeb4783 lotus-wallet: Better interactive prompts 2021-02-05 12:47:55 +01:00
Łukasz Magiera
ab5345860c gofmt 2021-02-05 12:44:07 +01:00
Łukasz Magiera
06d3892e38 lotus-wallet: Add interactive mode 2021-02-05 12:44:01 +01:00
Łukasz Magiera
ebfde330dd shed: Support miner nodes in rpc util 2021-02-04 20:06:05 +01:00
whyrusleeping
0efb4ca3a7 add an api endpoint to get pending transactions for multisigs 2021-02-03 20:46:10 -08:00
Łukasz Magiera
6e7fcb7ee6 Merge remote-tracking branch 'origin/master' into feat/refactor-fsm-input 2021-02-03 14:01:56 +01:00
whyrusleeping
ed6ccef5cf export metrics for lotus-gateway 2021-02-01 13:13:20 -08:00
Łukasz Magiera
fed3e82ff8
Merge pull request #5047 from zgfzgf/feat-shed-bitfield 
shed bitfield: Cleanup, add merge command
 2021-01-30 12:54:39 +01:00
Łukasz Magiera
b1dfd41c19
Merge pull request #4950 from zgfzgf/feat-bench-policy 
delete bench policy
 2021-01-30 12:34:40 +01:00
Łukasz Magiera
a541a2500a
Merge pull request #5282 from filecoin-project/chore/snake_context_through_blockstore_init 
Snake a context through the Chain-blockstore creation
 2021-01-30 11:55:17 +01:00
Raúl Kripalani
af429dba0f Merge branch 'master' into chore/snake_context_through_blockstore_init 2021-01-29 21:21:46 +00:00
Raúl Kripalani
b0cbc932bd consolidate all blockstores in blockstore package. 2021-01-29 20:01:00 +00:00
Jakub Sztandera
0a0fc47655
Make lint happy 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-01-28 19:58:28 +01:00
Jakub Sztandera
608793314f
Fix url Values 
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-01-28 19:58:12 +01:00
Jakub Sztandera
ba94f271db
Add spam protection to fountain 
Uses reCAPTCHAv3, `RECAPTCHA_SITE_KEY` and `RECAPTCHA_SECRET_KEY` need to be
set in env.

Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
 2021-01-28 19:58:09 +01:00
Łukasz Magiera
94009f247f
Merge pull request #5411 from filecoin-project/feat/sealing-handle-batch-publish 
Handle batch publish storage deals message in sealing recovery
 2021-01-28 19:33:49 +01:00
Łukasz Magiera
3634bdcfa5
Merge pull request #5220 from filecoin-project/feat/verifpower-sector-list 
Show verified power in miner sector list
 2021-01-28 19:09:19 +01:00
Raúl Kripalani
a1da1dab85 add context to LockedRepo#Datastore(). 2021-01-26 11:01:43 +00:00
Raúl Kripalani
ce3af308ed Merge branch 'master' into chore/snake_context_through_blockstore_init 2021-01-25 19:31:41 +00:00
Dirk McCormick
1b494acc9e feat: handle batch publish storage deals message in sealing recovery 2021-01-25 14:10:05 +01:00
Steven Allen
a1e6781f0c fix tests 2021-01-21 15:21:20 -08:00
Steven Allen
ab4f135208 test actors v3 2021-01-21 15:21:20 -08:00
Steven Allen
026d236252 more reliably close in tests 2021-01-21 15:21:20 -08:00
Steven Allen
126a5c1cf8 chore: fix nolint comment 2021-01-21 15:21:19 -08:00
Aayush Rajasekaran
ab90a3b2bc Actors update: MinerInfo.SealProofType has been removed 2021-01-21 15:21:19 -08:00
Łukasz Magiera
b9a9f23204 storagefsm: Add stub AddPieceFailed state 2021-01-21 17:40:50 +01:00
Łukasz Magiera
239d6f8f4d storagefsm: Rewrite input handling 2021-01-21 17:40:50 +01:00
Aayush Rajasekaran
23ce0645ed
Merge pull request #5342 from filecoin-project/asr/change-owner 
Allow multisigs to be set as miner owner address
 2021-01-20 21:30:37 -05:00
Frank
e0209c74ad add verified indicator for storage-deals list 2021-01-20 15:14:51 +08:00
Łukasz Magiera
388c53e637
Merge pull request #5352 from filcloud/color-amount 
add color for token amount
 2021-01-19 11:47:22 +01:00
Dirk McCormick
827a473391 feat: expose StateSearchMessage on gateway 2021-01-19 11:27:43 +01:00
Łukasz Magiera
38f9559080 Implement full-node restore option 2021-01-15 20:57:32 +01:00
Frank
7747bcd859 remove some colorTokenAmount 2021-01-15 11:48:07 +08:00
Łukasz Magiera
1e53ed6a82 Print proving window info in pending termination list 2021-01-14 17:27:28 +01:00
Łukasz Magiera
9632a3836a Print msg cid in terminate flush 2021-01-14 16:47:30 +01:00