Commit Graph

446 Commits

Author SHA1 Message Date
Łukasz Magiera
43d9cc36a4
OpenRPC Support (#5843)
* main: init implement rpc.Discover RPC method

This implement the basic functionality for the method
over HTTP RPC.

Signed-off-by: meows <b5c6@protonmail.com>

* main,go.mod,go.sum: init example with go-openrpc-reflect lib

Signed-off-by: meows <b5c6@protonmail.com>

 Conflicts:
	go.mod
	go.sum

* main: make variable name human-friendly

Signed-off-by: meows <b5c6@protonmail.com>

* main,go.mod,go.sum: init impl of go-openrp-reflect printing document

Signed-off-by: meows <b5c6@protonmail.com>

 Conflicts:
	go.mod
	go.sum

* go.mod,go.sum: use go-openrpc-reflect and open-rpc/meta-schema hackforks

This is for development only.
Versions need to be bumped when they're ready for use
as canonical remotes.

Signed-off-by: meows <b5c6@protonmail.com>

* main,openrpc,main: refactor openrpc supporting code to own package

This eliminates code duplication.

Signed-off-by: meows <b5c6@protonmail.com>

* main: add rpc.Discover to openrpc document

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: fix rpc.discover method name casing

Also fixes casing stuff for the rest of Filecoin.
methods.

Signed-off-by: meows <b5c6@protonmail.com>

* Revert "main: add rpc.Discover to openrpc document"

This reverts commit 116898efb10f33e405ac74acb1aa6daefcd46a62.

* main: fix document creation method name

This fixes an issue caused with the latest reverting
commit.

Signed-off-by: meows <b5c6@protonmail.com>

* main,docgen,openrpc: refactor to share api parsing, etc as docgen exported stuff

Signed-off-by: meows <b5c6@protonmail.com>

Makefile: fix docgen refactoring for makefile use of command

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: add schema.examples to app reflector

There are quite of few of these already registered
for the docgen command, so it makes sense to use
those!

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: init method pairing examples

Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: bump go.mod to use latest meta-schema and openrpc-reflect versions

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: init SchemaType mapper function

This function will handle the manual configurations
for app-specific data types w/r/t their json schema
representation.

This is useful for cases where the reflect library
is unable to provide a sufficient representation
automatically.

Provided in this commit is an initial implementation
for the integerD type (assuming number are represented
in the API as hexs), and a commonly used cid.Cid type.

Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: tame dependencies by bumping etclabscore/go-openrpc-reflect

This removes a problematic dependency
on github.com/ethereum/go-ethereum, which was
imported as a dependency for a couple github.com/etclabscore/go-openrpc-reflect
tests.

etclabscore/go-openrpc-reflect v0.0.36 has removed this
dependency, so this commit is the result of bumping
that version and then running 'go mod tidy'

This is in response to a review at
https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205

Date: 2020-11-21 06:52:48-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main: add 'miner' arg to openrpc gen cmd

This allows the command to EITHER
generate the doc for Full or Miner APIs.

See comment for usage.

Date: 2020-11-21 07:48:05-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgen: add missing examples for Miner API

Generating the Miner API OpenRPC doc
(via 'go run ./api/openrpc/cmd miner') caused
the example logic to panic because some types
were missing.

This commit adds those missing types, although
I'm not an expert in the API so I can't
suggest that the example values provided are
ideal or well representative.

Date: 2020-11-21 07:50:21-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build/openrpc/full.json,build/openrpc/miner.json: add build/openrpc/[full/miner].json docs

These will be used as static documents
provided by the rpc.discover method.

Date: 2020-11-21 07:51:39-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: init go-rice openrpc static assets

Date: 2020-11-21 08:23:06-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main: remove rpc.discover implementation from runtime plugin

Instead of generating the doc on the fly,
we're going to serve a static asset.
Rel https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205
This removes the runtime implementation from the
RPC server construction.

Date: 2020-11-21 08:41:20-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api,apistruct,common: add Discover(ctx) method to CommonAPI interface and structs

Date: 2020-11-21 08:41:56-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main: use rpc server method aliasing for rpc.discover

This depends on a currently-forked change at
filecoin-project/go-jsonrpc 8350f9463ee451b187d35c492e32f1b999e80210
which establishes this new method RPCServer.AliasMethod.

This solves the problem that the OpenRPC
spec says that the document should be served
at the system extension-prefixed endpoing
rpc.discover (not Filecoin.Discover).

In fact, the document will be available at BOTH
endpoints, but that duplicity is harmless.

Date: 2020-11-21 09:18:26-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api,apistruct,build,common: rpc.discover: return json object instead of string

Instead of casting the JSON asset from bytes to string,
unmarshal it to a map[string]interface{} so the
server will provide it as a JSON object.

Date: 2020-11-21 09:27:11-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* Makefile: merge resolve: docsgen command path

Date: 2020-11-22 07:19:36-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* apistruct,main,docgen,openrpc: merge resolve: fix func exporteds, signatures

Date: 2020-11-22 07:31:03-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: 'get get' auto-bumps version

Date: 2020-11-22 07:31:44-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* Makefile,docgen,main,build/openrpc: refactor openrpc documentation generation

This creates Makefile command docsgen-openrpc-json,
and refactors the docsgen command to generate both
the markdown and openrpc json documents, redirecting
the output of the openrpc json documentation to
the build/openrpc/ directory, where those json
files will be compiled as static assets via go-rice
boxes.

The api/openrpc/cmd now uses usage argumentation
congruent to that of the docgen command (switching
on API context).

Date: 2020-11-22 08:01:18-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main,docgen_openrpc: rename api/openrpc -> api/docgen-openrpc

Renames the package as well.

This is intended to parallel the
existing docgen package and command
namespacing.

Date: 2020-11-22 10:34:46-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api,apistruct,docgen,build,build/openrpc: use typed Discover response

Instead of using a map[string]interface{}, use
a typed response for the Discover method implementation.

This avoids having to set a docgen Example for
the generic map[string]interface{} (as an openrpc document)
which both pollutes the generic type and lacks
useful information for the Discover method example.

Date: 2020-11-22 08:31:16-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* apistruct,build,main,impl: implement Discover method for Worker and StorageMiner APIs

Methods return static compiled assets respective
to the APIs.

Date: 2020-11-22 08:57:18-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgen_openrpc,build/openrpc: remove timestamping from openrpc doc info

This should allow openrpc docs generated at different
times to be equal. This is important because the CI
(Circle) runs the docgen command and tests that
the output and the source are unchanged (via git diff).

Date: 2020-11-22 10:47:07-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main,docgen_openrpc,main,build: fix lint issues

Fixes goimports, staticcheck, golint issues.

Date: 2020-11-22 11:06:46-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgenopenrpc: fix: don't use an underscore in package name (golint)

Date: 2020-11-22 11:07:53-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.sum: fix: mod-tidy-check (run 'go mod tidy')

Date: 2020-11-22 11:09:48-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: bump filecoin-project/go-jsonrpc dep to latest

This version includes the necessary RPCServer.AliasMethod
method.

Date: 2020-11-23 12:16:15-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* Makefile,main,build,build/openrpc: init gzipped openrpc static docs

Date: 2020-11-24 06:15:06-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: refactor gzip reading

Date: 2020-11-24 06:18:34-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: add basic test for openrpc doc from static assets

Date: 2020-11-24 06:30:23-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: handle reader Close error

This keeps the errcheck linter happy.

Date: 2020-11-24 06:33:14-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.sum: run 'go mod tidy'

Date: 2020-11-24 06:36:07-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: go mod tidy

Tidying up after resolving the merge conflicts
with master at go.mod

Date: 2020-11-24 06:40:45-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: bump filecoin-project/go-jsonrpc to latest

This is a repeat of 76e6fd2, since the latest merge
to master seems to have reverted this.

Date: 2020-11-24 06:42:30-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgenopenrpc,build/openrpc: remove method example pairings, improve schema examples

Removing method example pairings since they were
redundant to schema examples and were not
implemented well.

Improved schema examples by using the ExampleValue
method instead of the map lookup.
Made a note in the comment here that this is
not ideal, since we have to make a shortcut assumption
/workaround by using 'unknown' as the method name
and the typea as its own parent.

Luckily these values aren't heavily used by the
method logic.

Date: 2020-11-27 12:57:36-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgenopenrpc: use generic number jsonschema for number types

Previously used an integer schema assuming
hex encoding. It appears, based on review some
of the examples, that this may not be the case.

Obvioussly this schema could be more descriptive,
but just shooting for mostly likely to be
not wrong at this point.

Date: 2020-12-15 14:44:37-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* cmd/lotus,go.mod,go.sum: maybe fix straggling merge resolution conflicts

Date: 2021-01-19 12:30:42-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build/openrpc/full.json.gz,build/openrpc/miner.json.gz,build/openrpc/worker.json.gz: run 'make docsgen'

Date: 2021-01-19 12:33:55-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/apistruct,node/impl: (lint) gofmt

Date: 2021-01-19 12:39:48-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/docgen: maybe fix parse error:  open ./api: no such file or directory

Date: 2021-01-19 12:52:04-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/docgen,build/openrpc: maybe fix no such file error and run 'make docsgen'

Date: 2021-01-19 12:55:52-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/docgen: return if AST comment/groupdoc parsing encounters any error

This will returns empty comments/docs maps.
This should fix issues like:
https://app.circleci.com/pipelines/github/filecoin-project/lotus/12445/workflows/4ebadce9-a298-4ad1-939b-f19ef4c0a5bf/jobs/107218

where the environment makes file lookups hard or
impossible.

Date: 2021-01-19 13:04:58-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api: Don't depend on build/

* make: support parallel docsgen

* openrpc gen: Use simple build version

* methodgen

* goimports

Co-authored-by: meows <b5c6@protonmail.com>
2021-03-19 19:22:46 +01:00
Łukasz Magiera
1c62d7a40f
Merge pull request #5729 from filecoin-project/feat/api-no-dep-build
Feat/api no dep build
2021-03-09 16:01:53 +01:00
vyzo
4a74f752c0 implement extended peer info in net peers cli 2021-03-06 19:14:13 +02:00
whyrusleeping
e7a1d72ba8 extract build/version to api/version, remove api package dep on build 2021-03-05 14:28:13 -08:00
Łukasz Magiera
fd90c03018 Roturn SectorID from PledgeSector 2021-02-16 19:16:35 +01:00
Łukasz Magiera
3ff6a6f59f address review; flush tablewriter 2021-02-05 22:33:53 +01:00
Łukasz Magiera
b3f4e50c58 market: APIs to manage pending deals 2021-02-05 18:58:55 +01:00
whyrusleeping
0efb4ca3a7 add an api endpoint to get pending transactions for multisigs 2021-02-03 20:46:10 -08:00
Łukasz Magiera
ba571794dc
Merge pull request #5309 from filecoin-project/feat/deal-batch-publish
batch publish deal messages
2021-02-04 01:19:56 +01:00
whyrusleeping
2505ed051e properly wire up StateReadState on gateway api 2021-02-03 15:31:09 -08:00
Dirk McCormick
adac340f3f feat: batch publish deal messages 2021-01-25 14:25:57 +01:00
Dirk McCormick
827a473391 feat: expose StateSearchMessage on gateway 2021-01-19 11:27:43 +01:00
Łukasz Magiera
144b5a1350 perning termination API 2021-01-14 12:37:23 +01:00
Łukasz Magiera
1564db1fce Sector termination test 2021-01-14 00:11:41 +01:00
Łukasz Magiera
3522c8d45a SectorTerminateFlush API 2021-01-13 23:32:04 +01:00
Łukasz Magiera
52cc2cd3eb Initial sector termination support 2021-01-13 00:42:01 +01:00
Dirk McCormick
c58086ee27 feat: better CLI for wallet market withdraw and client info 2021-01-07 10:18:29 +01:00
Dirk McCormick
c6820ec059 feat: lotus wallet market add (adds funds to storage market actor) 2021-01-06 14:32:46 +01:00
Łukasz Magiera
2e154ef6d0
Merge pull request #5176 from filecoin-project/shaodan-miner-sectors-info
Shaodan miner sectors info
2020-12-10 20:09:15 +01:00
Łukasz Magiera
0cb2f51549
Merge pull request #5094 from filecoin-project/asr/verified-only
Allow miners to filter (un)verified deals
2020-12-10 19:48:22 +01:00
Łukasz Magiera
d347e7ab8f Merge branch 'miner-sectors-info' of github.com:shaodan/lotus into shaodan-miner-sectors-info 2020-12-10 19:32:26 +01:00
Dan Shao
d56170d70e Optimize sectors info loading 2020-12-07 13:42:37 +08:00
Ingar Shu
13c8a235b6
Use FundManager to withdraw funds, add MarketWithdraw to API 2020-12-03 13:08:18 -08:00
Łukasz Magiera
783971d2c8 docsgen, gofmt 2020-12-02 20:46:40 +01:00
Łukasz Magiera
f74a1b70ab miner: Fix actor control list cli 2020-12-02 20:46:07 +01:00
Aayush Rajasekaran
370817eb60 Allow miners to filter (un)verified deals 2020-12-02 01:32:34 -05:00
Łukasz Magiera
c6f85886bf miner: Add slow mode to proving check 2020-12-02 00:32:14 +01:00
Łukasz Magiera
614f45dcfb
Merge pull request #5041 from filecoin-project/feat/5013
Sector check command
2020-12-01 14:59:01 +01:00
Łukasz Magiera
a4f3758f4c worker api: better grouping 2020-11-30 23:16:30 +01:00
Łukasz Magiera
a6658a858f Fix tests 2020-11-26 19:59:15 +01:00
Łukasz Magiera
3672053ae9 worker: Support setting task types at runtime 2020-11-26 17:33:34 +01:00
s1m0n21
663b3f4f21 add a sector check command 2020-11-26 15:02:43 +08:00
Dirk McCormick
239e180683 feat: markets - separate watching for pre-commit from prove-commit 2020-11-24 13:24:41 -08:00
Łukasz Magiera
94763c2aaa
Merge pull request #4849 from filecoin-project/feat/conngater
Connection Gater support
2020-11-20 16:25:02 +01:00
Łukasz Magiera
b8853aa4d5 Add error codes to worker return 2020-11-17 16:17:55 +01:00
Łukasz Magiera
6bea9dd178 Making sealing logic work with multiple seal proof types 2020-11-16 19:03:30 +01:00
vyzo
91e2530e11 remove dtypes dependency from api; move NetBlockList type to api/types.go 2020-11-13 21:11:17 +02:00
vyzo
5a0b9f4197 connectiong gater API scaffolding 2020-11-13 12:17:48 +02:00
Łukasz Magiera
8ac495723e gofmt 2020-11-11 17:42:23 +01:00
Łukasz Magiera
09f9f871a3 Create a command to abort sealing calls 2020-11-11 17:39:12 +01:00
Dirk McCormick
0d243bb824 refactor: integrate new FundManager 2020-11-10 21:11:05 -08:00
Łukasz Magiera
a5c05f87f1
Merge pull request #4105 from filecoin-project/asr/decode-params
Add a StateDecodeParams method
2020-10-31 01:05:50 +01:00
Łukasz Magiera
6fcdf3c7ed
Merge pull request #4615 from filecoin-project/feat/worker-pause
worker: Commands to pause/resume task processing
2020-10-31 00:26:37 +01:00
Łukasz Magiera
7fbb868513 Debug flag to force running sealing scheduler 2020-10-30 11:07:35 +01:00
Aayush Rajasekaran
5ff42505e8 Add a StateDecodeParams method 2020-10-29 22:04:33 -04:00
Łukasz Magiera
553408d573 worker: Commands to pause/resume task processing 2020-10-28 22:10:17 +01:00
Łukasz Magiera
413643a712 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-27 16:56:29 +01:00
Łukasz Magiera
8c60069bb8
Merge pull request #4535 from filecoin-project/feat/clientdeal-cache-commd
client deal: Cache CommD when creating multiple deals
2020-10-27 03:58:25 +01:00
Łukasz Magiera
e1da874258 Merge remote-tracking branch 'origin/master' into feat/async-restartable-workers 2020-10-27 03:31:07 +01:00
Łukasz Magiera
a37c372d06
Merge pull request #4572 from filecoin-project/feat/data-transfer-utils
Flesh out data transfer features
2020-10-26 13:29:26 +01:00