Commit Graph

1008 Commits

Author SHA1 Message Date
Anton Evangelatov
139ff4ae64 resolved conflicts 2021-03-30 16:15:42 +03:00
Anton Evangelatov
e79927a77b use go-fil-markets v1.2.4 2021-03-30 15:32:44 +03:00
Anton Evangelatov
a43f4b33d9 upgrade go-data-transfer 2021-03-29 19:25:35 +03:00
Anton Evangelatov
14e6ce843e use latest go-fil-markets and go-data-transfer 2021-03-29 18:31:10 +03:00
Łukasz Magiera
2c18921bce update go-commp-utils for >10x faster client commp calculation 2021-03-27 10:12:45 +01:00
Anton Evangelatov
b4d4766615 use go-data-transfer from personal fork 2021-03-23 14:30:26 +02:00
Anton Evangelatov
22217b7cae Merge branch 'master' into nonsense/cli-show-deals 2021-03-23 13:58:26 +02:00
Łukasz Magiera
43d9cc36a4
OpenRPC Support (#5843)
* main: init implement rpc.Discover RPC method

This implement the basic functionality for the method
over HTTP RPC.

Signed-off-by: meows <b5c6@protonmail.com>

* main,go.mod,go.sum: init example with go-openrpc-reflect lib

Signed-off-by: meows <b5c6@protonmail.com>

 Conflicts:
	go.mod
	go.sum

* main: make variable name human-friendly

Signed-off-by: meows <b5c6@protonmail.com>

* main,go.mod,go.sum: init impl of go-openrp-reflect printing document

Signed-off-by: meows <b5c6@protonmail.com>

 Conflicts:
	go.mod
	go.sum

* go.mod,go.sum: use go-openrpc-reflect and open-rpc/meta-schema hackforks

This is for development only.
Versions need to be bumped when they're ready for use
as canonical remotes.

Signed-off-by: meows <b5c6@protonmail.com>

* main,openrpc,main: refactor openrpc supporting code to own package

This eliminates code duplication.

Signed-off-by: meows <b5c6@protonmail.com>

* main: add rpc.Discover to openrpc document

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: fix rpc.discover method name casing

Also fixes casing stuff for the rest of Filecoin.
methods.

Signed-off-by: meows <b5c6@protonmail.com>

* Revert "main: add rpc.Discover to openrpc document"

This reverts commit 116898efb10f33e405ac74acb1aa6daefcd46a62.

* main: fix document creation method name

This fixes an issue caused with the latest reverting
commit.

Signed-off-by: meows <b5c6@protonmail.com>

* main,docgen,openrpc: refactor to share api parsing, etc as docgen exported stuff

Signed-off-by: meows <b5c6@protonmail.com>

Makefile: fix docgen refactoring for makefile use of command

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: add schema.examples to app reflector

There are quite of few of these already registered
for the docgen command, so it makes sense to use
those!

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: init method pairing examples

Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: bump go.mod to use latest meta-schema and openrpc-reflect versions

Signed-off-by: meows <b5c6@protonmail.com>

* openrpc: init SchemaType mapper function

This function will handle the manual configurations
for app-specific data types w/r/t their json schema
representation.

This is useful for cases where the reflect library
is unable to provide a sufficient representation
automatically.

Provided in this commit is an initial implementation
for the integerD type (assuming number are represented
in the API as hexs), and a commonly used cid.Cid type.

Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: tame dependencies by bumping etclabscore/go-openrpc-reflect

This removes a problematic dependency
on github.com/ethereum/go-ethereum, which was
imported as a dependency for a couple github.com/etclabscore/go-openrpc-reflect
tests.

etclabscore/go-openrpc-reflect v0.0.36 has removed this
dependency, so this commit is the result of bumping
that version and then running 'go mod tidy'

This is in response to a review at
https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205

Date: 2020-11-21 06:52:48-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main: add 'miner' arg to openrpc gen cmd

This allows the command to EITHER
generate the doc for Full or Miner APIs.

See comment for usage.

Date: 2020-11-21 07:48:05-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgen: add missing examples for Miner API

Generating the Miner API OpenRPC doc
(via 'go run ./api/openrpc/cmd miner') caused
the example logic to panic because some types
were missing.

This commit adds those missing types, although
I'm not an expert in the API so I can't
suggest that the example values provided are
ideal or well representative.

Date: 2020-11-21 07:50:21-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build/openrpc/full.json,build/openrpc/miner.json: add build/openrpc/[full/miner].json docs

These will be used as static documents
provided by the rpc.discover method.

Date: 2020-11-21 07:51:39-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: init go-rice openrpc static assets

Date: 2020-11-21 08:23:06-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main: remove rpc.discover implementation from runtime plugin

Instead of generating the doc on the fly,
we're going to serve a static asset.
Rel https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205
This removes the runtime implementation from the
RPC server construction.

Date: 2020-11-21 08:41:20-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api,apistruct,common: add Discover(ctx) method to CommonAPI interface and structs

Date: 2020-11-21 08:41:56-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main: use rpc server method aliasing for rpc.discover

This depends on a currently-forked change at
filecoin-project/go-jsonrpc 8350f9463ee451b187d35c492e32f1b999e80210
which establishes this new method RPCServer.AliasMethod.

This solves the problem that the OpenRPC
spec says that the document should be served
at the system extension-prefixed endpoing
rpc.discover (not Filecoin.Discover).

In fact, the document will be available at BOTH
endpoints, but that duplicity is harmless.

Date: 2020-11-21 09:18:26-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api,apistruct,build,common: rpc.discover: return json object instead of string

Instead of casting the JSON asset from bytes to string,
unmarshal it to a map[string]interface{} so the
server will provide it as a JSON object.

Date: 2020-11-21 09:27:11-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* Makefile: merge resolve: docsgen command path

Date: 2020-11-22 07:19:36-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* apistruct,main,docgen,openrpc: merge resolve: fix func exporteds, signatures

Date: 2020-11-22 07:31:03-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: 'get get' auto-bumps version

Date: 2020-11-22 07:31:44-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* Makefile,docgen,main,build/openrpc: refactor openrpc documentation generation

This creates Makefile command docsgen-openrpc-json,
and refactors the docsgen command to generate both
the markdown and openrpc json documents, redirecting
the output of the openrpc json documentation to
the build/openrpc/ directory, where those json
files will be compiled as static assets via go-rice
boxes.

The api/openrpc/cmd now uses usage argumentation
congruent to that of the docgen command (switching
on API context).

Date: 2020-11-22 08:01:18-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main,docgen_openrpc: rename api/openrpc -> api/docgen-openrpc

Renames the package as well.

This is intended to parallel the
existing docgen package and command
namespacing.

Date: 2020-11-22 10:34:46-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api,apistruct,docgen,build,build/openrpc: use typed Discover response

Instead of using a map[string]interface{}, use
a typed response for the Discover method implementation.

This avoids having to set a docgen Example for
the generic map[string]interface{} (as an openrpc document)
which both pollutes the generic type and lacks
useful information for the Discover method example.

Date: 2020-11-22 08:31:16-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* apistruct,build,main,impl: implement Discover method for Worker and StorageMiner APIs

Methods return static compiled assets respective
to the APIs.

Date: 2020-11-22 08:57:18-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgen_openrpc,build/openrpc: remove timestamping from openrpc doc info

This should allow openrpc docs generated at different
times to be equal. This is important because the CI
(Circle) runs the docgen command and tests that
the output and the source are unchanged (via git diff).

Date: 2020-11-22 10:47:07-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* main,docgen_openrpc,main,build: fix lint issues

Fixes goimports, staticcheck, golint issues.

Date: 2020-11-22 11:06:46-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgenopenrpc: fix: don't use an underscore in package name (golint)

Date: 2020-11-22 11:07:53-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.sum: fix: mod-tidy-check (run 'go mod tidy')

Date: 2020-11-22 11:09:48-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: bump filecoin-project/go-jsonrpc dep to latest

This version includes the necessary RPCServer.AliasMethod
method.

Date: 2020-11-23 12:16:15-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* Makefile,main,build,build/openrpc: init gzipped openrpc static docs

Date: 2020-11-24 06:15:06-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: refactor gzip reading

Date: 2020-11-24 06:18:34-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: add basic test for openrpc doc from static assets

Date: 2020-11-24 06:30:23-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build: handle reader Close error

This keeps the errcheck linter happy.

Date: 2020-11-24 06:33:14-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.sum: run 'go mod tidy'

Date: 2020-11-24 06:36:07-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: go mod tidy

Tidying up after resolving the merge conflicts
with master at go.mod

Date: 2020-11-24 06:40:45-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* go.mod,go.sum: bump filecoin-project/go-jsonrpc to latest

This is a repeat of 76e6fd2, since the latest merge
to master seems to have reverted this.

Date: 2020-11-24 06:42:30-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgenopenrpc,build/openrpc: remove method example pairings, improve schema examples

Removing method example pairings since they were
redundant to schema examples and were not
implemented well.

Improved schema examples by using the ExampleValue
method instead of the map lookup.
Made a note in the comment here that this is
not ideal, since we have to make a shortcut assumption
/workaround by using 'unknown' as the method name
and the typea as its own parent.

Luckily these values aren't heavily used by the
method logic.

Date: 2020-11-27 12:57:36-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* docgenopenrpc: use generic number jsonschema for number types

Previously used an integer schema assuming
hex encoding. It appears, based on review some
of the examples, that this may not be the case.

Obvioussly this schema could be more descriptive,
but just shooting for mostly likely to be
not wrong at this point.

Date: 2020-12-15 14:44:37-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* cmd/lotus,go.mod,go.sum: maybe fix straggling merge resolution conflicts

Date: 2021-01-19 12:30:42-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* build/openrpc/full.json.gz,build/openrpc/miner.json.gz,build/openrpc/worker.json.gz: run 'make docsgen'

Date: 2021-01-19 12:33:55-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/apistruct,node/impl: (lint) gofmt

Date: 2021-01-19 12:39:48-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/docgen: maybe fix parse error:  open ./api: no such file or directory

Date: 2021-01-19 12:52:04-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/docgen,build/openrpc: maybe fix no such file error and run 'make docsgen'

Date: 2021-01-19 12:55:52-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api/docgen: return if AST comment/groupdoc parsing encounters any error

This will returns empty comments/docs maps.
This should fix issues like:
https://app.circleci.com/pipelines/github/filecoin-project/lotus/12445/workflows/4ebadce9-a298-4ad1-939b-f19ef4c0a5bf/jobs/107218

where the environment makes file lookups hard or
impossible.

Date: 2021-01-19 13:04:58-06:00
Signed-off-by: meows <b5c6@protonmail.com>

* api: Don't depend on build/

* make: support parallel docsgen

* openrpc gen: Use simple build version

* methodgen

* goimports

Co-authored-by: meows <b5c6@protonmail.com>
2021-03-19 19:22:46 +01:00
Anton Evangelatov
86772a09c7 use new channelmonitor.Config 2021-03-19 13:53:03 +02:00
Anton Evangelatov
9d6c77d93f remove interactive UX; add inspect-deal cmd 2021-03-18 14:53:03 +02:00
Łukasz Magiera
8546e56d38 update go-statestore to avoid redundant log writes 2021-03-11 13:34:35 +01:00
Łukasz Magiera
504affd266
Merge pull request #5757 from filecoin-project/chore/sr2_stats_factored_out
Remove the SR2 stats, leave just the network totals
2021-03-10 14:15:35 +01:00
Peter Rabbitson
bdafdf8060 Remove the SR2 stats, leave just the network totals
The SR2 aggregator now lives at https://github.com/filecoin-project/slingshot-stats
2021-03-09 23:36:03 +00:00
Łukasz Magiera
c49c6fd514
Merge pull request #5746 from filecoin-project/chore/update-graphsync
chore: update go-graphsync to 0.6.0
2021-03-08 22:43:37 +01:00
Łukasz Magiera
6591af960f
Merge pull request #4992 from filecoin-project/feat/splitstore
hot/cold blockstore segregation (aka. splitstore)
2021-03-08 22:01:06 +01:00
Steven Allen
3a2e9887bd chore: update go-graphsync to 0.6.0
This fixes a bunch of bugs and should improve deal reliability.
2021-03-08 12:31:31 -08:00
vyzo
667fffb302 go get go-libp2p-pubsub@master 2021-03-06 19:54:25 +02:00
Raúl Kripalani
1b51c10d78 split off lmdb support to a different branch. 2021-03-05 14:46:18 +02:00
vyzo
cb1789ea6e gomod: use bolt 2021-03-05 14:46:17 +02:00
vyzo
723e48b399 gomod:update go-bs-lmdb to v1.0.3 2021-03-05 14:46:17 +02:00
vyzo
5872f246ff go get go-bs-lmdb@v1.0.2 2021-03-05 14:46:17 +02:00
Raúl Kripalani
877ecab960 update go-bs-lmdb and migrate to ledgerwatch/lmdb-go. 2021-03-05 14:46:17 +02:00
vyzo
3f8da19a34 go get go-bs-lmdb@v0.0.3 2021-03-05 14:46:17 +02:00
vyzo
e87ce6cb60 go get go-bs-lmdb 2021-03-05 14:46:17 +02:00
vyzo
df856b7315 gomod: get lmdb-go 2021-03-05 14:46:17 +02:00
Dirk McCormick
84803f82ab feat: show deals CLI command 2021-03-04 16:56:49 +01:00
Łukasz Magiera
448813d2fe
Merge pull request #5695 from filecoin-project/feat/segregate-blockstores
segregate chain and state blockstores
2021-03-03 10:53:46 +01:00
Raúl Kripalani
3795cc2bd2 segregate chain and state blockstores.
This paves the way for better object lifetime management.

Concretely, it makes it possible to:
- have different stores backing chain and state data.
- having the same datastore library, but using different parameters.
- attach different caching layers/policies to each class of data, e.g.
  sizing caches differently.
- specifying different retention policies for chain and state data.

This separation is important because:
- access patterns/frequency of chain and state data are different.
- state is derivable from chain, so one could never expunge the chain
  store, and only retain state objects reachable from the last finality
  in the state store.
2021-02-28 22:49:44 +00:00
Jakub Sztandera
303a0fec87
Revert "Revert "Refactor send command for better testability"" 2021-02-23 15:50:47 +01:00
Jakub Sztandera
bebc11522e
Revert "Refactor send command for better testability" 2021-02-23 15:25:19 +01:00
Jakub Sztandera
7721ea20ba
Merge pull request #5432 from filecoin-project/refac/send
Refactor send command for better testability
2021-02-23 13:18:11 +01:00
Łukasz Magiera
35759fa07e
Merge pull request #5635 from filecoin-project/deps/cbg-soft-map
Update markets, cbor-gen with soft map decoding
2021-02-22 18:27:38 +01:00
Dirk McCormick
a8b40b77fa update to go-fil-markets v1.1.9 2021-02-22 17:01:28 +01:00
Dirk McCormick
7046e2721a feat: update go-fil-markets version 2021-02-22 14:25:29 +01:00
Steven Allen
b177e67a7a chore: normalize deps 2021-02-19 15:27:38 -08:00
Aayush Rajasekaran
df32f5d060 Update actors to v3.0.2 2021-02-19 15:08:30 -05:00
Łukasz Magiera
26399dba70 Update markets, cbor-gen with soft map decoding 2021-02-19 20:11:43 +01:00
Jakub Sztandera
bad67acb4b
Generate mocks, integrate send service test
Signed-off-by: Jakub Sztandera <kubuxu@protocol.ai>
2021-02-18 15:21:31 +01:00
Łukasz Magiera
d12545af6f
Merge pull request #5469 from filecoin-project/feat/faucet-captcha
Add spam protection to fountain
2021-02-17 20:17:28 +01:00
Łukasz Magiera
5f998038c7 Update go-jsonrpc 2021-02-17 18:59:31 +01:00
Łukasz Magiera
5ad4d72fe0 Update go-jsonrpc fixing websocket reconnecting 2021-02-16 21:16:13 +01:00
Dirk McCormick
a6e3856776 Revert "feat: deals - show data transfer %"
This reverts commit b6c9ddccff.
2021-02-16 12:32:45 +01:00
Steven Allen
6b6f0acaf0 update go-bitfield
optimizes subtract operations
2021-02-15 13:14:13 -08:00
Łukasz Magiera
95e47cf998
Merge pull request #5553 from filecoin-project/feat/data-xfer-percent
show data transfer % for storage deals
2021-02-11 13:29:43 +01:00
Dirk McCormick
b6c9ddccff feat: deals - show data transfer % 2021-02-10 09:56:23 +01:00
Anton Evangelatov
307fc14093 go.mod: do not depend on gocheck 2021-02-09 17:08:19 +01:00
Dirk McCormick
e7a63b7dd5 feat: updated to go-fil-markets@1.1.7 2021-02-05 12:12:48 +01:00
Dirk McCormick
6baddccff2 feat: update to go-fil-markets v1.1.6 2021-02-05 09:41:42 +01:00
Dirk McCormick
fc8ee481a8 refactor: move waiting for publish deals from markets into lotus 2021-02-03 10:36:38 +01:00
Dirk McCormick
a19d6ce8a3 fix: zero price-per-byte retrieval 2021-02-01 16:55:18 +01:00