lotus/go.mod

181 lines
7.9 KiB
Modula-2
Raw Normal View History

module github.com/filecoin-project/lotus
2021-05-28 06:52:19 +00:00
go 1.16
require (
contrib.go.opencensus.io/exporter/prometheus v0.4.0
github.com/BurntSushi/toml v0.4.1
github.com/GeertJohan/go.rice v1.0.2
github.com/Gurpartap/async v0.0.0-20180927173644-4f7f499dd9ee
github.com/Kubuxu/imtui v0.0.0-20210401140320-41663d68d0fa
github.com/StackExchange/wmi v1.2.1 // indirect
2020-08-04 17:07:17 +00:00
github.com/acarl005/stripansi v0.0.0-20180116102854-5a71ef0e047d
OpenRPC Support (#5843) * main: init implement rpc.Discover RPC method This implement the basic functionality for the method over HTTP RPC. Signed-off-by: meows <b5c6@protonmail.com> * main,go.mod,go.sum: init example with go-openrpc-reflect lib Signed-off-by: meows <b5c6@protonmail.com> Conflicts: go.mod go.sum * main: make variable name human-friendly Signed-off-by: meows <b5c6@protonmail.com> * main,go.mod,go.sum: init impl of go-openrp-reflect printing document Signed-off-by: meows <b5c6@protonmail.com> Conflicts: go.mod go.sum * go.mod,go.sum: use go-openrpc-reflect and open-rpc/meta-schema hackforks This is for development only. Versions need to be bumped when they're ready for use as canonical remotes. Signed-off-by: meows <b5c6@protonmail.com> * main,openrpc,main: refactor openrpc supporting code to own package This eliminates code duplication. Signed-off-by: meows <b5c6@protonmail.com> * main: add rpc.Discover to openrpc document Signed-off-by: meows <b5c6@protonmail.com> * openrpc: fix rpc.discover method name casing Also fixes casing stuff for the rest of Filecoin. methods. Signed-off-by: meows <b5c6@protonmail.com> * Revert "main: add rpc.Discover to openrpc document" This reverts commit 116898efb10f33e405ac74acb1aa6daefcd46a62. * main: fix document creation method name This fixes an issue caused with the latest reverting commit. Signed-off-by: meows <b5c6@protonmail.com> * main,docgen,openrpc: refactor to share api parsing, etc as docgen exported stuff Signed-off-by: meows <b5c6@protonmail.com> Makefile: fix docgen refactoring for makefile use of command Signed-off-by: meows <b5c6@protonmail.com> * openrpc: add schema.examples to app reflector There are quite of few of these already registered for the docgen command, so it makes sense to use those! Signed-off-by: meows <b5c6@protonmail.com> * openrpc: init method pairing examples Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump go.mod to use latest meta-schema and openrpc-reflect versions Signed-off-by: meows <b5c6@protonmail.com> * openrpc: init SchemaType mapper function This function will handle the manual configurations for app-specific data types w/r/t their json schema representation. This is useful for cases where the reflect library is unable to provide a sufficient representation automatically. Provided in this commit is an initial implementation for the integerD type (assuming number are represented in the API as hexs), and a commonly used cid.Cid type. Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: tame dependencies by bumping etclabscore/go-openrpc-reflect This removes a problematic dependency on github.com/ethereum/go-ethereum, which was imported as a dependency for a couple github.com/etclabscore/go-openrpc-reflect tests. etclabscore/go-openrpc-reflect v0.0.36 has removed this dependency, so this commit is the result of bumping that version and then running 'go mod tidy' This is in response to a review at https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205 Date: 2020-11-21 06:52:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: add 'miner' arg to openrpc gen cmd This allows the command to EITHER generate the doc for Full or Miner APIs. See comment for usage. Date: 2020-11-21 07:48:05-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgen: add missing examples for Miner API Generating the Miner API OpenRPC doc (via 'go run ./api/openrpc/cmd miner') caused the example logic to panic because some types were missing. This commit adds those missing types, although I'm not an expert in the API so I can't suggest that the example values provided are ideal or well representative. Date: 2020-11-21 07:50:21-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build/openrpc/full.json,build/openrpc/miner.json: add build/openrpc/[full/miner].json docs These will be used as static documents provided by the rpc.discover method. Date: 2020-11-21 07:51:39-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: init go-rice openrpc static assets Date: 2020-11-21 08:23:06-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: remove rpc.discover implementation from runtime plugin Instead of generating the doc on the fly, we're going to serve a static asset. Rel https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205 This removes the runtime implementation from the RPC server construction. Date: 2020-11-21 08:41:20-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,common: add Discover(ctx) method to CommonAPI interface and structs Date: 2020-11-21 08:41:56-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: use rpc server method aliasing for rpc.discover This depends on a currently-forked change at filecoin-project/go-jsonrpc 8350f9463ee451b187d35c492e32f1b999e80210 which establishes this new method RPCServer.AliasMethod. This solves the problem that the OpenRPC spec says that the document should be served at the system extension-prefixed endpoing rpc.discover (not Filecoin.Discover). In fact, the document will be available at BOTH endpoints, but that duplicity is harmless. Date: 2020-11-21 09:18:26-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,build,common: rpc.discover: return json object instead of string Instead of casting the JSON asset from bytes to string, unmarshal it to a map[string]interface{} so the server will provide it as a JSON object. Date: 2020-11-21 09:27:11-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile: merge resolve: docsgen command path Date: 2020-11-22 07:19:36-06:00 Signed-off-by: meows <b5c6@protonmail.com> * apistruct,main,docgen,openrpc: merge resolve: fix func exporteds, signatures Date: 2020-11-22 07:31:03-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: 'get get' auto-bumps version Date: 2020-11-22 07:31:44-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile,docgen,main,build/openrpc: refactor openrpc documentation generation This creates Makefile command docsgen-openrpc-json, and refactors the docsgen command to generate both the markdown and openrpc json documents, redirecting the output of the openrpc json documentation to the build/openrpc/ directory, where those json files will be compiled as static assets via go-rice boxes. The api/openrpc/cmd now uses usage argumentation congruent to that of the docgen command (switching on API context). Date: 2020-11-22 08:01:18-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main,docgen_openrpc: rename api/openrpc -> api/docgen-openrpc Renames the package as well. This is intended to parallel the existing docgen package and command namespacing. Date: 2020-11-22 10:34:46-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,docgen,build,build/openrpc: use typed Discover response Instead of using a map[string]interface{}, use a typed response for the Discover method implementation. This avoids having to set a docgen Example for the generic map[string]interface{} (as an openrpc document) which both pollutes the generic type and lacks useful information for the Discover method example. Date: 2020-11-22 08:31:16-06:00 Signed-off-by: meows <b5c6@protonmail.com> * apistruct,build,main,impl: implement Discover method for Worker and StorageMiner APIs Methods return static compiled assets respective to the APIs. Date: 2020-11-22 08:57:18-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgen_openrpc,build/openrpc: remove timestamping from openrpc doc info This should allow openrpc docs generated at different times to be equal. This is important because the CI (Circle) runs the docgen command and tests that the output and the source are unchanged (via git diff). Date: 2020-11-22 10:47:07-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main,docgen_openrpc,main,build: fix lint issues Fixes goimports, staticcheck, golint issues. Date: 2020-11-22 11:06:46-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc: fix: don't use an underscore in package name (golint) Date: 2020-11-22 11:07:53-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.sum: fix: mod-tidy-check (run 'go mod tidy') Date: 2020-11-22 11:09:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump filecoin-project/go-jsonrpc dep to latest This version includes the necessary RPCServer.AliasMethod method. Date: 2020-11-23 12:16:15-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile,main,build,build/openrpc: init gzipped openrpc static docs Date: 2020-11-24 06:15:06-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: refactor gzip reading Date: 2020-11-24 06:18:34-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: add basic test for openrpc doc from static assets Date: 2020-11-24 06:30:23-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: handle reader Close error This keeps the errcheck linter happy. Date: 2020-11-24 06:33:14-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.sum: run 'go mod tidy' Date: 2020-11-24 06:36:07-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: go mod tidy Tidying up after resolving the merge conflicts with master at go.mod Date: 2020-11-24 06:40:45-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump filecoin-project/go-jsonrpc to latest This is a repeat of 76e6fd2, since the latest merge to master seems to have reverted this. Date: 2020-11-24 06:42:30-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc,build/openrpc: remove method example pairings, improve schema examples Removing method example pairings since they were redundant to schema examples and were not implemented well. Improved schema examples by using the ExampleValue method instead of the map lookup. Made a note in the comment here that this is not ideal, since we have to make a shortcut assumption /workaround by using 'unknown' as the method name and the typea as its own parent. Luckily these values aren't heavily used by the method logic. Date: 2020-11-27 12:57:36-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc: use generic number jsonschema for number types Previously used an integer schema assuming hex encoding. It appears, based on review some of the examples, that this may not be the case. Obvioussly this schema could be more descriptive, but just shooting for mostly likely to be not wrong at this point. Date: 2020-12-15 14:44:37-06:00 Signed-off-by: meows <b5c6@protonmail.com> * cmd/lotus,go.mod,go.sum: maybe fix straggling merge resolution conflicts Date: 2021-01-19 12:30:42-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build/openrpc/full.json.gz,build/openrpc/miner.json.gz,build/openrpc/worker.json.gz: run 'make docsgen' Date: 2021-01-19 12:33:55-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/apistruct,node/impl: (lint) gofmt Date: 2021-01-19 12:39:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen: maybe fix parse error: open ./api: no such file or directory Date: 2021-01-19 12:52:04-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen,build/openrpc: maybe fix no such file error and run 'make docsgen' Date: 2021-01-19 12:55:52-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen: return if AST comment/groupdoc parsing encounters any error This will returns empty comments/docs maps. This should fix issues like: https://app.circleci.com/pipelines/github/filecoin-project/lotus/12445/workflows/4ebadce9-a298-4ad1-939b-f19ef4c0a5bf/jobs/107218 where the environment makes file lookups hard or impossible. Date: 2021-01-19 13:04:58-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api: Don't depend on build/ * make: support parallel docsgen * openrpc gen: Use simple build version * methodgen * goimports Co-authored-by: meows <b5c6@protonmail.com>
2021-03-19 18:22:46 +00:00
github.com/alecthomas/jsonschema v0.0.0-20200530073317-71f438968921
github.com/buger/goterm v1.0.3
github.com/chzyer/readline v0.0.0-20180603132655-2972be24d48e
github.com/containerd/cgroups v0.0.0-20201119153540-4cbc285b3327
github.com/coreos/go-systemd/v22 v22.3.2
github.com/detailyang/go-fallocate v0.0.0-20180908115635-432fa640bd2e
github.com/dgraph-io/badger/v2 v2.2007.3
2021-12-13 12:17:05 +00:00
github.com/dgraph-io/ristretto v0.1.0 // indirect
github.com/docker/go-units v0.4.0
2021-12-15 23:34:21 +00:00
github.com/drand/drand v1.3.0
2021-12-13 12:17:05 +00:00
github.com/drand/kyber v1.1.7
github.com/dustin/go-humanize v1.0.0
github.com/elastic/go-sysinfo v1.7.0
github.com/elastic/gosigar v0.14.1
OpenRPC Support (#5843) * main: init implement rpc.Discover RPC method This implement the basic functionality for the method over HTTP RPC. Signed-off-by: meows <b5c6@protonmail.com> * main,go.mod,go.sum: init example with go-openrpc-reflect lib Signed-off-by: meows <b5c6@protonmail.com> Conflicts: go.mod go.sum * main: make variable name human-friendly Signed-off-by: meows <b5c6@protonmail.com> * main,go.mod,go.sum: init impl of go-openrp-reflect printing document Signed-off-by: meows <b5c6@protonmail.com> Conflicts: go.mod go.sum * go.mod,go.sum: use go-openrpc-reflect and open-rpc/meta-schema hackforks This is for development only. Versions need to be bumped when they're ready for use as canonical remotes. Signed-off-by: meows <b5c6@protonmail.com> * main,openrpc,main: refactor openrpc supporting code to own package This eliminates code duplication. Signed-off-by: meows <b5c6@protonmail.com> * main: add rpc.Discover to openrpc document Signed-off-by: meows <b5c6@protonmail.com> * openrpc: fix rpc.discover method name casing Also fixes casing stuff for the rest of Filecoin. methods. Signed-off-by: meows <b5c6@protonmail.com> * Revert "main: add rpc.Discover to openrpc document" This reverts commit 116898efb10f33e405ac74acb1aa6daefcd46a62. * main: fix document creation method name This fixes an issue caused with the latest reverting commit. Signed-off-by: meows <b5c6@protonmail.com> * main,docgen,openrpc: refactor to share api parsing, etc as docgen exported stuff Signed-off-by: meows <b5c6@protonmail.com> Makefile: fix docgen refactoring for makefile use of command Signed-off-by: meows <b5c6@protonmail.com> * openrpc: add schema.examples to app reflector There are quite of few of these already registered for the docgen command, so it makes sense to use those! Signed-off-by: meows <b5c6@protonmail.com> * openrpc: init method pairing examples Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump go.mod to use latest meta-schema and openrpc-reflect versions Signed-off-by: meows <b5c6@protonmail.com> * openrpc: init SchemaType mapper function This function will handle the manual configurations for app-specific data types w/r/t their json schema representation. This is useful for cases where the reflect library is unable to provide a sufficient representation automatically. Provided in this commit is an initial implementation for the integerD type (assuming number are represented in the API as hexs), and a commonly used cid.Cid type. Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: tame dependencies by bumping etclabscore/go-openrpc-reflect This removes a problematic dependency on github.com/ethereum/go-ethereum, which was imported as a dependency for a couple github.com/etclabscore/go-openrpc-reflect tests. etclabscore/go-openrpc-reflect v0.0.36 has removed this dependency, so this commit is the result of bumping that version and then running 'go mod tidy' This is in response to a review at https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205 Date: 2020-11-21 06:52:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: add 'miner' arg to openrpc gen cmd This allows the command to EITHER generate the doc for Full or Miner APIs. See comment for usage. Date: 2020-11-21 07:48:05-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgen: add missing examples for Miner API Generating the Miner API OpenRPC doc (via 'go run ./api/openrpc/cmd miner') caused the example logic to panic because some types were missing. This commit adds those missing types, although I'm not an expert in the API so I can't suggest that the example values provided are ideal or well representative. Date: 2020-11-21 07:50:21-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build/openrpc/full.json,build/openrpc/miner.json: add build/openrpc/[full/miner].json docs These will be used as static documents provided by the rpc.discover method. Date: 2020-11-21 07:51:39-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: init go-rice openrpc static assets Date: 2020-11-21 08:23:06-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: remove rpc.discover implementation from runtime plugin Instead of generating the doc on the fly, we're going to serve a static asset. Rel https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205 This removes the runtime implementation from the RPC server construction. Date: 2020-11-21 08:41:20-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,common: add Discover(ctx) method to CommonAPI interface and structs Date: 2020-11-21 08:41:56-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: use rpc server method aliasing for rpc.discover This depends on a currently-forked change at filecoin-project/go-jsonrpc 8350f9463ee451b187d35c492e32f1b999e80210 which establishes this new method RPCServer.AliasMethod. This solves the problem that the OpenRPC spec says that the document should be served at the system extension-prefixed endpoing rpc.discover (not Filecoin.Discover). In fact, the document will be available at BOTH endpoints, but that duplicity is harmless. Date: 2020-11-21 09:18:26-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,build,common: rpc.discover: return json object instead of string Instead of casting the JSON asset from bytes to string, unmarshal it to a map[string]interface{} so the server will provide it as a JSON object. Date: 2020-11-21 09:27:11-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile: merge resolve: docsgen command path Date: 2020-11-22 07:19:36-06:00 Signed-off-by: meows <b5c6@protonmail.com> * apistruct,main,docgen,openrpc: merge resolve: fix func exporteds, signatures Date: 2020-11-22 07:31:03-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: 'get get' auto-bumps version Date: 2020-11-22 07:31:44-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile,docgen,main,build/openrpc: refactor openrpc documentation generation This creates Makefile command docsgen-openrpc-json, and refactors the docsgen command to generate both the markdown and openrpc json documents, redirecting the output of the openrpc json documentation to the build/openrpc/ directory, where those json files will be compiled as static assets via go-rice boxes. The api/openrpc/cmd now uses usage argumentation congruent to that of the docgen command (switching on API context). Date: 2020-11-22 08:01:18-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main,docgen_openrpc: rename api/openrpc -> api/docgen-openrpc Renames the package as well. This is intended to parallel the existing docgen package and command namespacing. Date: 2020-11-22 10:34:46-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,docgen,build,build/openrpc: use typed Discover response Instead of using a map[string]interface{}, use a typed response for the Discover method implementation. This avoids having to set a docgen Example for the generic map[string]interface{} (as an openrpc document) which both pollutes the generic type and lacks useful information for the Discover method example. Date: 2020-11-22 08:31:16-06:00 Signed-off-by: meows <b5c6@protonmail.com> * apistruct,build,main,impl: implement Discover method for Worker and StorageMiner APIs Methods return static compiled assets respective to the APIs. Date: 2020-11-22 08:57:18-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgen_openrpc,build/openrpc: remove timestamping from openrpc doc info This should allow openrpc docs generated at different times to be equal. This is important because the CI (Circle) runs the docgen command and tests that the output and the source are unchanged (via git diff). Date: 2020-11-22 10:47:07-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main,docgen_openrpc,main,build: fix lint issues Fixes goimports, staticcheck, golint issues. Date: 2020-11-22 11:06:46-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc: fix: don't use an underscore in package name (golint) Date: 2020-11-22 11:07:53-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.sum: fix: mod-tidy-check (run 'go mod tidy') Date: 2020-11-22 11:09:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump filecoin-project/go-jsonrpc dep to latest This version includes the necessary RPCServer.AliasMethod method. Date: 2020-11-23 12:16:15-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile,main,build,build/openrpc: init gzipped openrpc static docs Date: 2020-11-24 06:15:06-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: refactor gzip reading Date: 2020-11-24 06:18:34-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: add basic test for openrpc doc from static assets Date: 2020-11-24 06:30:23-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: handle reader Close error This keeps the errcheck linter happy. Date: 2020-11-24 06:33:14-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.sum: run 'go mod tidy' Date: 2020-11-24 06:36:07-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: go mod tidy Tidying up after resolving the merge conflicts with master at go.mod Date: 2020-11-24 06:40:45-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump filecoin-project/go-jsonrpc to latest This is a repeat of 76e6fd2, since the latest merge to master seems to have reverted this. Date: 2020-11-24 06:42:30-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc,build/openrpc: remove method example pairings, improve schema examples Removing method example pairings since they were redundant to schema examples and were not implemented well. Improved schema examples by using the ExampleValue method instead of the map lookup. Made a note in the comment here that this is not ideal, since we have to make a shortcut assumption /workaround by using 'unknown' as the method name and the typea as its own parent. Luckily these values aren't heavily used by the method logic. Date: 2020-11-27 12:57:36-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc: use generic number jsonschema for number types Previously used an integer schema assuming hex encoding. It appears, based on review some of the examples, that this may not be the case. Obvioussly this schema could be more descriptive, but just shooting for mostly likely to be not wrong at this point. Date: 2020-12-15 14:44:37-06:00 Signed-off-by: meows <b5c6@protonmail.com> * cmd/lotus,go.mod,go.sum: maybe fix straggling merge resolution conflicts Date: 2021-01-19 12:30:42-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build/openrpc/full.json.gz,build/openrpc/miner.json.gz,build/openrpc/worker.json.gz: run 'make docsgen' Date: 2021-01-19 12:33:55-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/apistruct,node/impl: (lint) gofmt Date: 2021-01-19 12:39:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen: maybe fix parse error: open ./api: no such file or directory Date: 2021-01-19 12:52:04-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen,build/openrpc: maybe fix no such file error and run 'make docsgen' Date: 2021-01-19 12:55:52-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen: return if AST comment/groupdoc parsing encounters any error This will returns empty comments/docs maps. This should fix issues like: https://app.circleci.com/pipelines/github/filecoin-project/lotus/12445/workflows/4ebadce9-a298-4ad1-939b-f19ef4c0a5bf/jobs/107218 where the environment makes file lookups hard or impossible. Date: 2021-01-19 13:04:58-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api: Don't depend on build/ * make: support parallel docsgen * openrpc gen: Use simple build version * methodgen * goimports Co-authored-by: meows <b5c6@protonmail.com>
2021-03-19 18:22:46 +00:00
github.com/etclabscore/go-openrpc-reflect v0.0.36
github.com/fatih/color v1.13.0
2021-12-14 22:50:18 +00:00
github.com/filecoin-project/dagstore v0.4.4
github.com/filecoin-project/filecoin-ffi v0.30.4-0.20200910194244-f640612a1a1f
github.com/filecoin-project/go-address v0.0.6
github.com/filecoin-project/go-bitfield v0.2.4
github.com/filecoin-project/go-cbor-util v0.0.1
github.com/filecoin-project/go-commp-utils v0.1.3
github.com/filecoin-project/go-crypto v0.0.1
2022-01-20 09:23:52 +00:00
github.com/filecoin-project/go-data-transfer v1.14.0
This pulls in forgotten parts properly implementing PR#5988 ( previous testing focused exclusively on offline dealflow .cars ) Allows a workflow of: ~$ dd if=/dev/urandom bs=1M count=1 | ~/go-ipfs/cmd/ipfs/ipfs add --pin=false added QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 ~$ ~/go-ipfs/cmd/ipfs/ipfs dag export QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 > test_mib.car ~$ lotus client import --car ~/test_mib.car Import 2, Root QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 ~$ ~/go/bin/stream-commp -p $(( 256 * 1024 * 1024 )) < test_mib.car CommP: 54e4e75ddc3fffa8fd33d3ededc06e564603ac0fe62543ec6463d51b553be40b CommPCid: baga6ea4seaqfjzhhlxod775i7uz5h3pnybxfmrqdvqh6mjkd5rsghvi3ku56icy Raw bytes: 1049073 bytes Unpadded piece: 266338304 bytes Padded piece: 268435456 bytes CARv1 detected in stream: Blocks: 5 Roots: 1 1: QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 ~$ curl http://127.0.0.1:1234/rpc/v0 -X POST -H "Authorization: Bearer $(cat ~/.lotus/token)" -H "Content-Type: application/json" --data ' { "jsonrpc": "2.0", "id":1, "method": "Filecoin.ClientStartDeal", "params": [ { "Wallet":"t01004", "Miner":"t01005", "EpochPrice":"0", "MinBlocksDuration":518400, "Data": { "Root":{ "/":"QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2" }, "PieceCid":{ "/":"baga6ea4seaqfjzhhlxod775i7uz5h3pnybxfmrqdvqh6mjkd5rsghvi3ku56icy" }, "PieceSize": 266338304 } } ] } ' ~$ ~/go/bin/stream-commp -p $(( 128 * 1024 * 1024 )) < test_mib.car CommP: ed904105399ed346f6b03844abc14710a1748854c2781824d6bd1100e63b1807 CommPCid: baga6ea4seaqo3ecbau4z5u2g62ydqrflyfdrbilurbkme6ayetll2eia4y5rqby Raw bytes: 1049073 bytes Unpadded piece: 133169152 bytes Padded piece: 134217728 bytes CARv1 detected in stream: Blocks: 5 Roots: 1 1: QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2 ~$ curl http://127.0.0.1:1234/rpc/v0 -X POST -H "Authorization: Bearer $(cat ~/.lotus/token)" -H "Content-Type: application/json" --data ' { "jsonrpc": "2.0", "id":1, "method": "Filecoin.ClientStatelessDeal", "params": [ { "Wallet":"t01004", "Miner":"t01005", "EpochPrice":"0", "ProviderCollateral":"0", "MinBlocksDuration":518400, "Data": { "TransferType": "manual", "Root":{ "/":"QmcFLqjyh2kvixuuvxgNUoHy55Rb6N6uuSq4CNfvtPoTJ2" }, "PieceCid":{ "/":"baga6ea4seaqo3ecbau4z5u2g62ydqrflyfdrbilurbkme6ayetll2eia4y5rqby" }, "PieceSize": 133169152 } } ] } ' {"jsonrpc":"2.0","result":{"/":"bafyreianhjvev3w6q5lteap3h7tkxbe2jaobwlsi7vzbcoobjpicg3foqi"},"id":1} ~$ lotus-miner storage-deals import-data bafyreianhjvev3w6q5lteap3h7tkxbe2jaobwlsi7vzbcoobjpicg3foqi ~/test_mib.car
2021-04-07 22:14:14 +00:00
github.com/filecoin-project/go-fil-commcid v0.1.0
github.com/filecoin-project/go-fil-commp-hashhash v0.1.0
2022-01-20 10:46:17 +00:00
github.com/filecoin-project/go-fil-markets v1.19.0
github.com/filecoin-project/go-jsonrpc v0.1.5
github.com/filecoin-project/go-padreader v0.0.1
2022-02-08 17:00:55 +00:00
github.com/filecoin-project/go-paramfetch v0.0.4
2021-12-20 01:44:19 +00:00
github.com/filecoin-project/go-state-types v0.1.3
2021-07-29 14:34:50 +00:00
github.com/filecoin-project/go-statemachine v1.0.1
2021-12-07 22:24:58 +00:00
github.com/filecoin-project/go-statestore v0.2.0
github.com/filecoin-project/go-storedcounter v0.1.0
2021-05-19 01:41:21 +00:00
github.com/filecoin-project/specs-actors v0.9.14
2021-12-14 17:49:18 +00:00
github.com/filecoin-project/specs-actors/v2 v2.3.6
2021-05-19 01:41:21 +00:00
github.com/filecoin-project/specs-actors/v3 v3.1.1
github.com/filecoin-project/specs-actors/v4 v4.0.1
github.com/filecoin-project/specs-actors/v5 v5.0.4
github.com/filecoin-project/specs-actors/v6 v6.0.1
2022-01-10 23:08:21 +00:00
github.com/filecoin-project/specs-actors/v7 v7.0.0-rc1
github.com/filecoin-project/specs-storage v0.1.1-0.20211228030229-6d460d25a0c9
2020-10-15 11:49:23 +00:00
github.com/filecoin-project/test-vectors/schema v0.0.5
github.com/gbrlsnchs/jwt/v3 v3.0.1
github.com/gdamore/tcell/v2 v2.2.0
github.com/go-kit/kit v0.12.0
2021-12-13 12:17:05 +00:00
github.com/golang/glog v1.0.0 // indirect
2021-07-28 12:59:16 +00:00
github.com/golang/mock v1.6.0
2021-09-21 11:10:04 +00:00
github.com/google/uuid v1.3.0
2020-03-27 23:00:21 +00:00
github.com/gorilla/mux v1.7.4
2020-04-21 22:50:04 +00:00
github.com/gorilla/websocket v1.4.2
2020-09-30 11:33:42 +00:00
github.com/hako/durafmt v0.0.0-20200710122514-c0fb7b4da026
github.com/hannahhoward/go-pubsub v0.0.0-20200423002714-8d62886cc36e
2021-09-21 11:10:04 +00:00
github.com/hashicorp/go-multierror v1.1.1
2020-02-17 05:51:18 +00:00
github.com/hashicorp/golang-lru v0.5.4
github.com/icza/backscanner v0.0.0-20210726202459-ac2ffc679f94
github.com/influxdata/influxdb1-client v0.0.0-20200827194710-b269163b24ab
github.com/ipfs/bbloom v0.0.4
github.com/ipfs/go-bitswap v0.5.1
2021-02-19 23:27:38 +00:00
github.com/ipfs/go-block-format v0.0.3
github.com/ipfs/go-blockservice v0.2.1
2021-08-18 10:13:11 +00:00
github.com/ipfs/go-cid v0.1.0
github.com/ipfs/go-cidutil v0.0.2
2021-12-07 22:24:58 +00:00
github.com/ipfs/go-datastore v0.5.1
2021-12-14 21:33:38 +00:00
github.com/ipfs/go-ds-badger2 v0.1.2
github.com/ipfs/go-ds-leveldb v0.5.0
2021-12-11 21:03:00 +00:00
github.com/ipfs/go-ds-measure v0.2.0
github.com/ipfs/go-fs-lock v0.0.6
github.com/ipfs/go-graphsync v0.12.0
2021-12-10 03:46:30 +00:00
github.com/ipfs/go-ipfs-blockstore v1.1.2
integrate DAG store and CARv2 in deal-making (#6671) This commit removes badger from the deal-making processes, and moves to a new architecture with the dagstore as the cental component on the miner-side, and CARv2s on the client-side. Every deal that has been handed off to the sealing subsystem becomes a shard in the dagstore. Shards are mounted via the LotusMount, which teaches the dagstore how to load the related piece when serving retrievals. When the miner starts the Lotus for the first time with this patch, we will perform a one-time migration of all active deals into the dagstore. This is a lightweight process, and it consists simply of registering the shards in the dagstore. Shards are backed by the unsealed copy of the piece. This is currently a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so when it's time to acquire a shard to serve a retrieval, the unsealed CARv1 is joined with its index (safeguarded by the dagstore), to form a read-only blockstore, thus taking the place of the monolithic badger. Data transfers have been adjusted to interface directly with CARv2 files. On inbound transfers (client retrievals, miner storage deals), we stream the received data into a CARv2 ReadWrite blockstore. On outbound transfers (client storage deals, miner retrievals), we serve the data off a CARv2 ReadOnly blockstore. Client-side imports are managed by the refactored *imports.Manager component (when not using IPFS integration). Just like it before, we use the go-filestore library to avoid duplicating the data from the original file in the resulting UnixFS DAG (concretely the leaves). However, the target of those imports are what we call "ref-CARv2s": CARv2 files placed under the `$LOTUS_PATH/imports` directory, containing the intermediate nodes in full, and the leaves as positional references to the original file on disk. Client-side retrievals are placed into CARv2 files in the location: `$LOTUS_PATH/retrievals`. A new set of `Dagstore*` JSON-RPC operations and `lotus-miner dagstore` subcommands have been introduced on the miner-side to inspect and manage the dagstore. Despite moving to a CARv2-backed system, the IPFS integration has been respected, and it continues to be possible to make storage deals with data held in an IPFS node, and to perform retrievals directly into an IPFS node. NOTE: because the "staging" and "client" Badger blockstores are no longer used, existing imports on the client will be rendered useless. On startup, Lotus will enumerate all imports and print WARN statements on the log for each import that needs to be reimported. These log lines contain these messages: - import lacks carv2 path; import will not work; please reimport - import has missing/broken carv2; please reimport At the end, we will print a "sanity check completed" message indicating the count of imports found, and how many were deemed broken. Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com> Co-authored-by: Raúl Kripalani <raul@protocol.ai> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
2021-08-16 22:34:32 +00:00
github.com/ipfs/go-ipfs-blocksutil v0.0.1
github.com/ipfs/go-ipfs-chunker v0.0.5
github.com/ipfs/go-ipfs-ds-help v1.1.0
github.com/ipfs/go-ipfs-exchange-interface v0.1.0
github.com/ipfs/go-ipfs-exchange-offline v0.1.1
github.com/ipfs/go-ipfs-files v0.0.9
2021-09-22 06:44:42 +00:00
github.com/ipfs/go-ipfs-http-client v0.0.6
github.com/ipfs/go-ipfs-routing v0.2.1
2020-11-01 13:05:07 +00:00
github.com/ipfs/go-ipfs-util v0.0.2
2021-12-14 22:53:11 +00:00
github.com/ipfs/go-ipld-cbor v0.0.6
github.com/ipfs/go-ipld-format v0.2.0
2021-12-10 02:01:20 +00:00
github.com/ipfs/go-ipld-legacy v0.1.1 // indirect
2022-01-18 12:19:30 +00:00
github.com/ipfs/go-log/v2 v2.5.0
github.com/ipfs/go-merkledag v0.5.1
github.com/ipfs/go-metrics-interface v0.0.1
github.com/ipfs/go-metrics-prometheus v0.0.2
github.com/ipfs/go-path v0.0.7
github.com/ipfs/go-unixfs v0.3.1
2021-09-21 11:10:04 +00:00
github.com/ipfs/interface-go-ipfs-core v0.4.0
2021-12-14 23:06:28 +00:00
github.com/ipld/go-car v0.3.3
2021-12-14 21:54:22 +00:00
github.com/ipld/go-car/v2 v2.1.1
github.com/ipld/go-codec-dagpb v1.3.0
github.com/ipld/go-ipld-prime v0.14.4
github.com/ipld/go-ipld-selector-text-lite v0.0.1
2021-12-13 12:17:05 +00:00
github.com/jonboulle/clockwork v0.2.2 // indirect
github.com/kelseyhightower/envconfig v1.4.0
2020-11-01 13:05:07 +00:00
github.com/libp2p/go-buffer-pool v0.0.2
github.com/libp2p/go-eventbus v0.2.1
2022-02-08 09:51:14 +00:00
github.com/libp2p/go-libp2p v0.18.0-rc4
2022-01-18 12:19:30 +00:00
github.com/libp2p/go-libp2p-connmgr v0.3.1 // indirect
github.com/libp2p/go-libp2p-core v0.14.0
2021-12-07 22:24:58 +00:00
github.com/libp2p/go-libp2p-discovery v0.6.0
2021-12-11 21:03:00 +00:00
github.com/libp2p/go-libp2p-kad-dht v0.15.0
2021-12-07 22:24:58 +00:00
github.com/libp2p/go-libp2p-noise v0.3.0
2021-12-14 13:26:15 +00:00
github.com/libp2p/go-libp2p-peerstore v0.6.0
2022-01-18 14:57:59 +00:00
github.com/libp2p/go-libp2p-pubsub v0.6.1
2022-02-08 09:51:14 +00:00
github.com/libp2p/go-libp2p-quic-transport v0.16.1
2020-07-30 17:36:24 +00:00
github.com/libp2p/go-libp2p-record v0.1.3
2022-02-03 17:32:30 +00:00
github.com/libp2p/go-libp2p-resource-manager v0.1.3
2020-06-04 19:32:11 +00:00
github.com/libp2p/go-libp2p-routing-helpers v0.2.3
2022-01-21 08:45:45 +00:00
github.com/libp2p/go-libp2p-swarm v0.10.1
2021-12-07 22:24:58 +00:00
github.com/libp2p/go-libp2p-tls v0.3.1
2022-02-08 09:51:14 +00:00
github.com/libp2p/go-libp2p-yamux v0.8.2
2020-06-04 19:35:33 +00:00
github.com/libp2p/go-maddr-filter v0.1.0
github.com/mattn/go-isatty v0.0.14
2019-07-05 14:36:08 +00:00
github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1
2019-07-10 17:09:57 +00:00
github.com/mitchellh/go-homedir v1.1.0
github.com/multiformats/go-base32 v0.0.4
2022-01-18 12:19:30 +00:00
github.com/multiformats/go-multiaddr v0.5.0
github.com/multiformats/go-multiaddr-dns v0.3.1
2020-06-22 23:25:41 +00:00
github.com/multiformats/go-multibase v0.0.3
2021-12-10 03:46:30 +00:00
github.com/multiformats/go-multihash v0.1.0
integrate DAG store and CARv2 in deal-making (#6671) This commit removes badger from the deal-making processes, and moves to a new architecture with the dagstore as the cental component on the miner-side, and CARv2s on the client-side. Every deal that has been handed off to the sealing subsystem becomes a shard in the dagstore. Shards are mounted via the LotusMount, which teaches the dagstore how to load the related piece when serving retrievals. When the miner starts the Lotus for the first time with this patch, we will perform a one-time migration of all active deals into the dagstore. This is a lightweight process, and it consists simply of registering the shards in the dagstore. Shards are backed by the unsealed copy of the piece. This is currently a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so when it's time to acquire a shard to serve a retrieval, the unsealed CARv1 is joined with its index (safeguarded by the dagstore), to form a read-only blockstore, thus taking the place of the monolithic badger. Data transfers have been adjusted to interface directly with CARv2 files. On inbound transfers (client retrievals, miner storage deals), we stream the received data into a CARv2 ReadWrite blockstore. On outbound transfers (client storage deals, miner retrievals), we serve the data off a CARv2 ReadOnly blockstore. Client-side imports are managed by the refactored *imports.Manager component (when not using IPFS integration). Just like it before, we use the go-filestore library to avoid duplicating the data from the original file in the resulting UnixFS DAG (concretely the leaves). However, the target of those imports are what we call "ref-CARv2s": CARv2 files placed under the `$LOTUS_PATH/imports` directory, containing the intermediate nodes in full, and the leaves as positional references to the original file on disk. Client-side retrievals are placed into CARv2 files in the location: `$LOTUS_PATH/retrievals`. A new set of `Dagstore*` JSON-RPC operations and `lotus-miner dagstore` subcommands have been introduced on the miner-side to inspect and manage the dagstore. Despite moving to a CARv2-backed system, the IPFS integration has been respected, and it continues to be possible to make storage deals with data held in an IPFS node, and to perform retrievals directly into an IPFS node. NOTE: because the "staging" and "client" Badger blockstores are no longer used, existing imports on the client will be rendered useless. On startup, Lotus will enumerate all imports and print WARN statements on the log for each import that needs to be reimported. These log lines contain these messages: - import lacks carv2 path; import will not work; please reimport - import has missing/broken carv2; please reimport At the end, we will print a "sanity check completed" message indicating the count of imports found, and how many were deemed broken. Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com> Co-authored-by: Raúl Kripalani <raul@protocol.ai> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
2021-08-16 22:34:32 +00:00
github.com/multiformats/go-varint v0.0.6
OpenRPC Support (#5843) * main: init implement rpc.Discover RPC method This implement the basic functionality for the method over HTTP RPC. Signed-off-by: meows <b5c6@protonmail.com> * main,go.mod,go.sum: init example with go-openrpc-reflect lib Signed-off-by: meows <b5c6@protonmail.com> Conflicts: go.mod go.sum * main: make variable name human-friendly Signed-off-by: meows <b5c6@protonmail.com> * main,go.mod,go.sum: init impl of go-openrp-reflect printing document Signed-off-by: meows <b5c6@protonmail.com> Conflicts: go.mod go.sum * go.mod,go.sum: use go-openrpc-reflect and open-rpc/meta-schema hackforks This is for development only. Versions need to be bumped when they're ready for use as canonical remotes. Signed-off-by: meows <b5c6@protonmail.com> * main,openrpc,main: refactor openrpc supporting code to own package This eliminates code duplication. Signed-off-by: meows <b5c6@protonmail.com> * main: add rpc.Discover to openrpc document Signed-off-by: meows <b5c6@protonmail.com> * openrpc: fix rpc.discover method name casing Also fixes casing stuff for the rest of Filecoin. methods. Signed-off-by: meows <b5c6@protonmail.com> * Revert "main: add rpc.Discover to openrpc document" This reverts commit 116898efb10f33e405ac74acb1aa6daefcd46a62. * main: fix document creation method name This fixes an issue caused with the latest reverting commit. Signed-off-by: meows <b5c6@protonmail.com> * main,docgen,openrpc: refactor to share api parsing, etc as docgen exported stuff Signed-off-by: meows <b5c6@protonmail.com> Makefile: fix docgen refactoring for makefile use of command Signed-off-by: meows <b5c6@protonmail.com> * openrpc: add schema.examples to app reflector There are quite of few of these already registered for the docgen command, so it makes sense to use those! Signed-off-by: meows <b5c6@protonmail.com> * openrpc: init method pairing examples Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump go.mod to use latest meta-schema and openrpc-reflect versions Signed-off-by: meows <b5c6@protonmail.com> * openrpc: init SchemaType mapper function This function will handle the manual configurations for app-specific data types w/r/t their json schema representation. This is useful for cases where the reflect library is unable to provide a sufficient representation automatically. Provided in this commit is an initial implementation for the integerD type (assuming number are represented in the API as hexs), and a commonly used cid.Cid type. Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: tame dependencies by bumping etclabscore/go-openrpc-reflect This removes a problematic dependency on github.com/ethereum/go-ethereum, which was imported as a dependency for a couple github.com/etclabscore/go-openrpc-reflect tests. etclabscore/go-openrpc-reflect v0.0.36 has removed this dependency, so this commit is the result of bumping that version and then running 'go mod tidy' This is in response to a review at https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205 Date: 2020-11-21 06:52:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: add 'miner' arg to openrpc gen cmd This allows the command to EITHER generate the doc for Full or Miner APIs. See comment for usage. Date: 2020-11-21 07:48:05-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgen: add missing examples for Miner API Generating the Miner API OpenRPC doc (via 'go run ./api/openrpc/cmd miner') caused the example logic to panic because some types were missing. This commit adds those missing types, although I'm not an expert in the API so I can't suggest that the example values provided are ideal or well representative. Date: 2020-11-21 07:50:21-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build/openrpc/full.json,build/openrpc/miner.json: add build/openrpc/[full/miner].json docs These will be used as static documents provided by the rpc.discover method. Date: 2020-11-21 07:51:39-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: init go-rice openrpc static assets Date: 2020-11-21 08:23:06-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: remove rpc.discover implementation from runtime plugin Instead of generating the doc on the fly, we're going to serve a static asset. Rel https://github.com/filecoin-project/lotus/pull/4711#pullrequestreview-535686205 This removes the runtime implementation from the RPC server construction. Date: 2020-11-21 08:41:20-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,common: add Discover(ctx) method to CommonAPI interface and structs Date: 2020-11-21 08:41:56-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main: use rpc server method aliasing for rpc.discover This depends on a currently-forked change at filecoin-project/go-jsonrpc 8350f9463ee451b187d35c492e32f1b999e80210 which establishes this new method RPCServer.AliasMethod. This solves the problem that the OpenRPC spec says that the document should be served at the system extension-prefixed endpoing rpc.discover (not Filecoin.Discover). In fact, the document will be available at BOTH endpoints, but that duplicity is harmless. Date: 2020-11-21 09:18:26-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,build,common: rpc.discover: return json object instead of string Instead of casting the JSON asset from bytes to string, unmarshal it to a map[string]interface{} so the server will provide it as a JSON object. Date: 2020-11-21 09:27:11-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile: merge resolve: docsgen command path Date: 2020-11-22 07:19:36-06:00 Signed-off-by: meows <b5c6@protonmail.com> * apistruct,main,docgen,openrpc: merge resolve: fix func exporteds, signatures Date: 2020-11-22 07:31:03-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: 'get get' auto-bumps version Date: 2020-11-22 07:31:44-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile,docgen,main,build/openrpc: refactor openrpc documentation generation This creates Makefile command docsgen-openrpc-json, and refactors the docsgen command to generate both the markdown and openrpc json documents, redirecting the output of the openrpc json documentation to the build/openrpc/ directory, where those json files will be compiled as static assets via go-rice boxes. The api/openrpc/cmd now uses usage argumentation congruent to that of the docgen command (switching on API context). Date: 2020-11-22 08:01:18-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main,docgen_openrpc: rename api/openrpc -> api/docgen-openrpc Renames the package as well. This is intended to parallel the existing docgen package and command namespacing. Date: 2020-11-22 10:34:46-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api,apistruct,docgen,build,build/openrpc: use typed Discover response Instead of using a map[string]interface{}, use a typed response for the Discover method implementation. This avoids having to set a docgen Example for the generic map[string]interface{} (as an openrpc document) which both pollutes the generic type and lacks useful information for the Discover method example. Date: 2020-11-22 08:31:16-06:00 Signed-off-by: meows <b5c6@protonmail.com> * apistruct,build,main,impl: implement Discover method for Worker and StorageMiner APIs Methods return static compiled assets respective to the APIs. Date: 2020-11-22 08:57:18-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgen_openrpc,build/openrpc: remove timestamping from openrpc doc info This should allow openrpc docs generated at different times to be equal. This is important because the CI (Circle) runs the docgen command and tests that the output and the source are unchanged (via git diff). Date: 2020-11-22 10:47:07-06:00 Signed-off-by: meows <b5c6@protonmail.com> * main,docgen_openrpc,main,build: fix lint issues Fixes goimports, staticcheck, golint issues. Date: 2020-11-22 11:06:46-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc: fix: don't use an underscore in package name (golint) Date: 2020-11-22 11:07:53-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.sum: fix: mod-tidy-check (run 'go mod tidy') Date: 2020-11-22 11:09:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump filecoin-project/go-jsonrpc dep to latest This version includes the necessary RPCServer.AliasMethod method. Date: 2020-11-23 12:16:15-06:00 Signed-off-by: meows <b5c6@protonmail.com> * Makefile,main,build,build/openrpc: init gzipped openrpc static docs Date: 2020-11-24 06:15:06-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: refactor gzip reading Date: 2020-11-24 06:18:34-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: add basic test for openrpc doc from static assets Date: 2020-11-24 06:30:23-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build: handle reader Close error This keeps the errcheck linter happy. Date: 2020-11-24 06:33:14-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.sum: run 'go mod tidy' Date: 2020-11-24 06:36:07-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: go mod tidy Tidying up after resolving the merge conflicts with master at go.mod Date: 2020-11-24 06:40:45-06:00 Signed-off-by: meows <b5c6@protonmail.com> * go.mod,go.sum: bump filecoin-project/go-jsonrpc to latest This is a repeat of 76e6fd2, since the latest merge to master seems to have reverted this. Date: 2020-11-24 06:42:30-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc,build/openrpc: remove method example pairings, improve schema examples Removing method example pairings since they were redundant to schema examples and were not implemented well. Improved schema examples by using the ExampleValue method instead of the map lookup. Made a note in the comment here that this is not ideal, since we have to make a shortcut assumption /workaround by using 'unknown' as the method name and the typea as its own parent. Luckily these values aren't heavily used by the method logic. Date: 2020-11-27 12:57:36-06:00 Signed-off-by: meows <b5c6@protonmail.com> * docgenopenrpc: use generic number jsonschema for number types Previously used an integer schema assuming hex encoding. It appears, based on review some of the examples, that this may not be the case. Obvioussly this schema could be more descriptive, but just shooting for mostly likely to be not wrong at this point. Date: 2020-12-15 14:44:37-06:00 Signed-off-by: meows <b5c6@protonmail.com> * cmd/lotus,go.mod,go.sum: maybe fix straggling merge resolution conflicts Date: 2021-01-19 12:30:42-06:00 Signed-off-by: meows <b5c6@protonmail.com> * build/openrpc/full.json.gz,build/openrpc/miner.json.gz,build/openrpc/worker.json.gz: run 'make docsgen' Date: 2021-01-19 12:33:55-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/apistruct,node/impl: (lint) gofmt Date: 2021-01-19 12:39:48-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen: maybe fix parse error: open ./api: no such file or directory Date: 2021-01-19 12:52:04-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen,build/openrpc: maybe fix no such file error and run 'make docsgen' Date: 2021-01-19 12:55:52-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api/docgen: return if AST comment/groupdoc parsing encounters any error This will returns empty comments/docs maps. This should fix issues like: https://app.circleci.com/pipelines/github/filecoin-project/lotus/12445/workflows/4ebadce9-a298-4ad1-939b-f19ef4c0a5bf/jobs/107218 where the environment makes file lookups hard or impossible. Date: 2021-01-19 13:04:58-06:00 Signed-off-by: meows <b5c6@protonmail.com> * api: Don't depend on build/ * make: support parallel docsgen * openrpc gen: Use simple build version * methodgen * goimports Co-authored-by: meows <b5c6@protonmail.com>
2021-03-19 18:22:46 +00:00
github.com/open-rpc/meta-schema v0.0.0-20201029221707-1b72ef2ea333
2020-07-30 17:36:24 +00:00
github.com/opentracing/opentracing-go v1.2.0
github.com/polydawn/refmt v0.0.0-20201211092308-30ac6d18308e
2021-09-21 11:10:04 +00:00
github.com/prometheus/client_golang v1.11.0
2020-07-10 14:43:14 +00:00
github.com/raulk/clock v1.1.0
2021-12-14 13:27:34 +00:00
github.com/raulk/go-watchdog v1.2.0
github.com/streadway/quantile v0.0.0-20150917103942-b0c588724d25
github.com/stretchr/testify v1.7.0
github.com/syndtr/goleveldb v1.0.0
github.com/uber/jaeger-client-go v2.25.0+incompatible // indirect
github.com/urfave/cli/v2 v2.2.0
github.com/whyrusleeping/bencher v0.0.0-20190829221104-bb6607aa8bba
integrate DAG store and CARv2 in deal-making (#6671) This commit removes badger from the deal-making processes, and moves to a new architecture with the dagstore as the cental component on the miner-side, and CARv2s on the client-side. Every deal that has been handed off to the sealing subsystem becomes a shard in the dagstore. Shards are mounted via the LotusMount, which teaches the dagstore how to load the related piece when serving retrievals. When the miner starts the Lotus for the first time with this patch, we will perform a one-time migration of all active deals into the dagstore. This is a lightweight process, and it consists simply of registering the shards in the dagstore. Shards are backed by the unsealed copy of the piece. This is currently a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so when it's time to acquire a shard to serve a retrieval, the unsealed CARv1 is joined with its index (safeguarded by the dagstore), to form a read-only blockstore, thus taking the place of the monolithic badger. Data transfers have been adjusted to interface directly with CARv2 files. On inbound transfers (client retrievals, miner storage deals), we stream the received data into a CARv2 ReadWrite blockstore. On outbound transfers (client storage deals, miner retrievals), we serve the data off a CARv2 ReadOnly blockstore. Client-side imports are managed by the refactored *imports.Manager component (when not using IPFS integration). Just like it before, we use the go-filestore library to avoid duplicating the data from the original file in the resulting UnixFS DAG (concretely the leaves). However, the target of those imports are what we call "ref-CARv2s": CARv2 files placed under the `$LOTUS_PATH/imports` directory, containing the intermediate nodes in full, and the leaves as positional references to the original file on disk. Client-side retrievals are placed into CARv2 files in the location: `$LOTUS_PATH/retrievals`. A new set of `Dagstore*` JSON-RPC operations and `lotus-miner dagstore` subcommands have been introduced on the miner-side to inspect and manage the dagstore. Despite moving to a CARv2-backed system, the IPFS integration has been respected, and it continues to be possible to make storage deals with data held in an IPFS node, and to perform retrievals directly into an IPFS node. NOTE: because the "staging" and "client" Badger blockstores are no longer used, existing imports on the client will be rendered useless. On startup, Lotus will enumerate all imports and print WARN statements on the log for each import that needs to be reimported. These log lines contain these messages: - import lacks carv2 path; import will not work; please reimport - import has missing/broken carv2; please reimport At the end, we will print a "sanity check completed" message indicating the count of imports found, and how many were deemed broken. Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com> Co-authored-by: Raúl Kripalani <raul@protocol.ai> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
2021-08-16 22:34:32 +00:00
github.com/whyrusleeping/cbor-gen v0.0.0-20210713220151-be142a5ae1a8
github.com/whyrusleeping/ledger-filecoin-go v0.9.1-0.20201010031517-c3dcc1bddce4
github.com/whyrusleeping/multiaddr-filter v0.0.0-20160516205228-e903e4adabd7
2021-02-09 16:08:19 +00:00
github.com/whyrusleeping/pubsub v0.0.0-20190708150250-92bcb0691325
github.com/xorcare/golden v0.6.1-0.20191112154924-b87f686d7542
go.opencensus.io v0.23.0
go.opentelemetry.io/otel v1.3.0
go.opentelemetry.io/otel/bridge/opencensus v0.25.0
go.opentelemetry.io/otel/exporters/jaeger v1.2.0
go.opentelemetry.io/otel/sdk v1.2.0
go.uber.org/dig v1.10.0 // indirect
go.uber.org/fx v1.9.0
integrate DAG store and CARv2 in deal-making (#6671) This commit removes badger from the deal-making processes, and moves to a new architecture with the dagstore as the cental component on the miner-side, and CARv2s on the client-side. Every deal that has been handed off to the sealing subsystem becomes a shard in the dagstore. Shards are mounted via the LotusMount, which teaches the dagstore how to load the related piece when serving retrievals. When the miner starts the Lotus for the first time with this patch, we will perform a one-time migration of all active deals into the dagstore. This is a lightweight process, and it consists simply of registering the shards in the dagstore. Shards are backed by the unsealed copy of the piece. This is currently a CARv1. However, the dagstore keeps CARv2 indices for all pieces, so when it's time to acquire a shard to serve a retrieval, the unsealed CARv1 is joined with its index (safeguarded by the dagstore), to form a read-only blockstore, thus taking the place of the monolithic badger. Data transfers have been adjusted to interface directly with CARv2 files. On inbound transfers (client retrievals, miner storage deals), we stream the received data into a CARv2 ReadWrite blockstore. On outbound transfers (client storage deals, miner retrievals), we serve the data off a CARv2 ReadOnly blockstore. Client-side imports are managed by the refactored *imports.Manager component (when not using IPFS integration). Just like it before, we use the go-filestore library to avoid duplicating the data from the original file in the resulting UnixFS DAG (concretely the leaves). However, the target of those imports are what we call "ref-CARv2s": CARv2 files placed under the `$LOTUS_PATH/imports` directory, containing the intermediate nodes in full, and the leaves as positional references to the original file on disk. Client-side retrievals are placed into CARv2 files in the location: `$LOTUS_PATH/retrievals`. A new set of `Dagstore*` JSON-RPC operations and `lotus-miner dagstore` subcommands have been introduced on the miner-side to inspect and manage the dagstore. Despite moving to a CARv2-backed system, the IPFS integration has been respected, and it continues to be possible to make storage deals with data held in an IPFS node, and to perform retrievals directly into an IPFS node. NOTE: because the "staging" and "client" Badger blockstores are no longer used, existing imports on the client will be rendered useless. On startup, Lotus will enumerate all imports and print WARN statements on the log for each import that needs to be reimported. These log lines contain these messages: - import lacks carv2 path; import will not work; please reimport - import has missing/broken carv2; please reimport At the end, we will print a "sanity check completed" message indicating the count of imports found, and how many were deemed broken. Co-authored-by: Aarsh Shah <aarshkshah1992@gmail.com> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com> Co-authored-by: Raúl Kripalani <raul@protocol.ai> Co-authored-by: Dirk McCormick <dirkmdev@gmail.com>
2021-08-16 22:34:32 +00:00
go.uber.org/multierr v1.7.0
go.uber.org/zap v1.19.1
2021-12-10 02:01:20 +00:00
golang.org/x/crypto v0.0.0-20211209193657-4570a0811e8b // indirect
golang.org/x/net v0.0.0-20211112202133-69e39bad7dc2
golang.org/x/sync v0.0.0-20210220032951-036812b2e83c
2021-12-10 02:01:20 +00:00
golang.org/x/sys v0.0.0-20211209171907-798191bca915
golang.org/x/time v0.0.0-20210723032227-1f47c861a9ac
golang.org/x/tools v0.1.7
golang.org/x/xerrors v0.0.0-20200804184101-5ec99f83aff1
gopkg.in/cheggaaa/pb.v1 v1.0.28
gotest.tools v2.2.0+incompatible
2021-12-10 02:01:20 +00:00
lukechampine.com/blake3 v1.1.7 // indirect
)
replace github.com/filecoin-project/filecoin-ffi => ./extern/filecoin-ffi
replace github.com/filecoin-project/test-vectors => ./extern/test-vectors
//replace github.com/filecoin-project/specs-actors/v7 => /Users/zenground0/pl/repos/specs-actors
// replace github.com/filecon-project/specs-storage => /Users/zenground0/pl/repos/specs-storage