cerc-io/lotus
16
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 27 Wiki Activity

Merge branch 'devnet/9' into feat/election-post

Browse Source
This commit is contained in:
Łukasz Magiera 2019-12-02 12:25:23 +01:00
commit a47fbfa799
24 changed files with 469 additions and 365 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
.gitignore vendored
View File

@ -14,6 +14,8 @@
**/*.h
**/*.a
**/*.pc
/**/*/.DS_STORE
.DS_STORE
build/.*
build/paramfetch.sh
/vendor

301
README.md
View File

@ -1,6 +1,6 @@
![Lotus](docs/images/lotus_logo_h.png)
![Lotus](documentation/images/lotus_logo_h.png)
# project lotus - 莲
# Project Lotus - 莲
Lotus is an experimental implementation of the Filecoin Distributed Storage Network. For more details about Filecoin, check out the [Filecoin Spec](https://github.com/filecoin-project/specs).
@ -8,301 +8,10 @@ Lotus is an experimental implementation of the Filecoin Distributed Storage Netw
All work is tracked via issues. An attempt at keeping an up-to-date view on remaining work is in the [lotus testnet github project board](https://github.com/filecoin-project/lotus/projects/1).
## Building & Documentation
## Building
We currently only provide the option to build lotus from source. Binary installation options are coming soon!
In order to run lotus, please do the following:
1. Make sure you have these dependencies installed:
- go (1.13 or higher)
- gcc (7.4.0 or higher)
- git (version 2 or higher)
- bzr (some go dependency needs this)
- jq
- pkg-config
- opencl-icd-loader
- opencl driver (like nvidia-opencl on arch) (for GPU acceleration)
- opencl-headers (build)
- rustup (proofs build)
- llvm (proofs build)
- clang (proofs build)
Arch (run):
```sh
sudo pacman -Syu opencl-icd-loader
```
Arch (build):
```sh
sudo pacman -Syu go gcc git bzr jq pkg-config opencl-icd-loader opencl-headers
```
Ubuntu / Debian (run):
```sh
sudo apt update
sudo apt install mesa-opencl-icd ocl-icd-opencl-dev
```
Ubuntu (build):
```sh
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt update
sudo apt install golang-go gcc git bzr jq pkg-config mesa-opencl-icd ocl-icd-opencl-dev
```
2. Clone this repo & `cd` into it
```
$ git clone https://github.com/filecoin-project/lotus.git
$ cd lotus/
```
3. Build and install the source code
```
$ make clean all
$ sudo make install
```
Now, you should be able to perform the commands listed below.
## Devnet
### Node setup
If you have run lotus before and want to remove all previous data: `rm -rf ~/.lotus ~/.lotusstorage`
The following sections describe how to use the lotus CLI. Alternately you can run lotus nodes and miners using the [Pond GUI](#pond).
### Genesis & Bootstrap
The current lotus build will automatically join the lotus Devnet using the genesis and bootstrap files in the `build/` directory. No configuration is needed.
### Start Daemon
```sh
$ lotus daemon
```
In another window check that you are connected to the network:
```sh
$ lotus net peers | wc -l
2 # number of peers
```
Wait for the chain to finish syncing:
```sh
$ lotus sync wait
```
You can view latest block height along with other network metrics at the https://lotus-metrics.kittyhawk.wtf/chain.
### Basics
Create a new address:
```sh
$ lotus wallet new bls
t3...
```
Grab some funds from faucet - go to https://lotus-faucet.kittyhawk.wtf/, paste the address
you just created, and press Send.
Check the wallet balance (balance is listed in attoFIL, where 1 attoFIL = 10^-18 FIL):
```sh
$ lotus wallet balance [optional address (t3...)]
```
(NOTE: If you see an error like `actor not found` after executing this command, it means that either your node isn't fully synced or there are no transactions to this address yet on chain. If the latter, using the faucet should 'fix' this).
### Mining
Ensure that at least one BLS address (`t3..`) in your wallet exists
```sh
$ lotus wallet list
t3...
```
With this address, go to https://lotus-faucet.kittyhawk.wtf/miner.html, and
click `Create Miner`
Wait for a page telling you the address of the newly created storage miner to
appear - It should be saying: `New storage miners address is: t0..`
Initialize storage miner:
```sh
$ lotus-storage-miner init --actor=t01.. --owner=t3....
```
This command should return successfully after miner is setup on-chain (30-60s)
Start mining:
```sh
$ lotus-storage-miner run
```
To view the miner id used for deals:
```sh
$ lotus-storage-miner info
```
e.g. miner id `t0111`
Seal random data to start producing PoSts:
```sh
$ lotus-storage-miner store-garbage
```
You can check miner power and sector usage with the miner id:
```sh
# Total power of the network
$ lotus-storage-miner state power
$ lotus-storage-miner state power <miner>
$ lotus-storage-miner state sectors <miner>
```
### Stage Data
Import some data:
```sh
# Create a simple file
$ echo "Hi my name is $USER" > hello.txt
# Import the file into lotus & get a Data CID
$ lotus client import ./hello.txt
<Data CID>
# List imported files by CID, name, size, status
$ lotus client local
```
(CID is short for Content Identifier, a self describing content address used throughout the IPFS ecosystem. It is a cryptographic hash that uniquely maps to the data and verifies it has not changed.)
### Make a deal
(It is possible for a Client to make a deal with a Miner on the same lotus Node.)
```sh
# List all miners in the system. Choose one to make a deal with.
$ lotus state list-miners
# List asks proposed by a miner
$ lotus client query-ask <miner>
# Propose a deal with a miner. Price is in attoFIL/byte/block. Duration is # of blocks.
$ lotus client deal <Data CID> <miner> <price> <duration>
```
For example `$ lotus client deal bafkre...qvtjsi t0111 36000 12` proposes a deal to store CID `bafkre...qvtjsi` with miner `t0111` at price `36000` for a duration of `12` blocks. If successful, the `client deal` command will return a deal CID.
### Search & Retrieval
If you've stored data with a miner in the network, you can search for it by CID:
```sh
# Search for data by CID
$ lotus client find <Data CID>
LOCAL
RETRIEVAL <miner>@<miner peerId>-<deal funds>-<size>
```
To retrieve data from a miner:
```sh
$ lotus client retrieve <Data CID> <outfile>
```
This will initiate a retrieval deal and write the data to the outfile. (This process may take some time.)
### Monitoring Dashboard
To see the latest network activity, including chain block height, blocktime, total network power, largest miners, and more, check out the monitoring dashboard at https://lotus-metrics.kittyhawk.wtf.
### Pond UI
-----
As an alternative to the CLI you can use Pond, a graphical testbed for lotus. It can be used to spin up nodes, connect them in a given topology, start them mining, and observe how they function over time.
Build:
```
$ make pond
```
Run:
```
$ ./pond run
Listening on http://127.0.0.1:2222
```
Now go to http://127.0.0.1:2222.
**Things to try:**
- The `Spawn Node` button starts a new lotus Node in a new draggable window.
- Click `[Spawn Storage Miner]` to start mining (make sure the Node's wallet has funds).
- Click on `[Client]` to open the Node's client interface and propose a deal with an existing Miner. If successful you'll see a payment channel open up with that Miner.
> Note: Don't leave Pond unattended for long periods of time (10h+), the web-ui tends to
> eventually consume all the available RAM.
### Troubleshooting
* Turn it off and on - Start at the top
* `rm -rf ~/.lotus ~/.lotusstorage/`
* Verify you have the correct versions of dependencies
* If stuck on a bad fork, try `lotus chain sethead --genesis`
* If that didn't help, open a new issue, ask in the [Community forum](https://discuss.filecoin.io) or reach out via [Community chat](https://github.com/filecoin-project/community#chat).
## Architecture
Lotus is architected modularly, and aims to keep clean API boundaries between everything, even if they are in the same process. Notably, the 'lotus full node' software, and the 'lotus storage miner' software are two separate programs.
The lotus storage miner is intended to be run on the machine that manages a single storage miner instance, and is meant to communicate with the full node via the websockets jsonrpc api for all of its chain interaction needs. This way, a mining operation may easily run one or many storage miners, connected to one or many full node instances.
## Notable Modules
### API
The systems API is defined in here. The RPC maps directly to the API defined here using the JSON RPC package in `lib/jsonrpc`. Initial API documentation in [docs/API.md](docs/API.md).
### Chain/Types
Implementation of data structures used by Filecoin and their serializations.
### Chain/Store
The chainstore manages all local chain state, including block headers, messages, and state.
### Chain/State
A package for dealing with the Filecoin state tree. Wraps the [HAMT](https://github.com/ipfs/go-hamt-ipld).
### Chain/Actors
Implementations of the builtin Filecoin network actors.
### Chain/Vm
The Filecoin state machine 'vm'. Implemented here are utilities to invoke Filecoin actor methods.
### Miner
The block producer logic. This package interfaces with the full node through the API, despite currently being implemented in the same process (very likely to be extracted as its own separate process in the near future).
### Storage
The storage miner logic. This package also interfaces with the full node through a subset of the api. This code is used to implement the `lotus-storage-miner` process.
## Pond
Pond is a graphical testbed for lotus. It can be used to spin up nodes, connect them in a given topology, start them mining, and observe how they function over time.
To try it out, run `make pond`, then run `./pond run`.
Once it is running, visit localhost:2222 in your browser.
## Tracing
Lotus has tracing built into many of its internals. To view the traces, first download jaeger](https://www.jaegertracing.io/download/) (Choose the 'all-in-one' binary). Then run it somewhere, start up the lotus daemon, and open up localhost:16686 in your browser.
For more details, see [this document](./docs/tracing.md).
For instructions on how to build lotus from source, please visit [https://docs.lotu.sh](https://docs.lotu.sh) or read the source [here](https://github.com/filecoin-project/lotus/tree/master/documentation).
## License
Dual-licensed under [MIT](https://github.com/filecoin-project/lotus/blob/master/LICENSE-MIT) + [Apache 2.0](https://github.com/filecoin-project/lotus/blob/master/LICENSE-APACHE)

26
chain/address/address.go
View File

@ -8,9 +8,9 @@ import (
"strconv"
bls "github.com/filecoin-project/filecoin-ffi"
"github.com/filecoin-project/go-leb128"
cbor "github.com/ipfs/go-ipld-cbor"
"github.com/minio/blake2b-simd"
"github.com/multiformats/go-varint"
"github.com/polydawn/refmt/obj/atlas"
"golang.org/x/xerrors"
@ -166,7 +166,7 @@ func (a *Address) Scan(value interface{}) error {
// NewIDAddress returns an address using the ID protocol.
func NewIDAddress(id uint64) (Address, error) {
return newAddress(ID, leb128.FromUInt64(id))
return newAddress(ID, varint.ToUvarint(id))
}
// NewSecp256k1Address returns an address using the SECP256K1 protocol.
@ -218,6 +218,14 @@ func addressHash(ingest []byte) []byte {
func newAddress(protocol Protocol, payload []byte) (Address, error) {
switch protocol {
case ID:
_, n, err := varint.FromUvarint(payload)
if err != nil {
return Undef, xerrors.Errorf("could not decode: %v: %w", err, ErrInvalidPayload)
}
if n != len(payload) {
return Undef, xerrors.Errorf("different varint length (v:%d != p:%d): %w",
n, len(payload), ErrInvalidPayload)
}
case SECP256K1, Actor:
if len(payload) != PayloadHashLength {
return Undef, ErrInvalidPayload
@ -258,7 +266,14 @@ func encode(network Network, addr Address) (string, error) {
cksm := Checksum(append([]byte{addr.Protocol()}, addr.Payload()...))
strAddr = ntwk + fmt.Sprintf("%d", addr.Protocol()) + AddressEncoding.WithPadding(-1).EncodeToString(append(addr.Payload(), cksm[:]...))
case ID:
strAddr = ntwk + fmt.Sprintf("%d", addr.Protocol()) + fmt.Sprintf("%d", leb128.ToUInt64(addr.Payload()))
i, n, err := varint.FromUvarint(addr.Payload())
if err != nil {
return UndefAddressString, xerrors.Errorf("could not decode varint: %w", err)
}
if n != len(addr.Payload()) {
return UndefAddressString, xerrors.Errorf("payload contains additional bytes")
}
strAddr = fmt.Sprintf("%s%d%d", ntwk, addr.Protocol(), i)
default:
return UndefAddressString, ErrUnknownProtocol
}
@ -304,7 +319,7 @@ func decode(a string) (Address, error) {
if err != nil {
return Undef, ErrInvalidPayload
}
return newAddress(protocol, leb128.FromUInt64(id))
return newAddress(protocol, varint.ToUvarint(id))
}
payloadcksm, err := AddressEncoding.WithPadding(-1).DecodeString(raw)
@ -395,5 +410,6 @@ func IDFromAddress(addr Address) (uint64, error) {
return 0, xerrors.Errorf("cannot get id from non id address")
}
return leb128.ToUInt64(addr.Payload()), nil
i, _, err := varint.FromUvarint(addr.Payload())
return i, err
}

13
chain/address/address_test.go
View File

@ -11,7 +11,8 @@ import (
"time"
ffi "github.com/filecoin-project/filecoin-ffi"
"github.com/filecoin-project/go-leb128"
"github.com/filecoin-project/go-bls-sigs"
"github.com/multiformats/go-varint"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@ -94,7 +95,9 @@ func TestVectorsIDAddress(t *testing.T) {
maybeAddr, err := NewFromString(tc.expected)
assert.NoError(err)
assert.Equal(ID, maybeAddr.Protocol())
assert.Equal(tc.input, leb128.ToUInt64(maybeAddr.Payload()))
id, _, err := varint.FromUvarint(maybeAddr.Payload())
assert.NoError(err)
assert.Equal(tc.input, id)
// Round trip to and from bytes
maybeAddrBytes, err := NewFromBytes(maybeAddr.Bytes())
@ -532,3 +535,9 @@ func BenchmarkCborUnmarshal(b *testing.B) {
}
}
}
func TestIDEdgeCase(t *testing.T) {
a, err := NewFromBytes([]byte{0, 0x80})
_ = a.String()
assert.Error(t, err)
}

9
chain/blocksync/blocksync_client.go
View File

@ -156,8 +156,9 @@ func (bs *BlockSync) GetChainMessages(ctx context.Context, h *types.TipSet, coun
var err error
for _, p := range perm {
res, err := bs.sendRequestToPeer(ctx, peers[p], req)
if err != nil {
res, rerr := bs.sendRequestToPeer(ctx, peers[p], req)
if rerr != nil {
err = rerr
log.Warnf("BlockSync request failed for peer %s: %s", peers[p].String(), err)
continue
}
@ -172,6 +173,10 @@ func (bs *BlockSync) GetChainMessages(ctx context.Context, h *types.TipSet, coun
}
}
if err == nil {
return nil, xerrors.Errorf("GetChainMessages failed, no peers connected")
}
// TODO: What if we have no peers (and err is nil)?
return nil, xerrors.Errorf("GetChainMessages failed with all peers(%d): %w", len(peers), err)
}

32
docs/API.md
View File

@ -1,32 +0,0 @@
TODO: make this into a nicer doc
### Endpoints
By default `127.0.0.1:1234` - daemon stores the api endpoint multiaddr in `~/.lotus/api`
* `http://[api:port]/rpc/v0` - JsonRPC http endpoint
* `ws://[api:port]/rpc/v0` - JsonRPC websocket endpoint
* `PUT http://[api:port]/rest/v0/import` - import file to the node repo
* Requires write permission
For JsonRPC interface definition see `api/api.go`. Required permissions are
defined in `api/struct.go`
### Auth:
JWT in the `Authorization: Bearer <token>` http header
Permissions:
* `read` - Read node state, no private data
* `write` - Write to local store / chain, read private data
* `sign` - Use private keys stored in wallet for signing
* `admin` - Manage permissions
Payload:
```json
{
"Allow": ["read", "write", ...]
}
```
Admin token is stored in `~/.lotus/token`

35
documentation/api.md Normal file
View File

@ -0,0 +1,35 @@
# API
The systems API is defined in here. The RPC maps directly to the API defined here using the [JSON RPC package](https://github.com/filecoin-project/lotus/tree/master/lib/jsonrpc).
## Overview
By default `127.0.0.1:1234` - daemon stores the api endpoint multiaddr in `~/.lotus/api`
- `http://[api:port]/rpc/v0` - JsonRPC http endpoint
- `ws://[api:port]/rpc/v0` - JsonRPC websocket endpoint
- `PUT http://[api:port]/rest/v0/import` - import file to the node repo, it requires write permission.
For JsonRPC interface definition see [api/api.go](https://github.com/filecoin-project/lotus/blob/master/api/api_full.go). Required permissions are
defined in [api/struct.go](https://github.com/filecoin-project/lotus/blob/master/api/struct.go)
## Auth
JWT in the `Authorization: Bearer <token>` http header
Permissions
- `read` - Read node state, no private data
- `write` - Write to local store / chain, read private data
- `sign` - Use private keys stored in wallet for signing
- `admin` - Manage permissions
Payload
```json
{
"Allow": ["read", "write", ...]
}
```
Admin token is stored in `~/.lotus/token`

18
documentation/getting-started.md Normal file
View File

@ -0,0 +1,18 @@
# Lotus
Lotus is an experimental implementation of the **Filecoin Distributed Storage Network**. For more details about Filecoin, check out the [Filecoin Spec](https://github.com/filecoin-project/specs).
## What can I learn here?
- If you want to [store](https://docs.lotu.sh/storing-data) or [retrieve](https://docs.lotu.sh/retrieving-data) data, you can install Lotus on [Arch Linux](https://docs.lotu.sh/install-lotus-arch), [Ubuntu](https://docs.lotu.sh/install-lotus-ubuntu), or [MacOS](https://docs.lotu.sh/install-lotus-macos) and get started.
- Learn how to join the DevNet with the [command line interface](https://docs.lotu.sh/join-devnet-cli).
- Learn how to test Lotus in a seperate local network using a [GUI](https://docs.lotu.sh/testing-with-gui).
- Learn how to mine Filecoin using the `lotus-storage-miner` in the [CLI](https://docs.lotu.sh/mining)
## What makes Lotus different?
Lotus is architected modularly to keep clean API boundaries while using the same process. The **Lotus Full Node** and the **Lotus Storage Miner** are two separate programs.
The **Lotus Storage Miner** is intended to be run on the machine that manages a single storage miner instance, and is meant to communicate with the full node via the websockets JSON RPC API for all of its chain interaction needs.
This way, a mining operation may easily run one or many storage miners, connected to one or many full node instances.

33
documentation/glossary.md Normal file
View File

@ -0,0 +1,33 @@
# Glossary
**Chain: Types**
Implementation of data structures used by Filecoin and their serializations.
**Chain: Store**
The chainstore manages all local chain state, including block headers, messages, and state.
**Chain: State**
A package for dealing with the Filecoin state tree. Wraps the [HAMT](https://github.com/ipfs/go-hamt-ipld).
**Chain: Actors**
Implementations of the builtin Filecoin network actors.
**Chain: VM**
The Filecoin state machine 'vm'. Implemented here are utilities to invoke Filecoin actor methods.
**Miner**
The block producer logic. This package interfaces with the full node through the API, despite currently being implemented in the same process (very likely to be extracted as its own separate process in the near future).
**Storage**
The storage miner logic. This package also interfaces with the full node through a subset of the api. This code is used to implement the `lotus-storage-miner` process.
**Pond**
[Pond](https://docs.lotu.sh/testing-with-gui) is a graphical testbed for lotus. It can be used to spin up nodes, connect them in a given topology, start them mining, and observe how they function over time.

5
documentation/hardware.md Normal file
View File

@ -0,0 +1,5 @@
# Hardware Requirements
Lotus can build and run on most [Linux](https://ubuntu.com/) and [MacOS](https://www.apple.com/macos) systems with at least 8GB of RAM.
Windows is not yet supported.

0
docs/images/lotus_logo_h.png → documentation/images/lotus_logo_h.png
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 5.8 KiB

After

Width:  |  Height:  |  Size: 5.8 KiB

0
docs/images/lotus_logo_h.svg → documentation/images/lotus_logo_h.svg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 4.8 KiB

After

Width:  |  Height:  |  Size: 4.8 KiB

44
documentation/install-lotus-arch.md Normal file
View File

@ -0,0 +1,44 @@
# Installing Lotus on Arch Linux
Install these dependencies for Arch Linux.
- go (1.13 or higher)
- gcc (7.4.0 or higher)
- git (version 2 or higher)
- bzr (some go dependency needs this)
- jq
- pkg-config
- opencl-icd-loader
- opencl driver (like nvidia-opencl on arch) (for GPU acceleration)
- opencl-headers (build)
- rustup (proofs build)
- llvm (proofs build)
- clang (proofs build)
Arch (run):
```sh
sudo pacman -Syu opencl-icd-loader
```
Arch (build):
```sh
sudo pacman -Syu go gcc git bzr jq pkg-config opencl-icd-loader opencl-headers
```
Clone
```sh
$ git clone https://github.com/filecoin-project/lotus.git
$ cd lotus/
```
Install
```sh
$ make clean all
$ sudo make install
```
Now you can use the command `lotus` in the command line.

32
documentation/install-lotus-macos.md Normal file
View File

@ -0,0 +1,32 @@
# Installing Lotus on MacOS
Install these dependencies for MacOS Catalina.
- go (1.13 or higher)
- gcc (7.4.0 or higher)
- git (version 2 or higher)
- bzr (some go dependency needs this)
- jq
- pkg-config
- opencl-icd-loader
- opencl driver (like nvidia-opencl on arch) (for GPU acceleration)
- opencl-headers (build)
- rustup (proofs build)
- llvm (proofs build)
- clang (proofs build)
Clone
```sh
$ git clone https://github.com/filecoin-project/lotus.git
$ cd lotus/
```
Install
```sh
$ make clean all
$ sudo make install
```
Now you can use the command `lotus` in the command line.

47
documentation/install-lotus-ubuntu.md Normal file
View File

@ -0,0 +1,47 @@
# Installing Lotus on Ubuntu
Install these dependencies for Ubuntu.
- go (1.13 or higher)
- gcc (7.4.0 or higher)
- git (version 2 or higher)
- bzr (some go dependency needs this)
- jq
- pkg-config
- opencl-icd-loader
- opencl driver (like nvidia-opencl on arch) (for GPU acceleration)
- opencl-headers (build)
- rustup (proofs build)
- llvm (proofs build)
- clang (proofs build)
Ubuntu / Debian (run):
```sh
sudo apt update
sudo apt install mesa-opencl-icd ocl-icd-opencl-dev
```
Ubuntu (build):
```sh
sudo add-apt-repository ppa:longsleep/golang-backports
sudo apt update
sudo apt install golang-go gcc git bzr jq pkg-config mesa-opencl-icd ocl-icd-opencl-dev
```
Clone
```sh
$ git clone https://github.com/filecoin-project/lotus.git
$ cd lotus/
```
Install
```sh
$ make clean all
$ sudo make install
```
Now you can use the command `lotus` in the command line.

74
documentation/join-devnet-cli.md Normal file
View File

@ -0,0 +1,74 @@
# Join Lotus Devnet
## Node CLI setup
If you have run lotus before and want to remove all previous data: `rm -rf ~/.lotus ~/.lotusstorage`
## Genesis & Bootstrap
The current lotus build will automatically join the lotus Devnet using the genesis and bootstrap files in the `build/` directory. No configuration is needed.
## Start Daemon
```sh
$ lotus daemon
```
In another window check that you are connected to the network:
```sh
$ lotus net peers | wc -l
2 # number of peers
```
Wait for the chain to finish syncing:
```sh
$ lotus sync wait
```
You can view latest block height along with other network metrics at the https://lotus-metrics.kittyhawk.wtf/chain.
## Basics
Create a new address:
```sh
$ lotus wallet new bls
t3...
```
Grab some funds from faucet - go to https://lotus-faucet.kittyhawk.wtf/, paste the address
you just created, and press Send.
Check the wallet balance (balance is listed in attoFIL, where 1 attoFIL = 10^-18 FIL):
```sh
$ lotus wallet balance [optional address (t3...)]
```
If you see an error like `actor not found` after executing this command, it means that either your node isn't fully synced or there are no transactions to this address yet on chain. If the latter, using the faucet should 'fix' this.
## Make a deal
It is possible for a Client to make a deal with a Miner on the same lotus Node.
```sh
# List all miners in the system. Choose one to make a deal with.
$ lotus state list-miners
# List asks proposed by a miner
$ lotus client query-ask <miner>
# Propose a deal with a miner. Price is in attoFIL/byte/block. Duration is # of blocks.
$ lotus client deal <Data CID> <miner> <price> <duration>
```
For example `\$ lotus client deal bafkre...qvtjsi t0111 36000 12` proposes a deal to store CID `bafkre...qvtjsi` with miner `t0111` at price `36000` for a duration of `12` blocks. If successful, the `client deal` command will return a deal CID.
## Monitoring Dashboard
To see the latest network activity, including chain block height, blocktime, total network power, largest miners, and more, check out the [monitoring dashboard](https://lotus-metrics.kittyhawk.wtf).

55
documentation/mining.md Normal file
View File

@ -0,0 +1,55 @@
# Getting started
Ensure that at least one BLS address (`t3..`) in your wallet exists
```sh
$ lotus wallet list
t3...
```
With this address, go to [the faucet](https://lotus-faucet.kittyhawk.wtf/miner.html), and
click `Create Miner`
Wait for a page telling you the address of the newly created storage miner to
appear.
The screen should show: `New storage miners address is: t0..`
## Initialize
```sh
$ lotus-storage-miner init --actor=t01.. --owner=t3....
```
This command should return successfully after miner is setup on-chain (30-60s)
## Start mining
```sh
$ lotus-storage-miner run
```
To view the miner id used for deals:
```sh
$ lotus-storage-miner info
```
e.g. miner id `t0111`
Seal random data to start producing PoSts:
```sh
$ lotus-storage-miner store-garbage
```
You can check miner power and sector usage with the miner id:
```sh
# Total power of the network
$ lotus-storage-miner state power
$ lotus-storage-miner state power <miner>
$ lotus-storage-miner state sectors <miner>
```

17
documentation/retrieving-data.md Normal file
View File

@ -0,0 +1,17 @@
# Retrieving Data
If you have stored data with a miner in the network, you can search for it by CID
```sh
$ lotus client find <Data CID>
LOCAL
RETRIEVAL <miner>@<miner peerId>-<deal funds>-<size>
```
Retrieve data from a miner
```sh
$ lotus client retrieve <Data CID> <outfile>
```
This will initiate a retrieval deal and write the data to the outfile. This process may take some time.

20
documentation/storing-data.md Normal file
View File

@ -0,0 +1,20 @@
# Storing Data
Start by creating a file, in this example we will use the command line to create `hello.txt`.
```sh
$ echo "Hi my name is $USER" > hello.txt
```
Afterwards you can import the file into lotus & get a **Data CID** as output.
```sh
$ lotus client import ./hello.txt
<Data CID>
```
To see a list of files by `CID`, `name`, `size`, `status`.
```sh
$ lotus client local
```

33
documentation/testing-with-gui.md Normal file
View File

@ -0,0 +1,33 @@
# Pond UI
Pond is a graphical testbed for [Lotus](https://docs.lotu.sh). Using it will setup a seperate local network which is helpful for debugging. Pond will spin up nodes, connect them in a given topology, start them mining, and observe how they function over time.
## Build
```sh
$ make pond
```
## Run
```sh
$ ./pond run
```
Now go to http://127.0.0.1:2222.
## What can I test?
- The `Spawn Node` button starts a new lotus Node in a new draggable window.
- Click `[Spawn Storage Miner]` to start mining. This require's the node's wallet to have funds.
- Click on `[Client]` to open the Node's client interface and propose a deal with an existing Miner. If successful you'll see a payment channel open up with that Miner.
Don't leave Pond unattended for long periods of time (10h+), the web-ui tends to eventually consume all the available RAM.
## Troubleshooting
- Turn it off and on - Start at the top
- `rm -rf ~/.lotus ~/.lotusstorage/`
- Verify you have the correct versions of dependencies
- If stuck on a bad fork, try `lotus chain sethead --genesis`
- If that didn't help, open a new issue, ask in the [Community forum](https://discuss.filecoin.io) or reach out via [Community chat](https://github.com/filecoin-project/community#chat).

10
docs/tracing.md → documentation/tracing.md
View File

@ -1,4 +1,8 @@
## Tracing
# Tracing
Lotus has tracing built into many of its internals. To view the traces, first download [jaeger](https://www.jaegertracing.io/download/) (Choose the 'all-in-one' binary). Then run it somewhere, start up the lotus daemon, and open up localhost:16686 in your browser.
## Open Census
Lotus uses [OpenCensus](https://opencensus.io/) for tracing application flow.
This generates spans
@ -10,8 +14,7 @@ fairly easy to swap in.
## Running Locally
To easily run and view tracing locally, first, install jaeger. The easiest way
to do this is to download the binaries from
https://www.jaegertracing.io/download/ and then run the `jaeger-all-in-one`
to do this is to [download the binaries](https://www.jaegertracing.io/download/) and then run the `jaeger-all-in-one`
binary. This will start up jaeger, listen for spans on `localhost:6831`, and
expose a web UI for viewing traces on `http://localhost:16686/`.
@ -22,6 +25,7 @@ Now, to view any generated traces, open up `http://localhost:16686/` in your
browser.
## Adding Spans
To annotate a new codepath with spans, add the following lines to the top of the function you wish to trace:
```go

3
go.mod
View File

@ -12,7 +12,6 @@ require (
github.com/filecoin-project/chain-validation v0.0.0-20191106200742-11986803c0f7
github.com/filecoin-project/filecoin-ffi v0.0.0-00010101000000-000000000000
github.com/filecoin-project/go-amt-ipld v0.0.0-20191122035745-59b9dfc0efc7
github.com/filecoin-project/go-leb128 v0.0.0-20190212224330-8d79a5489543
github.com/gbrlsnchs/jwt/v3 v3.0.0-beta.1
github.com/go-ole/go-ole v1.2.4 // indirect
github.com/google/go-cmp v0.3.1 // indirect
@ -70,7 +69,6 @@ require (
github.com/mattn/go-isatty v0.0.9 // indirect
github.com/mattn/go-runewidth v0.0.4 // indirect
github.com/mattn/go-sqlite3 v1.12.0
github.com/miekg/dns v1.1.16 // indirect
github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1
github.com/minio/sha256-simd v0.1.1
github.com/mitchellh/go-homedir v1.1.0
@ -79,6 +77,7 @@ require (
github.com/multiformats/go-multiaddr-dns v0.2.0
github.com/multiformats/go-multiaddr-net v0.1.0
github.com/multiformats/go-multihash v0.0.9
github.com/multiformats/go-varint v0.0.1
github.com/onsi/ginkgo v1.9.0 // indirect
github.com/onsi/gomega v1.6.0 // indirect
github.com/opentracing/opentracing-go v1.1.0

7
go.sum
View File

@ -449,8 +449,6 @@ github.com/mattn/go-sqlite3 v1.12.0/go.mod h1:FPy6KqzDD04eiIsT53CuJW3U88zkxoIYsO
github.com/matttproud/golang_protobuf_extensions v1.0.1/go.mod h1:D8He9yQNgCq6Z5Ld7szi9bcBfOoFv/3dc6xSMkL2PC0=
github.com/mgutz/ansi v0.0.0-20170206155736-9520e82c474b/go.mod h1:01TrycV0kFyexm33Z7vhZRXopbI8J3TDReVlkTgMUxE=
github.com/miekg/dns v1.1.12/go.mod h1:W1PPwlIAgtquWBMBEV9nkV9Cazfe8ScdGz/Lj7v3Nrg=
github.com/miekg/dns v1.1.16 h1:iMEQ/IVHxPTtx2Q07JP/k4CKRvSjiAZjZ0hnhgYEDmE=
github.com/miekg/dns v1.1.16/go.mod h1:YNV562EiewvSmpCB6/W4c6yqjK7Z+M/aIS1JHsIVeg8=
github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1 h1:lYpkrQH5ajf0OXOcUbGjvZxxijuBwbbmlSxLiuofa+g=
github.com/minio/blake2b-simd v0.0.0-20160723061019-3f5f724cb5b1/go.mod h1:pD8RvIylQ358TN4wwqatJ8rNavkEINozVn9DtGI3dfQ=
github.com/minio/sha256-simd v0.0.0-20190131020904-2d45a736cd16/go.mod h1:2FMWW+8GMoPweT6+pI63m9YE3Lmw4J71hV56Chs1E/U=
@ -498,6 +496,8 @@ github.com/multiformats/go-multihash v0.0.9/go.mod h1:YSLudS+Pi8NHE7o6tb3D8vrpKa
github.com/multiformats/go-multistream v0.0.1/go.mod h1:fJTiDfXJVmItycydCnNx4+wSzZ5NwG2FEVAI30fiovg=
github.com/multiformats/go-multistream v0.1.0 h1:UpO6jrsjqs46mqAK3n6wKRYFhugss9ArzbyUzU+4wkQ=
github.com/multiformats/go-multistream v0.1.0/go.mod h1:fJTiDfXJVmItycydCnNx4+wSzZ5NwG2FEVAI30fiovg=
github.com/multiformats/go-varint v0.0.1 h1:TR/0rdQtnNxuN2IhiB639xC3tWM4IUi7DkTBVTdGW/M=
github.com/multiformats/go-varint v0.0.1/go.mod h1:3Ls8CIEsrijN6+B7PbrXRPxHRPuXSrVKRY101jdMZYE=
github.com/mwitkow/go-conntrack v0.0.0-20161129095857-cc309e4a2223/go.mod h1:qRWi+5nqEBWmkhHvq77mSJWrCKwh8bxhgT7d/eI7P4U=
github.com/nkovacs/streamquote v0.0.0-20170412213628-49af9bddb229 h1:E2B8qYyeSgv5MXpmzZXRNp8IAQ4vjxIjhpAf5hv/tAg=
github.com/nkovacs/streamquote v0.0.0-20170412213628-49af9bddb229/go.mod h1:0aYXnNPJ8l7uZxf45rWW1a/uME32OF0rhiYGNQ2oF2E=
@ -628,7 +628,6 @@ go4.org v0.0.0-20190313082347-94abd6928b1d h1:JkRdGP3zvTtTbabWSAC6n67ka30y7gOzWA
go4.org v0.0.0-20190313082347-94abd6928b1d/go.mod h1:MkTOUMDaeVYJUOUsaDXIhWPZYa1yOyC1qaOBpL57BhE=
golang.org/x/crypto v0.0.0-20170930174604-9419663f5a44/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20180904163835-0709b304e793/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181001203147-e3636079e1a4/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20181203042331-505ab145d0a9/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190211182817-74369b46fc67/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
golang.org/x/crypto v0.0.0-20190225124518-7f87c0fbb88b/go.mod h1:6SG95UA2DQfeDnfUPMdvaQW0Q7yPrPDi9nlGo2tz2b4=
@ -651,7 +650,6 @@ golang.org/x/net v0.0.0-20180524181706-dfa909b99c79/go.mod h1:mL1N/T3taQHkDXs73r
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180906233101-161cd47e91fd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180926154720-4dfa2610cdf3/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181011144130-49bb7cea24b1/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20181114220301-adae6a3d119a/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190108225652-1e06a53dbb7e/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@ -677,7 +675,6 @@ golang.org/x/sync v0.0.0-20190423024810-112230192c58/go.mod h1:RxMgew5VJxzue5/jJ
golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180905080454-ebe1bf3edb33/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180909124046-d0be0721c37e/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20180928133829-e4b3c5e90611/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181116152217-5ac8a444bdc5/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181122145206-62eef0e2fa9b/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20181205085412-a5c9d58dba9a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=

18
node/modules/lp2p/discovery.go
View File

@ -6,7 +6,6 @@ import (
"github.com/libp2p/go-libp2p-core/host"
"github.com/libp2p/go-libp2p-core/peer"
"github.com/libp2p/go-libp2p/p2p/discovery"
"go.uber.org/fx"
"github.com/filecoin-project/lotus/node/modules/helpers"
@ -34,20 +33,3 @@ func DiscoveryHandler(mctx helpers.MetricsCtx, lc fx.Lifecycle, host host.Host)
host: host,
}
}
func SetupDiscovery(mdns bool, mdnsInterval int) func(helpers.MetricsCtx, fx.Lifecycle, host.Host, *discoveryHandler) error {
return func(mctx helpers.MetricsCtx, lc fx.Lifecycle, host host.Host, handler *discoveryHandler) error {
if mdns {
if mdnsInterval == 0 {
mdnsInterval = 5
}
service, err := discovery.NewMdnsService(helpers.LifecycleCtx(mctx, lc), host, time.Duration(mdnsInterval)*time.Second, discovery.ServiceTag)
if err != nil {
log.Errorw("mdns error", "error", err)
return nil
}
service.RegisterNotifee(handler)
}
return nil
}
}