From 89348d7fda3a5e6df9ad5a1aba5946dbea4f7ee6 Mon Sep 17 00:00:00 2001 From: Lucas Molas Date: Mon, 3 Aug 2020 20:06:05 -0300 Subject: [PATCH 1/7] doc: create miner (#1890) --- documentation/en/dev/create-miner.md | 90 ++++++++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 documentation/en/dev/create-miner.md diff --git a/documentation/en/dev/create-miner.md b/documentation/en/dev/create-miner.md new file mode 100644 index 000000000..44f4036a2 --- /dev/null +++ b/documentation/en/dev/create-miner.md @@ -0,0 +1,90 @@ +# Creating a storage miner + +This document explains the code flow of the [storage miner](https://filecoin-project.github.io/specs/#systems__filecoin_mining) creation process. It describes the flow on two dimensions: at the network level, from the local node to the rest of the Filecoin network, and at the VM level, from the CLI commands to the new chain state. It assumes the reader is already familiar with the general Lotus [architecture](architecture-2.md), relying especially on the descriptions of the CLI/API interface and the VM chain state. + +Note we are not following the [Storage Mining](https://lotu.sh/en+mining) user documentation where the miner is created along with the owner through the faucet, but we do those in separate stages try to exercise what will be the closest code path to Mainnet, using the `createStorageMiner()` call in the CLI (when `--actor` is not provided). + +FIXME: Check that all the information it assumes throughout this document is actually present in the main architecture doc. + +Topics with new information covered here (at different levels of detail) are: + * Addresses + * Wallet, key store + * Account actor + * Filecoin Message + * Storage node type + * Miner actor, owner and worker addresses + * Message generation and message pool + * Message execution in the VM + * Power actor + * PubSub/GossipSub + +## Wallet + +We start by creating a new wallet which will be associated with the miner. A "wallet" is just an abstraction over a Filecoin [address](https://filecoin-project.github.io/specs/#appendix__address) associated to an [account actor](https://filecoin-project.github.io/specs/#actor): an actor (object) in the VM that represents a user in the network. + +``` +lotus wallet new +# t1ildao7vmywh67lolnd7esq7bqkfkhnalxsj74wq +``` + +The returned (random-looking) string is the address associated with the new wallet. The first part of the string encodes its [`Protocol`](https://github.com/filecoin-project/go-address/blob/master/address.go), in this case type 1 (`t1`), which corresponds to a public-key address generated from the [`secp256k1`](https://en.bitcoin.it/wiki/Secp256k1) elliptic curve (default option in the `wallet` command, alternatively it can also generate the [BLS](https://en.wikipedia.org/wiki/Boneh%E2%80%93Lynn%E2%80%93Shacham) signature scheme). + +At this point the address still hasn't been transmitted to the network, it is just reflected on the key store of our local node. The key store (by default in `~/.lotus/keystore/`) contains all the keys generated by this node, including this new key which will be set as the *default* and will be used by other commands (like the miner creation one). (See the [`WalletNew()`](https://github.com/filecoin-project/lotus/blob/master/chain/wallet/wallet.go) API for more information about key generation and storage.) + +It is important to note that the key store also holds the *private* key associated with the public one (that represent a Filecoin address) but this one is *never* transmitted over the network, it is just used locally to sign messages and prove this node is in possession of it and hence represents the account actor associated to the address/wallet. + +## Account actor + +There is no explicit Filecoin message or CLI command that creates an account actor but rather it is implicit in the public-key address reflected by the wallet. The first time the VM sees it as the receiving address of a Filecoin message it will automatically create the associated account actor. + +We will trigger its creation then by sending funds to it, needed for the creation of the storage miner (note this document reflects the Filecoin Testnet, where we actually don't need funds to create a miner but will do the full procedure regardless to show the full cycle). In the Testnet we can send funds to an account trough the [Faucet](https://faucet.testnet.filecoin.io/funds.html), inserting the address returned by the `wallet new` command. + +FIXME: Is there an easy way to visualize the message generated by the Faucet? + +## Storage miner node + +The `lotus-storage-miner` command provides a set of tools to manage the miners associated with the local storage miner node. At this point it is important to note the different [node types](https://github.com/filecoin-project/lotus/blob/master/node/repo/fsrepo.go), in the previous document we always referred to a *single* local node, `FullNode`, which handled the sync process and any other communication with the Filecoin network (the term *full* stands for full validation of the consensus protocol, there are no *light* clients at the moment that do not do the full validation). We now create a new node of type [`StorageMiner`](https://github.com/filecoin-project/lotus/blob/master/node/repo/fsrepo.go), with its own repo (each node is always associated to its own repo), by default in `~/.lotusstorage`. The difference between the two nodes lies in the services they run (see build options in the main architecture document). + +The `lotus-storage-miner init` command option creates a new storage miner node. We will only be able to run the command once the chain has been synced by the full node (which needs to be running) and it will also require the download of the [proof parameters](https://filecoin.io/blog/filecoin-proof-system/) (of several GBs, so it may take some time). + +The main options that define a miner are the owner and worker addresses associated to it (stored in [`MinerInfo`](https://github.com/filecoin-project/specs-actors/blob/master/actors/builtin/miner/miner_state.go), a substructure of the Miner Actor State) and its peer ID. We use default values for all of these options in the command and briefly described them here: + +* Owner: address of the account actor that manages this miner (receives its payments, can change its information, etc.). By default this option is set to the default wallet address (and associated account actor), in the case of this example it will be the address just created with the `wallet new` command. + +* Worker: address of an account actor that signs blocks and submits proofs associated with this miner, it can be the same as the owner address. By default the command will create a new wallet address for it, of type BLS (the only one accepted for worker addresses). + +* [Peer ID](https://docs.libp2p.io/reference/glossary/#peerid): a network ID (belonging to the `libp2p` stack) used to contact the miner directly off-chain (e.g., to make a storage deal). Note the difference with the rest of the communication in the Filecoin network that happens largely inside the chain itself: when we "send" messages to the different actors that is actually a VM abstraction meaning we execute the method in the VM itself run by logic of the targeted actor, physically (at the network TCP/IP level) we broadcast the message to all of our peers to be included in a Filecoin block. + +With the miner information filled the command constructs a Filecoin message to broadcast to the network and be included in a Filecoin block by a miner (see [`createStorageMiner()`](https://github.com/filecoin-project/lotus/blob/master/cmd/lotus-storage-miner/init.go)). We will wait for that block to be synced to the chain (by the full node) before returning the miner ID address. The ID address is another way to refer to the miner through a unique ID in the chain, it has a type 0 and it is the address that is normally seen in chain visualization tools, e.g., `t01475` (since, in contrast with the public-key types of addresses, it is easily readable by humans). + +The Filecoin message constructed will be targeted to the [Power Actor](https://filecoin-project.github.io/specs/#systems__filecoin_blockchain__storage_power_consensus__storage_power_actor) (`StoragePowerActorAddr`), which tracks the amount of power (storage capacity) every miner has, and it will have the method number of the [`CreateMiner`](https://github.com/filecoin-project/specs-actors/blob/master/actors/builtin/methods.go) constant. + +## Message broadcast + +At the network level, the message (to be included in a block) is sent through the [message pool](https://filecoin-project.github.io/specs/#systems__filecoin_blockchain__message_pool) to the miners. The message pool is an abstraction over `libp2p`'s [PubSub protocol](https://docs.libp2p.io/reference/glossary/#pubsub), a communication pattern where peers subscribe to a topic (an identifying string) and other peers publish to it (broadcasting to all the subscribed peers it has registered). (Particularly in Lotus we use the [GossipSub](https://github.com/libp2p/specs/blob/master/pubsub/gossipsub/README.md) implementation of that protocol.) In this case the topic we publish to is [`/fil/msgs`](https://github.com/filecoin-project/lotus/blob/master/build/params_shared.go), to which the miners will be subscribed and will receive, validate and process the message (see [`HandleIncomingMessages`](https://github.com/filecoin-project/lotus/blob/master/node/modules/services.go) in the [miner node construction](https://github.com/filecoin-project/lotus/blob/master/node/builder.go) for more details). + +It is of interest to note that the GossipSub protocol is the same that miners themselves use to publish mined blocks in the `/fil/blocks` topic, to which the full node is subscribed to, used in the sync process (along with the `hello` protocol described in the architecture doc). + +Back to the CLI command, the [`MpoolPushMessage`](https://github.com/filecoin-project/lotus/blob/master/node/impl/full/mpool.go) API in the full node takes care of populating the rest of the Filecoin message (e.g., inserting the correct nonce) and serializing it before actually publishing to the GossipSub network. After that, it explicitly waits for the message to be included by a miner in a block and then for that block to reach the chain where the full node will receive it as part of the sync process (see [`StateWaitMsg`](https://github.com/filecoin-project/lotus/blob/master/node/impl/full/state.go)), checking its return code for success and its return value for the miner ID. + +## VM: message execution + +We describe here the code flow inside the VM when it executes the `CreateMiner` method (of the message sent by the `lotus-storage-miner` command included by a miner in a block). This execution will be the same seen by all participants in the Filecoin protocol, the miner including the message in the block, the full node syncing to it, and any other peer receiving also this message. + +There is a one-to-one mapping between the pair of actor and method number (`To:`/`Method:` fields) in a message in the VM, and the Go function in an actor's exported methods list that implement it. In this case, for the Power Actor list of method numbers defined in [`MethodsPower`](https://github.com/filecoin-project/specs-actors/blob/master/actors/builtin/methods.go), the `CreateMiner` method number 2 will correspond to the Go function with the same index in the list of methods returned by [`Exports()`](https://github.com/filecoin-project/specs-actors/blob/master/actors/builtin/power/power_actor.go) (and normally also the same name, here `(Actor).CreateMiner()`). + +The Power Actor in `CreateMiner()` will do two things: + +1. Send *another* message, `Exec`, to the Init Actor to instruct it to create the miner actor with the information provided by `lotus-storage-miner` and receive its ID address (this ID is the one returned to the CLI command). + +2. Generate an entry in its list of power claims ([`State.Claims`](https://github.com/filecoin-project/specs-actors/blob/master/actors/builtin/power/power_state.go)) for the newly created ID address of the miner. + +### Init Actor: create a new actor + +The [`Init`](https://github.com/filecoin-project/specs-actors/blob/master/actors/builtin/init/init_actor.go) actor is the only way to create actors (besides the initial actors included in the genesis block). It has a single method (besides its own constructor), `Exec`. An actor is nothing more than an entry in the `Init`'s [`State.AddressMap`](https://github.com/filecoin-project/specs-actors/blob/master/actors/builtin/init/init_actor_state.go), a [HAMT](https://en.wikipedia.org/wiki/Hash_array_mapped_trie) that maps an ([`Actor`](https://github.com/filecoin-project/specs-actors/blob/master/vendor/github.com/filecoin-project/go-address/address.go)) address to an ID one. The `Actor` address is a unique address provided by the `Runtime` ([`NewActorAddress()`](https://github.com/filecoin-project/lotus/blob/master/chain/vm/runtime.go)) derived from the public key of the message's origin address, in this case the `--owner` defined in the CLI command. + +### Miner Actor: constructor + +The Init actor will also call the constructor for the type of actor it created, in this case a [Miner](https://github.com/filecoin-project/specs-actors/blob/master/actors/builtin/miner/miner_actor.go). The constructor is always the method numbered 1 for any actor. + +FIXME: Do we want to say anything about the constructor? The setting of the proving period? Everything else seems like default values. From 0bb4c20e02acc5dbdd032b6e959049db3d9e6739 Mon Sep 17 00:00:00 2001 From: Brian Hoffman Date: Tue, 4 Aug 2020 11:33:29 -0400 Subject: [PATCH 2/7] Fix misspelling in HelloMessage --- node/hello/cbor_gen.go | 12 ++++++------ node/hello/hello.go | 12 ++++++------ 2 files changed, 12 insertions(+), 12 deletions(-) diff --git a/node/hello/cbor_gen.go b/node/hello/cbor_gen.go index 8a4f9a1bc..5bd4f51d3 100644 --- a/node/hello/cbor_gen.go +++ b/node/hello/cbor_gen.go @@ -172,13 +172,13 @@ func (t *LatencyMessage) MarshalCBOR(w io.Writer) error { scratch := make([]byte, 9) - // t.TArrial (int64) (int64) - if t.TArrial >= 0 { - if err := cbg.WriteMajorTypeHeaderBuf(scratch, w, cbg.MajUnsignedInt, uint64(t.TArrial)); err != nil { + // t.TArrival (int64) (int64) + if t.TArrival >= 0 { + if err := cbg.WriteMajorTypeHeaderBuf(scratch, w, cbg.MajUnsignedInt, uint64(t.TArrival)); err != nil { return err } } else { - if err := cbg.WriteMajorTypeHeaderBuf(scratch, w, cbg.MajNegativeInt, uint64(-t.TArrial-1)); err != nil { + if err := cbg.WriteMajorTypeHeaderBuf(scratch, w, cbg.MajNegativeInt, uint64(-t.TArrival-1)); err != nil { return err } } @@ -212,7 +212,7 @@ func (t *LatencyMessage) UnmarshalCBOR(r io.Reader) error { return fmt.Errorf("cbor input had wrong number of fields") } - // t.TArrial (int64) (int64) + // t.TArrival (int64) (int64) { maj, extra, err := cbg.CborReadHeaderBuf(br, scratch) var extraI int64 @@ -235,7 +235,7 @@ func (t *LatencyMessage) UnmarshalCBOR(r io.Reader) error { return fmt.Errorf("wrong type for int64 field: %d", maj) } - t.TArrial = int64(extraI) + t.TArrival = int64(extraI) } // t.TSent (int64) (int64) { diff --git a/node/hello/hello.go b/node/hello/hello.go index a2a5fbd4e..875edb422 100644 --- a/node/hello/hello.go +++ b/node/hello/hello.go @@ -32,8 +32,8 @@ type HelloMessage struct { GenesisHash cid.Cid } type LatencyMessage struct { - TArrial int64 - TSent int64 + TArrival int64 + TSent int64 } type NewStreamFunc func(context.Context, peer.ID, ...protocol.ID) (inet.Stream, error) @@ -84,8 +84,8 @@ func (hs *Service) HandleStream(s inet.Stream) { sent := time.Now() msg := &LatencyMessage{ - TArrial: arrived.UnixNano(), - TSent: sent.UnixNano(), + TArrival: arrived.UnixNano(), + TSent: sent.UnixNano(), } if err := cborutil.WriteCborRPC(s, msg); err != nil { log.Debugf("error while responding to latency: %v", err) @@ -169,8 +169,8 @@ func (hs *Service) SayHello(ctx context.Context, pid peer.ID) error { } if err == nil { - if lmsg.TArrial != 0 && lmsg.TSent != 0 { - t1 := time.Unix(0, lmsg.TArrial) + if lmsg.TArrival != 0 && lmsg.TSent != 0 { + t1 := time.Unix(0, lmsg.TArrival) t2 := time.Unix(0, lmsg.TSent) offset := t0.Sub(t1) + t3.Sub(t2) offset /= 2 From 14535cc4009ac1c3c96dfca5d64502f68e7ee9b6 Mon Sep 17 00:00:00 2001 From: vyzo Date: Tue, 4 Aug 2020 23:04:53 +0300 Subject: [PATCH 3/7] gomod: update go-libp2p --- go.mod | 2 +- go.sum | 8 ++++---- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/go.mod b/go.mod index b6e38fc2d..2b42b8d50 100644 --- a/go.mod +++ b/go.mod @@ -79,7 +79,7 @@ require ( github.com/kelseyhightower/envconfig v1.4.0 github.com/lib/pq v1.7.0 github.com/libp2p/go-eventbus v0.2.1 - github.com/libp2p/go-libp2p v0.10.2 + github.com/libp2p/go-libp2p v0.10.3 github.com/libp2p/go-libp2p-connmgr v0.2.4 github.com/libp2p/go-libp2p-core v0.6.1 github.com/libp2p/go-libp2p-discovery v0.5.0 diff --git a/go.sum b/go.sum index 41c8dd237..e6fbedfbd 100644 --- a/go.sum +++ b/go.sum @@ -745,8 +745,8 @@ github.com/libp2p/go-libp2p v0.8.3/go.mod h1:EsH1A+8yoWK+L4iKcbPYu6MPluZ+CHWI9El github.com/libp2p/go-libp2p v0.9.2/go.mod h1:cunHNLDVus66Ct9iXXcjKRLdmHdFdHVe1TAnbubJQqQ= github.com/libp2p/go-libp2p v0.10.0 h1:7ooOvK1wi8eLpyTppy8TeH43UHy5uI75GAHGJxenUi0= github.com/libp2p/go-libp2p v0.10.0/go.mod h1:yBJNpb+mGJdgrwbKAKrhPU0u3ogyNFTfjJ6bdM+Q/G8= -github.com/libp2p/go-libp2p v0.10.2 h1:VQOo/Pbj9Ijco9jiMYN5ImAg236IjTXfnUPJ2OvbpLM= -github.com/libp2p/go-libp2p v0.10.2/go.mod h1:BYckt6lmS/oA1SlRETSPWSUulCQKiZuTVsymVMc//HQ= +github.com/libp2p/go-libp2p v0.10.3 h1:Bc8/VjmC+pICtK6xG8YgVutZvCdK0MsroWCHP+6AdFQ= +github.com/libp2p/go-libp2p v0.10.3/go.mod h1:0ER6iPSaPeQjryNgOnm9bLNpMJCYmuw54xJXsVR17eE= github.com/libp2p/go-libp2p-autonat v0.0.2/go.mod h1:fs71q5Xk+pdnKU014o2iq1RhMs9/PMaG5zXRFNnIIT4= github.com/libp2p/go-libp2p-autonat v0.0.6/go.mod h1:uZneLdOkZHro35xIhpbtTzLlgYturpu4J5+0cZK3MqE= github.com/libp2p/go-libp2p-autonat v0.1.0/go.mod h1:1tLf2yXxiE/oKGtDwPYWTSYG3PtvYlJmg7NeVtPRqH8= @@ -756,8 +756,8 @@ github.com/libp2p/go-libp2p-autonat v0.2.1/go.mod h1:MWtAhV5Ko1l6QBsHQNSuM6b1sRk github.com/libp2p/go-libp2p-autonat v0.2.2/go.mod h1:HsM62HkqZmHR2k1xgX34WuWDzk/nBwNHoeyyT4IWV6A= github.com/libp2p/go-libp2p-autonat v0.2.3 h1:w46bKK3KTOUWDe5mDYMRjJu1uryqBp8HCNDp/TWMqKw= github.com/libp2p/go-libp2p-autonat v0.2.3/go.mod h1:2U6bNWCNsAG9LEbwccBDQbjzQ8Krdjge1jLTE9rdoMM= -github.com/libp2p/go-libp2p-autonat v0.3.1 h1:60sc3NuQz+RxEb4ZVCRp/7uPtD7gnlLcOIKYNulzSIo= -github.com/libp2p/go-libp2p-autonat v0.3.1/go.mod h1:0OzOi1/cVc7UcxfOddemYD5vzEqi4fwRbnZcJGLi68U= +github.com/libp2p/go-libp2p-autonat v0.3.2 h1:OhDSwVVaq7liTaRIsFFYvsaPp0pn2yi0WazejZ4DUmo= +github.com/libp2p/go-libp2p-autonat v0.3.2/go.mod h1:0OzOi1/cVc7UcxfOddemYD5vzEqi4fwRbnZcJGLi68U= github.com/libp2p/go-libp2p-autonat-svc v0.1.0/go.mod h1:fqi8Obl/z3R4PFVLm8xFtZ6PBL9MlV/xumymRFkKq5A= github.com/libp2p/go-libp2p-blankhost v0.0.1/go.mod h1:Ibpbw/7cPPYwFb7PACIWdvxxv0t0XCCI10t7czjAjTc= github.com/libp2p/go-libp2p-blankhost v0.1.1/go.mod h1:pf2fvdLJPsC1FsVrNP3DUUvMzUts2dsLLBEpo1vW1ro= From 323e07df4672e443225978c468a993ef93224f56 Mon Sep 17 00:00:00 2001 From: Your Name Date: Wed, 5 Aug 2020 23:26:40 +0800 Subject: [PATCH 4/7] fixed systemd inconsistencies --- Makefile | 91 ++++++++++---- documentation/en/install-systemd-services.md | 115 +++++++++++++++++- ...watch.service => lotus-chainwatch.service} | 0 3 files changed, 178 insertions(+), 28 deletions(-) rename scripts/{chainwatch.service => lotus-chainwatch.service} (100%) diff --git a/Makefile b/Makefile index 581c906db..a84a02330 100644 --- a/Makefile +++ b/Makefile @@ -98,26 +98,17 @@ an existing lotus binary in your PATH. This may cause problems if you don't run .PHONY: build -install: +install: install-daemon install-miner install-worker + +install-daemon: install -C ./lotus /usr/local/bin/lotus + +install-miner: install -C ./lotus-storage-miner /usr/local/bin/lotus-storage-miner + +install-worker: install -C ./lotus-seal-worker /usr/local/bin/lotus-seal-worker - -install-services: install - mkdir -p /usr/local/lib/systemd/system - mkdir -p /var/log/lotus - install -C -m 0644 ./scripts/lotus-daemon.service /usr/local/lib/systemd/system/lotus-daemon.service - install -C -m 0644 ./scripts/lotus-miner.service /usr/local/lib/systemd/system/lotus-miner.service - systemctl daemon-reload - @echo - @echo "lotus-daemon and lotus-miner services installed. Don't forget to 'systemctl enable lotus-daemon|lotus-miner' for it to be enabled on startup." - -clean-services: - rm -f /usr/local/lib/systemd/system/lotus-daemon.service - rm -f /usr/local/lib/systemd/system/lotus-miner.service - rm -f /usr/local/lib/systemd/system/chainwatch.service - systemctl daemon-reload - + # TOOLS lotus-seed: $(BUILD_DEPS) @@ -161,13 +152,6 @@ chainwatch: .PHONY: chainwatch BINS+=chainwatch -install-chainwatch-service: chainwatch - install -C ./chainwatch /usr/local/bin/chainwatch - install -C -m 0644 ./scripts/chainwatch.service /usr/local/lib/systemd/system/chainwatch.service - systemctl daemon-reload - @echo - @echo "chainwatch installed. Don't forget to 'systemctl enable chainwatch' for it to be enabled on startup." - bench: rm -f bench go build -o bench ./cmd/lotus-bench @@ -196,6 +180,65 @@ testground: .PHONY: testground BINS+=testground +install-chainwatch: chainwatch + install -C ./chainwatch /usr/local/bin/chainwatch + +# SYSTEMD + +install-daemon-service: install-daemon + mkdir -p /etc/systemd/system + mkdir -p /var/log/lotus + install -C -m 0644 ./scripts/lotus-daemon.service /etc/systemd/system/lotus-daemon.service + systemctl daemon-reload + @echo + @echo "lotus-daemon service installed. Don't forget to run 'sudo systemctl start lotus-daemon' to start it and 'sudo systemctl enable lotus-daemon' for it to be enabled on startup." + +install-miner-service: install-miner install-daemon-service + mkdir -p /etc/systemd/system + mkdir -p /var/log/lotus + install -C -m 0644 ./scripts/lotus-miner.service /etc/systemd/system/lotus-miner.service + systemctl daemon-reload + @echo + @echo "lotus-miner service installed. Don't forget to run 'sudo systemctl start lotus-miner' to start it and 'sudo systemctl enable lotus-miner' for it to be enabled on startup." + +install-chainwatch-service: install-chainwatch install-daemon-service + mkdir -p /etc/systemd/system + mkdir -p /var/log/lotus + install -C -m 0644 ./scripts/lotus-chainwatch.service /etc/systemd/system/lotus-chainwatch.service + systemctl daemon-reload + @echo + @echo "chainwatch service installed. Don't forget to run 'sudo systemctl start lotus-chainwatch' to start it and 'sudo systemctl enable lotus-chainwatch' for it to be enabled on startup." + +install-main-services: install-miner-service + +install-all-services: install-main-services install-chainwatch-service + +install-services: install-main-services + +clean-daemon-service: clean-miner-service clean-chainwatch-service + -systemctl stop lotus-daemon + -systemctl disable lotus-daemon + rm -f /etc/systemd/system/lotus-daemon.service + systemctl daemon-reload + +clean-miner-service: + -systemctl stop lotus-miner + -systemctl disable lotus-miner + rm -f /etc/systemd/system/lotus-miner.service + systemctl daemon-reload + +clean-chainwatch-service: + -systemctl stop chainwatch + -systemctl disable chainwatch + rm -f /etc/systemd/system/lotus-chainwatch.service + systemctl daemon-reload + +clean-main-services: clean-daemon-service + +clean-all-services: clean-main-services + +clean-services: clean-all-services + # MISC buildall: $(BINS) diff --git a/documentation/en/install-systemd-services.md b/documentation/en/install-systemd-services.md index 0bc23cfae..a18620ea7 100644 --- a/documentation/en/install-systemd-services.md +++ b/documentation/en/install-systemd-services.md @@ -1,17 +1,120 @@ # Use Lotus with systemd -Lotus is capable of running as a systemd service daemon. You can find installable service files for systemd in the [lotus repo scripts directory](https://github.com/filecoin-project/lotus/tree/master/scripts) as files with `.service` extension. In order to install these service files, you can copy these `.service` files to the default systemd service path. +Lotus is capable of running as a systemd service daemon. You can find installable service files for systemd in the [lotus repo scripts directory](https://github.com/filecoin-project/lotus/tree/master/scripts) as files with `.service` extension. In order to install these service files, you can copy these `.service` files to the default systemd unit load path. -## Installing via `make` +The services expect their binaries to be present in `/usr/local/bin/`. You can use `make` to install them by running: -NOTE: Before using lotus and lotus-miner as systemd services, don't forget to `sudo make install` to ensure the binaries are accessible by the root user. +```sh +$ sudo make install +``` -If your host uses the default systemd service path, it can be installed with `sudo make install-services`: +for `lotus(-daemon)` and `lotus-storage-miner` and + +```sh +$ sudo make install-chainwatch +``` + +for the `chainwatch` tool. + +## Installing services via `make` + +If your host uses the default systemd unit load path, the `lotus-daemon` and `lotus-miner` services can be installed by running: ```sh $ sudo make install-services ``` +To install the the `lotus-chainwatch` service run: + +```sh +$ sudo make install-chainwatch-service +``` + +You can install all services together by running: + +```sh +$ sudo make install-all-services +``` + +The `lotus-daemon` and the `lotus-miner` services can be installed individually too by running: + +```sh +$ sudo make install-daemon-service +``` + +and + +```sh +$ sudo make install-miner-service +``` + +### Notes + +When nstalling the `lotus-miner` and/or `lotus-chainwatch` service the `lotus-daemon` service gets automatically installed since the other two services depend on it being installed to run. + +All `install *service*` commands will install the latest binaries in the lotus build folders to `/usr/local/bin/`. If you do not want to use the latest build binaries please copy the `*.service` files by hand. + +## Removing via `make` + +All services can beremoved via `make`. To remove all services together run: + +```sh +$ sudo make clean-all-services +``` + +Individual services can be removed by running: + +```sh +$ sudo make clean-chainwatch-services +$ sudo make clean-miner-services +$ sudo make clean-daemon-services +``` + +### Notes + +Removing the `lotus-daemon` service will automatically remove the depending services `lotus-miner` and `lotus-chainwatch` + + +## Controlling services +### Start/Stop + +You can start the services by running: + +```sh +$ sudo systemctl start lotus-daemon +$ sudo systemctl start lotus-miner +$ sudo systemctl start lotus-chainwatch +``` + +and can be stopped by running: + +```sh +$ sudo systemctl stop lotus-daemon +$ sudo systemctl stop lotus-miner +$ sudo systemctl stop lotus-chainwatch +``` + +### Enabling services on startup + +To enable the services to run automatically on startup execute: + +```sh +$ sudo systemctl enable lotus-daemon +$ sudo systemctl enable lotus-miner +$ sudo systemctl enable lotus-chainwatch +``` + +To disable the services on startup run: + +```sh +$ sudo systemctl disable lotus-daemon +$ sudo systemctl disable lotus-miner +$ sudo systemctl disable lotus-chainwatch +``` +### Notes + +Systemd will not let services be enabled or started without their dependencies. Starting the `lotus-chainwatch` and/or `lotus-miner` service with automatically start the `lotus-daemon` service (if installed!). Stopping the `lotus-daemon` service will stop the other two services. The same pattern is executed for enabling and disabling the services. + ## Interacting with service logs Logs from the services can be reviewed using `journalctl`. @@ -27,3 +130,7 @@ $ sudo journalctl -u lotus-daemon -f ```sh $ sudo journalctl -u lotus-miner -r ``` + +### Log files + +Besides the systemd service logs all services save their own log files in `/var/log/lotus/`. diff --git a/scripts/chainwatch.service b/scripts/lotus-chainwatch.service similarity index 100% rename from scripts/chainwatch.service rename to scripts/lotus-chainwatch.service From 004a6d886861b7cfbc68bac1a217367366ae1404 Mon Sep 17 00:00:00 2001 From: Your Name Date: Thu, 6 Aug 2020 17:41:00 +0800 Subject: [PATCH 5/7] add more documentation --- documentation/en/install-systemd-services.md | 21 ++++++++++++++------ 1 file changed, 15 insertions(+), 6 deletions(-) diff --git a/documentation/en/install-systemd-services.md b/documentation/en/install-systemd-services.md index a18620ea7..5d6348413 100644 --- a/documentation/en/install-systemd-services.md +++ b/documentation/en/install-systemd-services.md @@ -8,7 +8,7 @@ The services expect their binaries to be present in `/usr/local/bin/`. You can u $ sudo make install ``` -for `lotus(-daemon)` and `lotus-storage-miner` and +for `lotus` and `lotus-storage-miner` and ```sh $ sudo make install-chainwatch @@ -50,9 +50,9 @@ $ sudo make install-miner-service ### Notes -When nstalling the `lotus-miner` and/or `lotus-chainwatch` service the `lotus-daemon` service gets automatically installed since the other two services depend on it being installed to run. +When installing the `lotus-miner` and/or `lotus-chainwatch` service the `lotus-daemon` service gets automatically installed since the other two services depend on it being installed to run. -All `install *service*` commands will install the latest binaries in the lotus build folders to `/usr/local/bin/`. If you do not want to use the latest build binaries please copy the `*.service` files by hand. +All `install-*-service*` commands will install the latest binaries in the lotus build folders to `/usr/local/bin/`. If you do not want to use the latest build binaries please copy the `*.service` files by hand. ## Removing via `make` @@ -72,11 +72,20 @@ $ sudo make clean-daemon-services ### Notes -Removing the `lotus-daemon` service will automatically remove the depending services `lotus-miner` and `lotus-chainwatch` +The services will be stoppend and disabled when removed. + +Removing the `lotus-daemon` service will automatically remove the depending services `lotus-miner` and `lotus-chainwatch`. ## Controlling services -### Start/Stop + +All service can be controlled with the `systemctl`. A few basic control commands are listed below. To get detailed infos about the cpabilities of the `systemctl` command please consult your distributions man pages by running: + +```sh +$ man systemctl +``` + +### Start/Stop services You can start the services by running: @@ -113,7 +122,7 @@ $ sudo systemctl disable lotus-chainwatch ``` ### Notes -Systemd will not let services be enabled or started without their dependencies. Starting the `lotus-chainwatch` and/or `lotus-miner` service with automatically start the `lotus-daemon` service (if installed!). Stopping the `lotus-daemon` service will stop the other two services. The same pattern is executed for enabling and disabling the services. +Systemd will not let services be enabled or started without their requirements. Starting the `lotus-chainwatch` and/or `lotus-miner` service with automatically start the `lotus-daemon` service (if installed!). Stopping the `lotus-daemon` service will stop the other two services. The same pattern is executed for enabling and disabling the services. ## Interacting with service logs From bd15650724c1acf61089670565fd4c6b4c814158 Mon Sep 17 00:00:00 2001 From: Your Name Date: Thu, 6 Aug 2020 17:56:37 +0800 Subject: [PATCH 6/7] fixed typos --- documentation/en/install-systemd-services.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/documentation/en/install-systemd-services.md b/documentation/en/install-systemd-services.md index 5d6348413..fbde1feec 100644 --- a/documentation/en/install-systemd-services.md +++ b/documentation/en/install-systemd-services.md @@ -79,7 +79,7 @@ Removing the `lotus-daemon` service will automatically remove the depending serv ## Controlling services -All service can be controlled with the `systemctl`. A few basic control commands are listed below. To get detailed infos about the cpabilities of the `systemctl` command please consult your distributions man pages by running: +All service can be controlled with the `systemctl`. A few basic control commands are listed below. To get detailed infos about the capabilities of the `systemctl` command please consult your distributions man pages by running: ```sh $ man systemctl From 429f119dbb09edd8ddd32ba5f33e6773a5b72eac Mon Sep 17 00:00:00 2001 From: Travis Person Date: Thu, 6 Aug 2020 16:01:30 +0000 Subject: [PATCH 7/7] release: write full sha512sum output to file With the full output written to the file, users can verify the tar.gz integrity with `sha512sum -c .tar.gz.sha512`. --- scripts/build-bundle.sh | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/scripts/build-bundle.sh b/scripts/build-bundle.sh index 16cab49d1..d4837dd30 100755 --- a/scripts/build-bundle.sh +++ b/scripts/build-bundle.sh @@ -45,8 +45,8 @@ do popd rm -rf "${ARCH}" - sha512sum "lotus_${CIRCLE_TAG}_${ARCH}-amd64.tar.gz" | cut -d" " -f1 > "lotus_${CIRCLE_TAG}_${ARCH}-amd64.tar.gz.sha512" + sha512sum "lotus_${CIRCLE_TAG}_${ARCH}-amd64.tar.gz" > "lotus_${CIRCLE_TAG}_${ARCH}-amd64.tar.gz.sha512" - ipfs add "lotus_${CIRCLE_TAG}_${ARCH}-amd64.tar.gz" | cut -d" " -f2 > "lotus_${CIRCLE_TAG}_${ARCH}-amd64.tar.gz.cid" + ipfs add -q "lotus_${CIRCLE_TAG}_${ARCH}-amd64.tar.gz" > "lotus_${CIRCLE_TAG}_${ARCH}-amd64.tar.gz.cid" done popd