Merge PR #4691: Add baseapp concept doc

* consolidate intro * start anatomy of sdk app * wokring * working * querier * working * workiiiing * finish * add dep and makefile * Apply suggestions from code review Co-Authored-By: Alessio Treglia <quadrispro@ubuntu.com> * typo * typo * Apply suggestions from code review Co-Authored-By: Alexander Bezobchuk <alexanderbez@users.noreply.github.com> Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com> Co-Authored-By: Alessio Treglia <quadrispro@ubuntu.com> Co-Authored-By: frog power 4000 <rigel.rozanski@gmail.com> * refactor for new module interface * karoly review * Apply suggestions from code review Co-Authored-By: Karoly Albert Szabo <szabo.karoly.a@gmail.com> Co-Authored-By: Federico Kunze <31522760+fedekunze@users.noreply.github.com> * encoding * working on baseapp doc * baseapp work * reorg * almost there * finish first draft * remove old files * hans review' * jack review + clarification on ABCI methods
2019-07-17 09:26:48 -07:00
parent 8c7ed19251
commit ae77f0080a
52 changed files with 362 additions and 10 deletions
@@ -0,0 +1,22 @@
+# Getting Started
+
+To start a REST server, we need to specify the following parameters:
+
+| Parameter   | Type      | Default                 | Required | Description                                          |
+| ----------- | --------- | ----------------------- | -------- | ---------------------------------------------------- |
+| chain-id    | string    | null                    | true     | chain id of the full node to connect                 |
+| node        | URL       | "tcp://localhost:46657" | true     | address of the full node to connect                  |
+| laddr       | URL       | "tcp://localhost:1317"  | true     | address to run the rest server on                    |
+| trust-node  | bool      | "false"                 | true     | Whether this LCD is connected to a trusted full node |
+| trust-store | DIRECTORY | "$HOME/.lcd"            | false    | directory for save checkpoints and validator sets    |
+
+For example:
+
+```bash
+gaiacli rest-server --chain-id=test \
+    --laddr=tcp://localhost:1317 \
+    --node tcp://localhost:26657 \
+    --trust-node=false
+```
+
+For more information about the Gaia-Lite RPC, see the [swagger documentation](https://cosmos.network/rpc/)
@@ -0,0 +1,86 @@
+# Light Client Overview
+
+**See the Cosmos SDK Light Client RPC documentation [here](https://cosmos.network/rpc/)**
+
+## Introduction
+
+A light client allows clients, such as mobile phones, to receive proofs of the state of the
+blockchain from any full node. Light clients do not have to trust any full node, since they are able
+to verify any proof they receive.
+
+A light client can provide the same security as a full node with minimal requirements for
+bandwidth, computing and storage resource. It can also provide modular functionality
+according to users' configuration. These features allow developers to build secure, efficient,
+and usable mobile apps, websites, and other applications without deploying or
+maintaining any full blockchain nodes.
+
+### What is a Light Client?
+
+The Cosmos SDK Light Client (Gaia-lite) is split into two separate components. The first component is generic for
+any Tendermint-based application. It handles the security and connectivity aspects of following the header
+chain and verify proofs from full nodes against a locally trusted validator set. Furthermore, it exposes the same
+API as any Tendermint Core node. The second component is specific for the Cosmos Hub (`gaiad`). It works as a query
+endpoint and exposes the application specific functionality, which can be arbitrary. All queries against the
+application state must go through the query endpoint. The advantage of the query endpoint is that it can verify
+the proofs that the application returns.
+
+### High-Level Architecture
+
+An application developer that wants to build a third party client application for the Cosmos Hub (or any
+other zone) should build it against its canonical API. That API is a combination of multiple parts.
+All zones have to expose ICS0 (TendermintAPI). Beyond that any zone is free to choose any
+combination of module APIs, depending on which modules the state machine uses. The Cosmos Hub will
+initially support [ICS0](https://cosmos.network/rpc/#/ICS0) (TendermintAPI), [ICS1](https://cosmos.network/rpc/#/ICS1) (KeyAPI), [ICS20](https://cosmos.network/rpc/#/ICS20) (TokenAPI), [ICS21](https://cosmos.network/rpc/#/ICS21) (StakingAPI),
+[ICS22](https://cosmos.network/rpc/#/ICS22) (GovernanceAPI) and [ICS23](https://cosmos.network/rpc/#/ICS23) (SlashingAPI).
+
+![high-level](./pics/high-level.png)
+
+All applications are expected to run only against Gaia-lite. Gaia-lite is the only piece of software
+that offers stability guarantees around the zone API.
+
+### Comparison
+
+A full node of ABCI is different from a light client in the following ways:
+
+|| Full Node | Gaia-lite | Description|
+|-| ------------- | ----- | -------------- |
+| Execute and verify transactions|Yes|No|A full node will execute and verify all transactions while Gaia-lite won't.|
+| Verify and save blocks|Yes|No|A full node will verify and save all blocks while Gaia-lite won't.|
+| Consensus participation|Yes|No|Only when a full node is a validator will it participate in consensus. Lite nodes never participate in consensus.|
+| Bandwidth cost|High|Low|A full node will receive all blocks. If bandwidth is limited, it will fall behind the main network. What's more, if it happens to be a validator, it will slow down the consensus process. Light clients require little bandwidth, only when serving local requests.|
+| Computing resources|High|Low|A full node will execute all transactions and verify all blocks, which requires considerable computing resources.|
+| Storage resources|High|Low|A full node will save all blocks and ABCI states. Gaia-lite just saves validator sets and some checkpoints.|
+| Power consumption|High|Low|Full nodes must be deployed on machines which have high performance and will be running all the time. Gaia-lite can be deployed on the same machines as users' applications, or on independent machines but with lower performance. Light clients can be shut down anytime when necessary. Gaia-lite consumes very little power, so even mobile devices can meet the power requirements.|
+| Provide APIs|All cosmos APIs|Modular APIs|A full node supports all Cosmos APIs. Gaia-lite provides modular APIs according to users' configuration.|
+| Secuity level| High|High|A full node will verify all transactions and blocks by itself. A light client can't do this, but it can query data from other full nodes and verify the data independently. Therefore, both full nodes and light clients don't need to trust any third nodes and can achieve high security.|
+
+According to the above table, Gaia-lite can meet many users' functionality and security requirements, but require little bandwidth, computing, storage, and power.
+
+## Achieving Security
+
+### Trusted Validator Set
+
+The base design philosophy of Gaia-lite follows two rules:
+
+1. **Doesn't trust any blockchain nodes, including validator nodes and other full nodes**
+2. **Only trusts the whole validator set**
+
+The original trusted validator set should be prepositioned into its trust store. Usually this
+validator set comes from a genesis file. During runtime, if Gaia-lite detects a different validator set,
+it will verify it and save the new validated validator set to the trust store.
+
+![validator-set-change](./pics/validatorSetChange.png)
+
+### Trust Propagation
+
+From the above section, we come to know how to get a trusted validator set and how lcd keeps track of
+validator set evolution. The validator set is the foundation of trust, and the trust can propagate to
+other blockchain data, such as blocks and transactions. The propagation architecture is shown as
+
+follows:
+
+![change-process](./pics/trustPropagate.png)
+
+In general, with a trusted validator set, a light client can verify each block commit which contains all pre-commit
+data and block header data. Then the block hash, data hash and appHash are trusted. Based on this
+and merkle proof, all transactions data and ABCI states can be verified too.
@@ -0,0 +1,209 @@
+# Specifications
+
+This specification describes how to implement the LCD. LCD supports modular APIs. Currently, only
+ICS0 (TendermintAPI), ICS1 (Key API) and ICS20 (Token API) are supported. Later, if necessary, more
+APIs can be included.
+
+## Build and Verify Proof of ABCI States
+
+As we all know,  storage of cosmos-sdk based application contains multi-substores. Each substore is
+implemented by a IAVL store. These substores are organized by simple Merkle tree. To build the tree,
+we need to extract name, height and store root hash from these substores to build a set of simple
+Merkle leaf nodes, then calculate hash from leaf nodes to root. The root hash of the simple Merkle
+tree is the AppHash which will be included in block header.
+
+![Simple Merkle Tree](./pics/simpleMerkleTree.png)
+
+As we have discussed in [LCD trust-propagation](https://github.com/irisnet/cosmos-sdk/tree/bianjie/lcd_spec/docs/spec/lcd#trust-propagation),
+the AppHash can be verified by checking voting power against a trusted validator set. Here we just
+need to build proof from ABCI state to AppHash. The proof contains two parts:
+
+* IAVL proof
+* Substore to AppHash proof
+
+### IAVL Proof
+
+The proof has two types: existence proof and absence proof. If the query key exists in the IAVL
+store, then it returns key-value and its existence proof. On the other hand, if the key doesn't
+exist, then it only returns absence proof which can demonstrate the key definitely doesn't exist.
+
+### IAVL Existence Proof
+
+```go
+type CommitID struct {
+    Version int64
+    Hash    []byte
+}
+
+type storeCore struct {
+    CommitID CommitID
+}
+
+type MultiStoreCommitID struct {
+    Name string
+    Core storeCore
+}
+
+type proofInnerNode struct {
+    Height  int8
+    Size    int64
+    Version int64
+    Left    []byte
+    Right   []byte
+}
+
+type KeyExistsProof struct {
+    MultiStoreCommitInfo []MultiStoreCommitID //All substore commitIDs
+    StoreName string //Current substore name
+    Height  int64 //The commit height of current substore
+    RootHash cmn.HexBytes //The root hash of this IAVL tree
+    Version  int64 //The version of the key-value in this IAVL tree
+    InnerNodes []proofInnerNode //The path from to root node to key-value leaf node
+}
+```
+
+The data structure of exist proof is shown as above. The process to build and verify existence proof
+is shown as follows:
+
+![Exist Proof](./pics/existProof.png)
+
+Steps to build proof:
+
+* Access the IAVL tree from the root node.
+* Record the visited nodes in InnerNodes,
+* Once the target leaf node is found, assign leaf node version to proof version
+* Assign the current IAVL tree height to proof height
+* Assign the current IAVL tree rootHash to proof rootHash
+* Assign the current substore name to proof StoreName
+* Read multistore commitInfo from db by height and assign it to proof StoreCommitInfo
+
+Steps to verify proof:
+
+* Build leaf node with key, value and proof version.
+* Calculate leaf node hash
+* Assign the hash to the first innerNode's rightHash, then calculate first innerNode hash
+* Propagate the hash calculation process. If prior innerNode is the left child of next innerNode, then assign the prior innerNode hash to the left hash of next innerNode. Otherwise, assign the prior innerNode hash to the right hash of next innerNode.
+* The hash of last innerNode should be equal to the rootHash of this proof. Otherwise, the proof is invalid.
+
+### IAVL Absence Proof
+
+As we all know, all IAVL leaf nodes are sorted by the key of each leaf nodes. So we can calculate
+the position of the target key in the whole key set of this IAVL tree. As shown below, we can find
+out the left key and the right key. If we can demonstrate that both left key and right key
+definitely exist, and they are adjacent nodes. Thus the target key definitely doesn't exist.
+
+![Absence Proof1](./pics/absence1.png)
+
+If the target key is larger than the right most leaf node or less than the left most key, then the
+target key definitely doesn't exist.
+
+![Absence Proof2](./pics/absence2.png)![Absence Proof3](./pics/absence3.png)
+
+```go
+type proofLeafNode struct {
+    KeyBytes   cmn.HexBytes
+    ValueBytes cmn.HexBytes
+    Version    int64
+}
+
+type pathWithNode struct {
+    InnerNodes []proofInnerNode
+    Node proofLeafNode
+}
+
+type KeyAbsentProof struct {
+    MultiStoreCommitInfo []MultiStoreCommitID
+    StoreName string
+    Height  int64
+    RootHash cmn.HexBytes
+    Left  *pathWithNode // Proof the left key exist
+    Right *pathWithNode  //Proof the right key exist
+}
+```
+
+The above is the data structure of absence proof. Steps to build proof:
+
+* Access the IAVL tree from the root node.
+* Get the deserved index(Marked as INDEX) of the key in whole key set.
+* If the returned index equals to 0, the right index should be 0 and left node doesn't exist
+* If the returned index equals to the size of the whole key set, the left node index should be INDEX-1 and the right node doesn't exist.
+* Otherwise, the right node index should be INDEX and the left node index should be INDEX-1
+* Assign the current IAVL tree height to proof height
+* Assign the current IAVL tree rootHash to proof rootHash
+* Assign the current substore name to proof StoreName
+* Read multistore commitInfo from db by height and assign it to proof StoreCommitInfo
+
+Steps to verify proof:
+
+* If only right node exist, verify its exist proof and verify if it is the left most node
+* If only left node exist, verify its exist proof and verify if it is the right most node.
+* If both right node and left node exist, verify if they are adjacent.
+
+### Substores to AppHash Proof
+
+After verify the IAVL proof, then we can start to verify substore proof against AppHash. Firstly,
+iterate MultiStoreCommitInfo and find the substore commitID by proof StoreName. Verify if yhe Hash
+in commitID equals to proof RootHash. If not, the proof is invalid. Then sort the substore
+commitInfo array by the hash of substore name. Finally, build the simple Merkle tree with all
+substore commitInfo array and verify if the Merkle root hash equal to appHash.
+
+![substore proof](./pics/substoreProof.png)
+
+```go
+func SimpleHashFromTwoHashes(left []byte, right []byte) []byte {
+    var hasher = ripemd160.New()
+
+    err := encodeByteSlice(hasher, left)
+    if err != nil {
+        panic(err)
+    }
+
+    err = encodeByteSlice(hasher, right)
+    if err != nil {
+        panic(err)
+    }
+
+    return hasher.Sum(nil)
+}
+
+func SimpleHashFromHashes(hashes [][]byte) []byte {
+    // Recursive impl.
+    switch len(hashes) {
+        case 0:
+            return nil
+        case 1:
+            return hashes[0]
+        default:
+            left := SimpleHashFromHashes(hashes[:(len(hashes)+1)/2])
+            right := SimpleHashFromHashes(hashes[(len(hashes)+1)/2:])
+            return SimpleHashFromTwoHashes(left, right)
+    }
+}
+```
+
+## Verify block header against validator set
+
+Above sections refer appHash frequently. But where does the trusted appHash come from? Actually,
+the appHash exist in block header, next we need to verify blocks header at specific height against
+LCD trusted validator set. The validation flow is shown as follows:
+
+![commit verification](./pics/commitValidation.png)
+
+When the trusted validator set doesn't match the block header, we need to try to update our
+validator set to the height of this block. LCD has a rule that each validator set change should not
+affect more than 1/3 voting power. Compare with the trusted validator set, if the voting power of
+target validator set changes more than 1/3. We have to verify if there are hidden validator set
+changes before the target validator set. Only when all validator set changes obey this rule, can our
+validator set update be accomplished.
+
+For instance:
+
+![Update validator set to height](./pics/updateValidatorToHeight.png)
+
+* Update to 10000, tooMuchChangeErr
+* Update to 5050,  tooMuchChangeErr
+* Update to 2575, Success
+* Update to 5050, Success
+* Update to 10000,tooMuchChangeErr
+* Update to 7525, Success
+* Update to 10000, Success