LaconicNetwork/cosmos-sdk
7
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 2 Wiki Activity

Merge branch 'adr-epoched-staking' of github.com:sikkatech/cosmos-sdk into adr-epoched-staking

Browse Source
This commit is contained in:
ValarDragon
2021-02-15 13:24:55 -06:00
parent 352a19e6eb 3a7cc3bd3a
commit f4d328cc27
1195 changed files with 102977 additions and 24921 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/README.md
+9 -4
View File
@@ -43,18 +43,22 @@ Read about the [PROCESS](./PROCESS.md).
- [ADR 001: Coin Source Tracing](./adr-001-coin-source-tracing.md)
- [ADR 002: SDK Documentation Structure](./adr-002-docs-structure.md)
- [ADR 004: Split Denomination Keys](./adr-004-split-denomination-keys.md)
- [ADR 006: Secret Store Replacement](./adr-006-secret-store-replacement.md)
- [ADR 009: Evidence Module](./adr-009-evidence-module.md)
- [ADR 010: Modular AnteHandler](./adr-010-modular-antehandler.md)
- [ADR 019: Protocol Buffer State Encoding](./adr-019-protobuf-state-encoding.md)
- [ADR 020: Protocol Buffer Transaction Encoding](./adr-020-protobuf-transaction-encoding.md)
- [ADR 021: Protocol Buffer Query Encoding](./adr-021-protobuf-query-encoding.md)
- [ADR 023: Protocol Buffer Naming and Versioning](./adr-023-protobuf-naming.md)
- [ADR 026: IBC Client Recovery Mechanisms](./adr-026-ibc-client-recovery-mechanisms.md)
- [ADR 029: Fee Grant Module](./adr-029-fee-grant-module.md)
- [ADR 030: Message Authorization Module](architecture/adr-030-authz-module.md)
- [ADR 031: Protobuf Msg Services](./adr-031-msg-service.md)
### Proposed
- [ADR 003: Dynamic Capability Store](./adr-003-dynamic-capability-store.md)
- [ADR 004: Split Denomination Keys](./adr-004-split-denomination-keys.md)
- [ADR 011: Generalize Genesis Accounts](./adr-011-generalize-genesis-accounts.md)
- [ADR 012: State Accessors](./adr-012-state-accessors.md)
- [ADR 013: Metrics](./adr-013-metrics.md)
@@ -62,12 +66,13 @@ Read about the [PROCESS](./PROCESS.md).
- [ADR 016: Validator Consensus Key Rotation](./adr-016-validator-consensus-key-rotation.md)
- [ADR 017: Historical Header Module](./adr-017-historical-header-module.md)
- [ADR 018: Extendable Voting Periods](./adr-018-extendable-voting-period.md)
- [ADR 021: Protocol Buffer Query Encoding](./adr-021-protobuf-query-encoding.md)
- [ADR 022: Custom baseapp panic handling](./adr-022-custom-panic-handling.md)
- [ADR 023: Protocol Buffer Naming and Versioning](./adr-023-protobuf-naming.md)
- [ADR 024: Coin Metadata](./adr-024-coin-metadata.md)
- [ADR 025: IBC Passive Channels](./adr-025-ibc-passive-channels.md)
- [ADR 027: Deterministic Protobuf Serialization](./adr-027-deterministic-protobuf-serialization.md)
- [ADR 028: Public Key Addresses](./adr-028-public-key-addresses.md)
- [ADR 031: Protobuf Msg Services](./adr-031-msg-service.md)
- [ADR 032: Typed Events](./adr-032-typed-events.md)
- [ADR 035: Rosetta API Support](./adr-035-rosetta-api-support.md)
- [ADR 037: Governance Split Votes](./adr-037-gov-split-vote.md)
- [ADR 038: State Listening](./adr-038-state-listening.md)
- [ADR 039: Epoched Staking](./adr-039-epoched-staking.md)
docs/architecture/adr-004-split-denomination-keys.md
+1 -1
View File
@@ -96,7 +96,7 @@ the balances and check that they match the expected total supply.
## Status
Proposed.
Accepted.
## Consequences
docs/architecture/adr-019-protobuf-state-encoding.md
+8 -11
View File
@@ -6,6 +6,7 @@
- 2020 Feb 24: Updates to handle messages with interface fields
- 2020 Apr 27: Convert usages of `oneof` for interfaces to `Any`
- 2020 May 15: Describe `cosmos_proto` extensions and amino compatibility
- 2020 Dec 4: Move and rename `MarshalAny` and `UnmarshalAny` into the `codec.Marshaler` interface.
## Status
@@ -221,23 +222,20 @@ every module that implements it in order to populate the `InterfaceRegistry`.
### Using `Any` to encode state
The SDK will provide support methods `MarshalAny` and `UnmarshalAny` to allow
easy encoding of state to `Any` in `Codec` implementations. Ex:
The SDK will provide support methods `MarshalInterface` and `UnmarshalInterface` to hide a complexity of wrapping interface types into `Any` and allow easy serialization.
```go
import "github.com/cosmos/cosmos-sdk/codec"
func (c *Codec) MarshalEvidence(evidenceI eviexported.Evidence) ([]byte, error) {
return codec.MarshalAny(evidenceI)
// note: eviexported.Evidence is an interface type
func MarshalEvidence(cdc codec.BinaryMarshaler, e eviexported.Evidence) ([]byte, error) {
return cdc.MarshalInterface(e)
}
func (c *Codec) UnmarshalEvidence(bz []byte) (eviexported.Evidence, error) {
func UnmarshalEvidence(cdc codec.BinaryMarshaler, bz []byte) (eviexported.Evidence, error) {
var evi eviexported.Evidence
err := codec.UnmarshalAny(c.interfaceContext, &evi, bz)
if err != nil {
return nil, err
}
return evi, nil
err := cdc.UnmarshalInterface(&evi, bz)
return err, nil
}
```
@@ -375,4 +373,3 @@ seamless.
1. https://github.com/cosmos/cosmos-sdk/issues/4977
2. https://github.com/cosmos/cosmos-sdk/issues/5444
docs/architecture/adr-021-protobuf-query-encoding.md
+1 -1
View File
@@ -6,7 +6,7 @@
## Status
Proposed
Accepted
## Context
docs/architecture/adr-023-protobuf-naming.md
+1 -1
View File
@@ -7,7 +7,7 @@
## Status
Proposed
Accepted
## Context
docs/architecture/adr-024-coin-metadata.md
+2 -1
View File
@@ -34,7 +34,7 @@ UX and remove the requirement for making any assumptions on the unit of denomina
The `x/bank` module will be updated to store and index metadata by `denom`, specifically the "base" or
smallest unit -- the unit the Cosmos SDK state-machine works with.
Metadata may also include a non-zero length list of denominations. Each entry containts the name of
Metadata may also include a non-zero length list of denominations. Each entry contains the name of
the denomination `denom`, the exponent to the base and a list of aliases. An entry is to be
interpreted as `1 denom = 10^exponent base_denom` (e.g. `1 ETH = 10^18 wei` and `1 uatom = 10^0 uatom`).
@@ -92,6 +92,7 @@ As an example, the ATOM's metadata can be defined as follows:
```
Given the above metadata, a client may infer the following things:
- 4.3atom = 4.3 * (10^6) = 4,300,000uatom
- The string "atom" can be used as a display name in a list of tokens.
- The balance 4300000 can be displayed as 4,300,000uatom or 4,300matom or 4.3atom.
docs/architecture/adr-026-ibc-client-recovery-mechanisms.md
+14 -7
View File
@@ -4,10 +4,11 @@
- 2020/06/23: Initial version
- 2020/08/06: Revisions per review & to reference version
- 2021/01/15: Revision to support substitute clients for unfreezing
## Status
*Proposed*
*Accepted*
## Context
@@ -36,18 +37,22 @@ We elect not to deal with chains which have actually halted, which is necessaril
1. `allow_governance_override_after_expiry` (boolean, default false)
1. Require Tendermint light clients (ICS 07) to expose the following additional internal query functions
1. `Expired() boolean`, which returns whether or not the client has passed the trusting period since the last update (in which case no headers can be validated)
1. Require Tendermint light clients (ICS 07) to expose the following additional state mutation functions
1. `Unfreeze()`, which unfreezes a light client after misbehaviour and clears any frozen height previously set
1. Require Tendermint light clients (ICS 07) & solo machine clients (ICS 06) to be created with the following additional flags
1. `allow_governance_override_after_misbehaviour` (boolean, default false)
1. Require Tendermint light clients (ICS 07) to expose the following additional state mutation functions
1. `Unfreeze()`, which unfreezes a light client after misbehaviour and clears any frozen height previously set
1. Add a new governance proposal type, `ClientUpdateProposal`, in the `x/ibc` module
1. Extend the base `Proposal` with a client identifier (`string`) and a header (`bytes`, encoded in a client-type-specific format)
1. If this governance proposal passes, the client is updated with the provided header, if and only if:
1. Extend the base `Proposal` with two client identifiers (`string`) and an initial height ('exported.Height').
1. The first client identifier is the proposed client to be updated. This client must be either frozen or expired.
1. The second client is a substitute client. It carries all the state for the client which may be updated. It must have identitical client and chain parameters to the client which may be updated (except for latest height and frozen height). It should be continually updated during the voting period.
1. The initial height represents the starting height consensus states which will be copied from the substitute client to the frozen/expired client.
1. If this governance proposal passes, the client on trial will be updated with all the state of the substitute, if and only if:
1. `allow_governance_override_after_expiry` is true and the client has expired (`Expired()` returns true)
1. `allow_governance_override_after_misbehaviour` is true and the client has been frozen (`Frozen()` returns true)
1. In this case, additionally, the client is unfrozen by calling `Unfreeze()`
Note additionally that the header submitted by governance must be new enough that it will be possible to update the light client after the new header is inserted into the client state (which will only happen after the governance proposal has passed).
Note that clients frozen due to misbehaviour must wait for the evidence to expire to avoid becoming refrozen.
This ADR does not address planned upgrades, which are handled separately as per the [specification](https://github.com/cosmos/ics/tree/master/spec/ics-007-tendermint-client#upgrades).
@@ -58,11 +63,13 @@ This ADR does not address planned upgrades, which are handled separately as per
- Establishes a mechanism for client recovery in the case of expiry
- Establishes a mechanism for client recovery in the case of misbehaviour
- Clients can elect to disallow this recovery mechanism if they do not wish to allow for it
- Constructing an ClientUpdate Proposal is as difficult as creating a new client
### Negative
- Additional complexity in client creation which must be understood by the user
- Governance participants must pick a new header, which is a bit different from their usual tasks
- Coping state of the substitute adds complexity
- Governance participants must vote on a substitute client
### Neutral
docs/architecture/adr-028-public-key-addresses.md
+250 -90
View File
@@ -3,6 +3,7 @@
## Changelog
- 2020/08/18: Initial version
- 2021/01/15: Analysis and algorithm update
## Status
@@ -10,42 +11,81 @@ Proposed
## Abstract
This ADR defines a canonical 20-byte address format for new public key algorithms, multisig public keys, and module
accounts using string prefixes.
This ADR defines an address format for all addressable SDK accounts. That includes: new public key algorithms, multisig public keys, and module accounts.
## Context
Issue [\#3685](https://github.com/cosmos/cosmos-sdk/issues/3685) identified that public key
address spaces are currently overlapping. One initial proposal was extending the address length and
adding prefixes for different types of addresses.
address spaces are currently overlapping. We confirmed that it significantly decreases security of Cosmos SDK.
### Problem
An attacker can control an input for an address generation function. This leads to a birthday attack, which significantly decreases the security space.
To overcome this, we need to separate the inputs for different kind of account types:
a security break of one account type shouldn't impact the security of other account types.
### Initial proposals
One initial proposal was extending the address length and
adding prefixes for different types of addresses.
@ethanfrey explained an alternate approach originally used in https://github.com/iov-one/weave:
> I spent quite a bit of time thinking about this issue while building weave... The other cosmos Sdk.
> Basically I define a condition to be a type and format as human readable string with some binary data appended. This condition is hashed into an Address (again at 20 bytes). The use of this prefix makes it impossible to find a preimage for a given address with a different condition (eg ed25519 vs secp256k1).
> This is explained in depth here https://weave.readthedocs.io/en/latest/design/permissions.html
> And the code is here, look mainly at the top where we process conditions. https://github.com/iov-one/weave/blob/master/conditions.go
And explained how this approach should be sufficiently collision resistant:
> Yeah, AFAIK, 20 bytes should be collision resistance when the preimages are unique and not malleable. A space of 2^160 would expect some collision to be likely around 2^80 elements (birthday paradox). And if you want to find a collision for some existing element in the database, it is still 2^160. 2^80 only is if all these elements are written to state.
> The good example you brought up was eg. a public key bytes being a valid public key on two algorithms supported by the codec. Meaning if either was broken, you would break accounts even if they were secured with the safer variant. This is only as the issue when no differentiating type info is present in the preimage (before hashing into an address).
> I would like to hear an argument if the 20 bytes space is an actual issue for security, as I would be happy to increase my address sizes in weave. I just figured cosmos and ethereum and bitcoin all use 20 bytes, it should be good enough. And the arguments above which made me feel it was secure. But I have not done a deeper analysis.
In discussions in [\#5694](https://github.com/cosmos/cosmos-sdk/issues/5694), we agreed to go with an
approach similar to this where essentially we take the first 20 bytes of the `sha256` hash of
the key type concatenated with the key bytes, summarized as `Sha256(KeyTypePrefix || Keybytes)[:20]`.
This led to the first proposal (which we proved to be not good enough):
we concatenate a key type with a public key, hash it and take the first 20 bytes of that hash, summarized as `sha256(keyTypePrefix || keybytes)[:20]`.
### Review and Discussions
In [\#5694](https://github.com/cosmos/cosmos-sdk/issues/5694) we discussed various solutions.
We agreed that 20 bytes it's not future proof, and extending the address length is the only way to allow addresses of different types, various signature types, etc.
This disqualifies the initial proposal.
In the issue we discussed various modifications:
+ Choice of the hash function.
+ Move the prefix out of the hash function: `keyTypePrefix + sha256(keybytes)[:20]` [post-hash-prefix-proposal].
+ Use double hashing: `sha256(keyTypePrefix + sha256(keybytes)[:20])`.
+ Increase to keybytes hash slice from 20 byte to 32 or 40 bytes. We concluded that 32 bytes, produced by a good hash functions is future secure.
### Requirements
+ Support currently used tools - we don't want to break an ecosystem, or add a long adaptation period. Ref: https://github.com/cosmos/cosmos-sdk/issues/8041
+ Try to keep the address length small - addresses are widely used in state, both as part of a key and object value.
### Scope
This ADR only defines a process for the generation of address bytes. For end-user interactions with addresses (through the API, or CLI, etc.), we still use bech32 to format these addresses as strings. This ADR doesn't change that.
Using bech32 for string encoding gives us support for checksum error codes and handling of user typos.
## Decision
We define the following account types, for which we define the address function:
1. simple accounts: represented by a regular public key (ie: secp256k1, sr25519)
2. naive multisig: accounts composed by other addressable objects (ie: naive multisig)
3. composed accounts with a native address key (ie: bls, group module accounts)
4. module accounts: basically any accounts which cannot sign transactions and which are managed internally by modules
### Legacy Public Key Addresses Don't Change
`secp256k1` and multisig public keys are currently in use in existing Cosmos SDK zones. They use the following
address formats:
Currently (Jan 2021), the only officially supported SDK user accounts are `secp256k1` basic accounts and legacy amino multisig.
They are used in existing Cosmos SDK zones. They use the following address formats:
- secp256k1: `ripemd160(sha256(pk_bytes))[:20]`
- legacy amino multisig: `sha256(aminoCdc.Marshal(pk))[:20]`
@@ -56,42 +96,142 @@ The current multisig public keys use amino serialization to generate the address
those public keys and their address formatting, and call them "legacy amino" multisig public keys
in protobuf. We will also create multisig public keys without amino addresses to be described below.
### Hash Function Choice
### Canonical Address Format
As in other parts of the Cosmos SDK, we will use `sha256`.
We have three types of accounts we would like to create addresses for in the future:
- regular public key addresses for new signature algorithms (ex. `sr25519`).
- public key addresses for multisig public keys that don't use amino encoding
- module accounts: basically any accounts which cannot sign transactions and
which are managed internally by modules
### Basic Address
To address all of these use cases we propose the following basic `AddressHash` function,
based on the discussions in [\#5694](https://github.com/cosmos/cosmos-sdk/issues/5694):
We start with defining a base hash algorithm for generating addresses. Notably, it's used for accounts represented by a single key pair. For each public key schema we have to have an associated `typ` string, which we discuss in a section below. `hash` is the cryptographic hash function defined in the previous section.
```go
func AddressHash(prefix string, contents []byte) []byte {
preImage := []byte(prefix)
if len(contents) != 0 {
preImage = append(preImage, 0)
preImage = append(preImage, contents...)
}
return sha256.Sum256(preImage)[:20]
const A_LEN = 32
func Hash(typ string, key []byte) []byte {
return hash(hash(typ) + key)[:A_LEN]
}
```
`AddressHash` always take a string `prefix` as a starting point which should represent the
type of public key (ex. `sr25519`) or module account being used (ex. `staking` or `group`).
For public keys, the `contents` parameter is used to specify the binary contents of the public
key. For module accounts, `contents` can be left empty (for modules which don't manage "sub-accounts"),
or can be some module-specific content to specify different pools (ex. `bonded` or `not-bonded` for `staking`)
or managed accounts (ex. different accounts managed by the `group` module).
The `+` is bytes concatenation, which doesn't use any separator.
In the `preImage`, the byte value `0` is used as the separator between `prefix` and `contents`. This is a logical
choice given that `0` is an invalid value for a string character and is commonly used as a null terminator.
This algorithm is the outcome of a consultation session with a professional cryptographer.
Motivation: this algorithm keeps the address relatively small (length of the `typ` doesn't impact the length of the final address)
and it's more secure than [post-hash-prefix-proposal] (which uses the first 20 bytes of a pubkey hash, significantly reducing the address space).
Moreover the cryptographer motivated the choice of adding `typ` in the hash to protect against a switch table attack.
### Canonical Public Key Address Prefixes
We use the `address.Hash` function for generating addresses for all accounts represented by a single key:
* simple public keys: `address.Hash(keyType, pubkey)`
+ aggregated keys (eg: BLS): `address.Hash(keyType, aggregatedPubKey)`
+ modules: `address.Hash("module", moduleName)`
All public key types will have a unique protobuf message type such as:
### Composed Addresses
For simple composed accounts (like new naive multisig), we generalize the `address.Hash`. The address is constructed by recursively creating addresses for the sub accounts, sorting the addresses and composing them into a single address. It ensures that the ordering of keys doesn't impact the resulting address.
```go
// We don't need a PubKey interface - we need anything which is addressable.
type Addressable interface {
Address() []byte
}
func NewComposed(typ string, subaccounts []Addressable) []byte {
addresses = map(subaccounts, \a -> LengthPrefix(a.Address()))
addresses = sort(addresses)
return address.Hash(typ, addresses[0] + ... + addresses[n])
}
```
The `typ` parameter should be a schema descriptor, containing all significant attributes with deterministic serialization (eg: utf8 string).
`LengthPrefix` is a function which prepends 1 byte to the address. The value of that byte is the length of the address bits before prepending. The address must be at most 255 bits long.
We are using `LengthPrefix` to eliminate conflicts - it assures, that for 2 lists of addresses: `as = {a1, a2, ..., an}` and `bs = {b1, b2, ..., bm}` such that every `bi` and `ai` is at most 255 long, `concatenate(map(as, \a -> LengthPrefix(a))) = map(bs, \b -> LengthPrefix(b))` iff `as = bs`.
Implementation Tip: account implementations should cache addresses.
#### Multisig Addresses
For new multisig public keys, we define the `typ` parameter not based on any encoding scheme (amino or protobuf). This avoids issues with non-determinism in the encoding scheme.
Example:
```proto
package cosmos.crypto.multisig;
message PubKey {
uint32 threshold = 1;
repeated google.protobuf.Any pubkeys = 2;
}
```
```go
func (multisig PubKey) Address() {
// first gather all nested pub keys
var keys []address.Addressable // cryptotypes.PubKey implements Addressable
for _, _key := range multisig.Pubkeys {
keys = append(keys, key.GetCachedValue().(cryptotypes.PubKey))
}
// form the type from the message name (cosmos.crypto.multisig.PubKey) and the threshold joined together
prefix := fmt.Sprintf("%s/%d", proto.MessageName(multisig), multisig.Threshold)
// use the Composed function defined above
return address.NewComposed(prefix, keys)
}
```
#### Module Account Addresses
NOTE: this section is not finalize and it's in active discussion.
In Basic Address section we defined a module account address as:
```
address.Hash("module", moduleName)
```
We use `"module"` as a schema type for all module derived addresses. Module accounts can have sub accounts. The derivation process has a defined order: module name, submodule key, subsubmodule key.
Module account addresses are heavily used in the SDK so it makes sense to optimize the derivation process: instead of using of using `LengthPrefix` for the module name, we use a null byte (`'\x00'`) as a separator. This works, because null byte is not a part of a valid module name.
```
func Module(moduleName string, key []byte) []byte{
return Hash("module", []byte(moduleName) + 0 + key)
}
```
**Example** A lending BTC pool address would be:
```
btcPool := address.Module("lending", btc.Addrress()})
```
If we want to create an address for a module account depending on more than one key, we can concatenate them:
```
btcAtomAMM := address.Module("amm", btc.Addrress() + atom.Address()})
```
We can continue the derivation process and can create an address for a submodule account.
```
func Submodule(address []byte, derivationKey []byte) {
return Hash("module", address + derivationKey)
}
```
NOTE: if `address` is not a hash based address (with `LEN` length) then we should use `LengthPrefix`. An alternative would be to use one `Module` function, which takes a slice of keys and mapped with `LengthPrefix`. For final version we need to validate what's the most common use.
**Example** For a cosmwasm smart-contract address we could use the following construction:
```
smartContractAddr := Submodule(Module("cosmwasm", smartContractsNamespace), smartContractKey)
```
### Schema Types
A `typ` parameter used in `Hash` function SHOULD be unique for each account type.
Since all SDK account types are serialized in the state, we propose to use the protobuf message name string.
Example: all public key types have a unique protobuf message type similar to:
```proto
package cosmos.crypto.sr25519;
@@ -100,69 +240,89 @@ message PubKey {
bytes key = 1;
}
```
All protobuf messages have unique fully qualified names, in this example `cosmos.crypto.sr25519.PubKey`.
These names are derived directly from .proto files in a standardized way and used
in other places such as the type URL in `Any`s. Since there is an easy and obvious
way to get this name for every protobuf type, we can use this message name as the
key type `prefix` when creating addresses. For all basic public keys, `contents`
should just be the raw unencoded public key bytes.
in other places such as the type URL in `Any`s. We can easily obtain the name using
`proto.MessageName(msg)`.
Thus the canonical address for new public key types would be `AddressHash(proto.MessageName(pk), pk.Bytes)`.
### Multisig Addresses
For new multisig public keys, we define a custom address format not based on any encoding scheme
(amino or protobuf). This avoids issues with non-determinism in the encoding scheme. It also
ensures that multisig public keys which differ simply in the ordering of keys have the same
address by sorting child public keys first.
First we define a proto message for multisig public keys:
```proto
package cosmos.crypto.multisig;
message PubKey {
uint32 threshold = 1;
repeated google.protobuf.Any public_keys = 2;
}
```
We define the following `Address()` function for this public key:
```
func (multisig PubKey) Address() {
// first gather all the addresses of each nested public key
var addresses [][]byte
for key := range multisig.Keys {
addresses = append(joinedAddresses, key.Address())
}
// then sort them in ascending order
addresses = Sort(addresses)
// then concatenate them together
var joinedAddresses []byte
for addr := range addresses {
joinedAddresses := append(joinedAddresses, addr...)
}
// form the string prefix from the message name (cosmos.crypto.multisig.PubKey) and the threshold joined together
prefix := fmt.Sprintf("%s/%d", proto.MessageName(multisig), multisig.Threshold)
// use the standard AddressHash function
return AddressHash(prefix, joinedAddresses)
}
```
## Consequences
### Backwards Compatibility
This ADR is compatible with what was committed and directly supported in the SDK repository.
### Positive
- a simple algorithm for generating addresses for new public keys and module accounts
- a simple algorithm for generating addresses for new public keys, complex accounts and modules
- the algorithm generalizes _native composed keys_
- increased security and collision resistance of addresses
- the approach is extensible for future use-cases - one can use other address types, as long as they don't conflict with the address length specified here (20 or 32 bytes).
- support new account types.
### Negative
- addresses do not communicate key type, a prefixed approach would have done this
- addresses are 60% longer and will consume more storage space
- requires a refactor of KVStore store keys to handle variable length addresses
### Neutral
- protobuf message names are used as key type prefixes
## References
## Further Discussions
Some accounts can have a fixed name or may be constructed in other way (eg: modules). We were discussing an idea of an account with a predefined name (eg: `me.regen`), which could be used by institutions.
Without going into details, these kinds of addresses are compatible with the hash based addresses described here as long as they don't have the same length.
More specifically, any special account address must not have a length equal to 20 or 32 bytes.
## Appendix: Consulting session
End of Dec 2020 we had a session with [Alan Szepieniec](https://scholar.google.be/citations?user=4LyZn8oAAAAJ&hl=en) to consult the approach presented above.
Alan general observations:
+ we don’t need 2-preimage resistance
+ we need 32bytes address space for collision resistance
+ when an attacker can control an input for object with an address then we have a problem with birthday attack
+ there is an issue with smart-contracts for hashing
+ sha2 mining can be use to breaking address pre-image
Hashing algorithm
+ any attack breaking blake3 will break blake2
+ Alan is pretty confident about the current security analysis of the blake hash algorithm. It was a finalist, and the author is well known in security analysis.
Algorithm:
+ Alan recommends to hash the prefix: `address(pub_key) = hash(hash(key_type) + pub_key)[:32]`, main benefits:
+ we are free to user arbitrary long prefix names
+ we still don’t risk collisions
+ switch tables
+ discussion about penalization -> about adding prefix post hash
+ Aaron asked about post hash prefixes (`address(pub_key) = key_type + hash(pub_key)`) and differences. Alan noted that this approach has longer address space and it’s stronger.
Algorithm for complex / composed keys:
+ merging tree like addresses with same algorithm are fine
Module addresses: Should module addresses have different size to differentiate it?
+ we will need to set a pre-image prefix for module addresse to keept them in 32-byte space: `hash(hash('module') + module_key)`
+ Aaron observation: we already need to deal with variable length (to not break secp256k1 keys).
Discssion about arithmetic hash function for ZKP
+ Posseidon / Rescue
+ Problem: much bigger risk because we don’t know much techniques and history of crypto-analysis of arithmetic constructions. It’s still a new ground and area of active research.
Post quantum signature size
+ Alan suggestion: Falcon: speed / size ration - very good.
+ Aaron - should we think about it?
Alan: based on early extrapolation this thing will get able to break EC cryptography in 2050 . But that’s a lot of uncertainty. But there is magic happening with recurions / linking / simulation and that can speedup the progress.
Other ideas
+ Let’s say we use same key and two different address algorithms for 2 different use cases. Is it still safe to use it? Alan: if we want to hide the public key (which is not our use case), then it’s less secure but there are fixes.
### References
+ [Notes](https://hackmd.io/_NGWI4xZSbKzj1BkCqyZMw)
docs/architecture/adr-030-authz-module.md
+230
View File
@@ -0,0 +1,230 @@
# ADR 030: Authorization Module
## Changelog
- 2019-11-06: Initial Draft
- 2020-10-12: Updated Draft
- 2021-11-13: Accepted
## Status
Accepted
## Abstract
This ADR defines the `x/authz` module which allows accounts to grant authorizations to perform actions
on behalf of that account to other accounts.
## Context
The concrete use cases which motivated this module include:
- the desire to delegate the ability to vote on proposals to other accounts besides the account which one has
delegated stake
- "sub-keys" functionality, as originally proposed in [\#4480](https://github.com/cosmos/cosmos-sdk/issues/4480) which
is a term used to describe the functionality provided by this module together with
the `fee_grant` module from [ADR 029](./adr-029-fee-grant-module.md) and the [group module](https://github.com/regen-network/cosmos-modules/tree/master/incubator/group).
The "sub-keys" functionality roughly refers to the ability for one account to grant some subset of its capabilities to
other accounts with possibly less robust, but easier to use security measures. For instance, a master account representing
an organization could grant the ability to spend small amounts of the organization's funds to individual employee accounts.
Or an individual (or group) with a multisig wallet could grant the ability to vote on proposals to any one of the member
keys.
The current
implementation is based on work done by the [Gaian's team at Hackatom Berlin 2019](https://github.com/cosmos-gaians/cosmos-sdk/tree/hackatom/x/delegation).
## Decision
We will create a module named `authz` which provides functionality for
granting arbitrary privileges from one account (the _granter_) to another account (the _grantee_). Authorizations
must be granted for a particular `Msg` service methods one by one using an implementation
of `Authorization`.
### Types
Authorizations determine exactly what privileges are granted. They are extensible
and can be defined for any `Msg` service method even outside of the module where
the `Msg` method is defined. `Authorization`s use the new `ServiceMsg` type from
ADR 031.
#### Authorization
```go
type Authorization interface {
// MethodName returns the fully-qualified Msg service method name as described in ADR 031.
MethodName() string
// Accept determines whether this grant permits the provided sdk.ServiceMsg to be performed, and if
// so provides an upgraded authorization instance.
// Returns:
// + allow: true if msg is authorized
// + updated: new Authorization instance which should overwrite the current one with new state
// + delete: true if Authorization has been exhausted and can be deleted from state
Accept(msg sdk.ServiceMsg, block abci.Header) (allow bool, updated Authorization, delete bool)
}
```
For example a `SendAuthorization` like this is defined for `MsgSend` that takes
a `SpendLimit` and updates it down to zero:
```go
type SendAuthorization struct {
// SpendLimit specifies the maximum amount of tokens that can be spent
// by this authorization and will be updated as tokens are spent. If it is
// empty, there is no spend limit and any amount of coins can be spent.
SpendLimit sdk.Coins
}
func (cap SendAuthorization) MethodName() string {
return "/cosmos.bank.v1beta1.Msg/Send"
}
func (cap SendAuthorization) Accept(msg sdk.ServiceMsg, block abci.Header) (allow bool, updated Authorization, delete bool) {
switch req := msg.Request.(type) {
case bank.MsgSend:
left, invalid := cap.SpendLimit.SafeSub(req.Amount)
if invalid {
return false, nil, false
}
if left.IsZero() {
return true, nil, true
}
return true, SendAuthorization{SpendLimit: left}, false
}
return false, nil, false
}
```
A different type of capability for `MsgSend` could be implemented
using the `Authorization` interface with no need to change the underlying
`bank` module.
### `Msg` Service
```proto
service Msg {
// GrantAuthorization grants the provided authorization to the grantee on the granter's
// account with the provided expiration time.
rpc GrantAuthorization(MsgGrantAuthorization) returns (MsgGrantAuthorizationResponse);
// ExecAuthorized attempts to execute the provided messages using
// authorizations granted to the grantee. Each message should have only
// one signer corresponding to the granter of the authorization.
// The grantee signing this message must have an authorization from the granter.
rpc ExecAuthorized(MsgExecAuthorized) returns (MsgExecAuthorizedResponse)
// RevokeAuthorization revokes any authorization corresponding to the provided method name on the
// granter's account that has been granted to the grantee.
rpc RevokeAuthorization(MsgRevokeAuthorization) returns (MsgRevokeAuthorizationResponse);
}
message MsgGrantAuthorization{
string granter = 1;
string grantee = 2;
google.protobuf.Any authorization = 3 [(cosmos_proto.accepts_interface) = "Authorization"];
google.protobuf.Timestamp expiration = 4;
}
message MsgExecAuthorized {
string grantee = 1;
repeated google.protobuf.Any msgs = 2;
}
message MsgRevokeAuthorization{
string granter = 1;
string grantee = 2;
string method_name = 3;
}
```
### Router Middleware
The `authz` `Keeper` will expose a `DispatchActions` method which allows other modules to send `ServiceMsg`s
to the router based on `Authorization` grants:
```go
type Keeper interface {
// DispatchActions routes the provided msgs to their respective handlers if the grantee was granted an authorization
// to send those messages by the first (and only) signer of each msg.
DispatchActions(ctx sdk.Context, grantee sdk.AccAddress, msgs []sdk.ServiceMsg) sdk.Result`
}
```
This allows the functionality provided by `authz` to be used for future inter-module object capabilities
permissions as described in [ADR 033](https://github.com/cosmos/cosmos-sdk/7459)
### CLI
#### `tx exec` Method
When a CLI user wants to run a transaction on behalf of another account using `MsgExecAuthorized`, they
can use the `exec` method. For instance `gaiacli tx gov vote 1 yes --from <grantee> --generate-only | gaiacli tx authz exec --send-as <granter> --from <grantee>`
would send a transaction like this:
```go
MsgExecAuthorized {
Grantee: mykey,
Msgs: []sdk.SericeMsg{
ServiceMsg {
MethodName:"/cosmos.gov.v1beta1.Msg/Vote"
Request: MsgVote {
ProposalID: 1,
Voter: cosmos3thsdgh983egh823
Option: Yes
}
}
}
}
```
#### `tx grant <grantee> <authorization> --from <granter>`
This CLI command will send a `MsgGrantAuthorization` transaction. `authorization` should be encoded as
JSON on the CLI.
#### `tx revoke <grantee> <method-name> --from <granter>`
This CLI command will send a `MsgRevokeAuthorization` transaction.
### Built-in Authorizations
#### `SendAuthorization`
```proto
// SendAuthorization allows the grantee to spend up to spend_limit coins from
// the granter's account.
message SendAuthorization {
repeated cosmos.base.v1beta1.Coin spend_limit = 1;
}
```
#### `GenericAuthorization`
```proto
// GenericAuthorization gives the grantee unrestricted permissions to execute
// the provide method on behalf of the granter's account.
message GenericAuthorization {
string method_name = 1;
}
```
## Consequences
### Positive
- Users will be able to authorize arbitrary actions on behalf of their accounts to other
users, improving key management for many use cases
- The solution is more generic than previously considered approaches and the
`Authorization` interface approach can be extended to cover other use cases by
SDK users
### Negative
### Neutral
## References
- Initial Hackatom implementation: https://github.com/cosmos-gaians/cosmos-sdk/tree/hackatom/x/delegation
- Post-Hackatom spec: https://gist.github.com/aaronc/b60628017352df5983791cad30babe56#delegation-module
- B-Harvest subkeys spec: https://github.com/cosmos/cosmos-sdk/issues/4480
docs/architecture/adr-031-msg-service.md
+16 -8
View File
@@ -6,7 +6,7 @@
## Status
Proposed
Accepted
## Abstract
@@ -53,7 +53,7 @@ This isn't necessarily bad, but it does add overhead to creating modules.
We decide to use protobuf `service` definitions for defining `Msg`s as well as
the code generated by them as a replacement for `Msg` handlers.
Below we define how this will look for the `SubmitProposal` message from `x/gov` module.
Below we define how this will look for the `SubmitProposal` message from `x/gov` module.
We start with a `Msg` `service` definition:
```proto
@@ -105,7 +105,7 @@ should use the more canonical `Msg...Request` names.
Currently, we are encoding `Msg`s as `Any` in `Tx`s which involves packing the
binary-encoded `Msg` with its type URL.
The type URL for `MsgSubmitProposal` based on the proto3 spec is `/cosmos.gov.MsgSubmitProposal`.
The type URL for `MsgSubmitProposal` based on the proto3 spec is `/cosmos.gov.MsgSubmitProposal`.
The fully-qualified name for the `SubmitProposal` service method above (also
based on the proto3 and gRPC specs) is `/cosmos.gov.Msg/SubmitProposal` which varies
@@ -117,7 +117,7 @@ In order to encode service methods in transactions, we encode them as `Any`s in
the same `TxBody.messages` field as other `Msg`s. We simply set `Any.type_url`
to the full-qualified method name (ex. `/cosmos.gov.Msg/SubmitProposal`) and
set `Any.value` to the protobuf encoding of the request message
(`MsgSubmitProposal` in this case).
(`MsgSubmitProposal` in this case).
### Decoding
@@ -125,7 +125,7 @@ When decoding, `TxBody.UnpackInterfaces` will need a special case
to detect if `Any` type URLs match the service method format (ex. `/cosmos.gov.Msg/SubmitProposal`)
by checking for two `/` characters. Messages that are method names plus request parameters
instead of a normal `Any` messages will get unpacked into the `ServiceMsg` struct:
```go
type ServiceMsg struct {
// MethodName is the fully-qualified service name
@@ -139,7 +139,7 @@ type ServiceMsg struct {
In the future, `service` definitions may become the primary method for defining
`Msg`s. As a starting point, we need to integrate with the SDK's existing routing
and `Msg` interface.
and `Msg` interface.
To do this, `ServiceMsg` implements the `sdk.Msg` interface and its handler does the
actual method routing, allowing this feature to be added incrementally on top of
@@ -218,17 +218,25 @@ Separate handler definition is no longer needed with this approach.
## Consequences
This design changes how a module functionality is exposed and accessed. It deprecates the existing `Handler` interface and `AppModule.Route` in favor of [Protocol Buffer Services](https://developers.google.com/protocol-buffers/docs/proto3#services) and Service Routing described above. This dramatically simplifies the code. We don't need to create handlers and keepers any more. Use of Protocol Buffer auto-generated clients clearly separates the communication interfaces between the module and a modules user. The control logic (aka handlers and keepers) is not exposed any more. A module interface can be seen as a black box accessible through a client API. It's worth to note that the client interfaces are also generated by Protocol Buffers.
This also allows us to change how we perform functional tests. Instead of mocking AppModules and Router, we will mock a client (server will stay hidden). More specifically: we will never mock `moduleA.MsgServer` in `moduleB`, but rather `moduleA.MsgClient`. One can think about it as working with external services (eg DBs, or online servers...). We assume that the transmission between clients and servers is correctly handled by generated Protocol Buffers.
Finally, closing a module to client API opens desirable OCAP patterns discussed in ADR-033. Since server implementation and interface is hidden, nobody can hold "keepers"/servers and will be forced to relay on the client interface, which will drive developers for correct encapsulation and software engineering patterns.
### Pros
- communicates return type clearly
- manual handler registration and return type marshaling is no longer needed, just implement the interface and register it
- some keeper code could be automatically generate, this would improve the UX of [\#7093](https://github.com/cosmos/cosmos-sdk/issues/7093) approach (1) if we chose to adopt that
- generated client code could be useful for clients
- communication interface is automatically generated, the developer can now focus only on the state transition methods - this would improve the UX of [\#7093](https://github.com/cosmos/cosmos-sdk/issues/7093) approach (1) if we chose to adopt that
- generated client code could be useful for clients and tests
- dramatically reduces and simplifies the code
### Cons
- supporting both this and the current concrete `Msg` type approach simultaneously could be confusing
(we could choose to deprecate the current approach)
- using `service` definitions outside the context of gRPC could be confusing (but doesn’t violate the proto3 spec)
## References
- [Initial Github Issue \#7122](https://github.com/cosmos/cosmos-sdk/issues/7122)
docs/architecture/adr-034-account-rekeying.md
+81
View File
@@ -0,0 +1,81 @@
# ADR 034: Account Rekeying
## Changelog
- 30-09-2020: Initial Draft
## Status
PROPOSED
## Abstract
Account rekeying is a process hat allows an account to replace its authentication pubkey with a new one.
## Context
Currently, in the Cosmos SDK, the address of an auth `BaseAccount` is based on the hash of the public key. Once an account is created, the public key for the account is set in stone, and cannot be changed. This can be a problem for users, as key rotation is a useful security practice, but is not possible currently. Furthermore, as multisigs are a type of pubkey, once a multisig for an account is set, it can not be updated. This is problematic, as multisigs are often used by organizations or companies, who may need to change their set of multisig signers for internal reasons.
Transferring all the assets of an account to a new account with the updated pubkey is not sufficient, because some "engagements" of an account are not easily transferable. For example, in staking, to transfer bonded Atoms, an account would have to unbond all delegations and wait the three week unbonding period. Even more significantly, for validator operators, ownership over a validator is not transferrable at all, meaning that the operator key for a validator can never be updated, leading to poor operational security for validators.
## Decision
We propose the addition of a new feature to `x/auth` that allows accounts to update the public key associated with their account, while keeping the address the same.
This is possible because the Cosmos SDK `BaseAccount` stores the public key for an account in state, instead of making the assumption that the public key is included in the transaction (whether explicitly or implicitly through the signature) as in other blockchains such as Bitcoin and Ethereum. Because the public key is stored on chain, it is okay for the public key to not hash to the address of an account, as the address is not pertinent to the signature checking process.
To build this system, we design a new Msg type as follows:
```protobuf
service Msg {
rpc ChangePubKey(MsgChangePubKey) returns (MsgChangePubKeyResponse);
}
message MsgChangePubKey {
string address = 1;
google.protobuf.Any pub_key = 2;
}
message MsgChangePubKeyResponse {}
```
The MsgChangePubKey transaction needs to be signed by the existing pubkey in state.
Once, approved, the handler for this message type, which takes in the AccountKeeper, will update the in-state pubkey for the account and replace it with the pubkey from the Msg.
An account that has had its pubkey changed cannot be automatically pruned from state. This is because if pruned, the original pubkey of the account would be needed to recreate the same address, but the owner of the address may not have the original pubkey anymore. Currently, we do not automatically prune any accounts anyways, but we would like to keep this option open the road (this is the purpose of account numbers). To resolve this, we charge an additional gas fee for this operation to compensate for this this externality (this bound gas amount is configured as parameter `PubKeyChangeCost`). The bonus gas is charged inside the handler, using the `ConsumeGas` function. Furthermore, in the future, we can allow accounts that have rekeyed manually prune themselves using a new Msg type such as `MsgDeleteAccount`. Manually pruning accounts can give a gas refund as an incentive for performing the action.
```go
amount := ak.GetParams(ctx).PubKeyChangeCost
ctx.GasMeter().ConsumeGas(amount, "pubkey change fee")
```
Everytime a key for an address is changed, we will store a log of this change in the state of the chain, thus creating a stack of all previous keys for an address and the time intervals for which they were active. This allows dapps and clients to easily query past keys for an account which may be useful for features such as verifying timestamped off-chain signed messages.
## Consequences
### Positive
* Will allow users and validator operators to employ better operational security practices with key rotation.
* Will allow organizations or groups to easily change and add/remove multisig signers.
### Negative
Breaks the current assumed relationship between address and pubkeys as H(pubkey) = address. This has a couple of consequences.
* This makes wallets that support this feature more complicated. For example, if an address on chain was updated, the corresponding key in the CLI wallet also needs to be updated.
* Cannot automatically prune accounts with 0 balance that have had their pubkey changed.
### Neutral
* While the purpose of this is intended to allow the owner of an account to update to a new pubkey they own, this could technically also be used to transfer ownership of an account to a new owner. For example, this could be use used to sell a staked position without unbonding or an account that has vesting tokens. However, the friction of this is very high as this would essentially have to be done as a very specific OTC trade. Furthermore, additional constraints could be added to prevent accouns with Vesting tokens to use this feature.
* Will require that PubKeys for an account are included in the genesis exports.
## References
+ https://www.algorand.com/resources/blog/announcing-rekeying
docs/architecture/adr-035-rosetta-api-support.md
+210
View File
@@ -0,0 +1,210 @@
# ADR 035: Rosetta API Support
## Authors
- Jonathan Gimeno (@jgimeno)
- David Grierson (@senormonito)
- Alessio Treglia (@alessio)
- Frojdy Dymylja (@fdymylja)
## Context
[Rosetta API](https://www.rosetta-api.org/) is an open-source specification and set of tools developed by Coinbase to
standardise blockchain interactions.
Through the use of a standard API for integrating blockchain applications it will
* Be easier for a user to interact with a given blockchain
* Allow exchanges to integrate new blockchains quickly and easily
* Enable application developers to build cross-blockchain applications such as block explorers, wallets and dApps at
considerably lower cost and effort.
## Decision
It is clear that adding Rosetta API support to the Cosmos SDK will bring value to all the developers and
Cosmos SDK based chains in the ecosystem. How it is implemented is key.
The driving principles of the proposed design are:
1. **Extensibility:** it must be as riskless and painless as possible for application developers to set-up network
configurations to expose Rosetta API-compliant services.
2. **Long term support:** This proposal aims to provide support for all the supported Cosmos SDK release series.
3. **Cost-efficiency:** Backporting changes to Rosetta API specifications from `master` to the various stable
branches of Cosmos SDK is a cost that needs to be reduced.
We will achieve these delivering on these principles by the following:
1. There will be an external repo called [cosmos-rosetta-gateway](https://github.com/tendermint/cosmos-rosetta-gateway)
for the implementation of the core Rosetta API features, particularly:
a. The types and interfaces (`Client`, `OfflineClient`...), this separates design from implementation detail.
b. The `Server` functionality as this is independent of the Cosmos SDK version.
c. The `Online/OfflineNetwork`, which is not exported, and implements the rosetta API using the `Client` interface to query the node, build tx and so on.
d. The `errors` package to extend rosetta errors.
2. Due to differences between the Cosmos release series, each series will have its own specific implementation of `Client` interface.
3. There will be two options for starting an API service in applications:
a. API shares the application process
b. API-specific process.
## Architecture
### The External Repo
As section will describe the proposed external library, including the service implementation, plus the defined types and interfaces.
#### Server
`Server` is a simple `struct` that is started and listens to the port specified in the settings. This is meant to be used across all the Cosmos SDK versions that are actively supported.
The constructor follows:
`func NewServer(settings Settings) (Server, error)`
`Settings`, which are used to construct a new server, are the following:
```go
// Settings define the rosetta server settings
type Settings struct {
// Network contains the information regarding the network
Network *types.NetworkIdentifier
// Client is the online API handler
Client crgtypes.Client
// Listen is the address the handler will listen at
Listen string
// Offline defines if the rosetta service should be exposed in offline mode
Offline bool
// Retries is the number of readiness checks that will be attempted when instantiating the handler
// valid only for online API
Retries int
// RetryWait is the time that will be waited between retries
RetryWait time.Duration
}
```
#### Types
Package types uses a mixture of rosetta types and custom defined type wrappers, that the client must parse and return while executing operations.
##### Interfaces
Every SDK version uses a different format to connect (rpc, gRPC, etc), query and build transactions, we have abstracted this in what is the `Client` interface.
The client uses rosetta types, whilst the `Online/OfflineNetwork` takes care of returning correctly parsed rosetta responses and errors.
Each Cosmos SDK release series will have their own `Client` implementations.
Developers can implement their own custom `Client`s as required.
```go
// Client defines the API the client implementation should provide.
type Client interface {
// Needed if the client needs to perform some action before connecting.
Bootstrap() error
// Ready checks if the servicer constraints for queries are satisfied
// for example the node might still not be ready, it's useful in process
// when the rosetta instance might come up before the node itself
// the servicer must return nil if the node is ready
Ready() error
// Data API
// Balances fetches the balance of the given address
// if height is not nil, then the balance will be displayed
// at the provided height, otherwise last block balance will be returned
Balances(ctx context.Context, addr string, height *int64) ([]*types.Amount, error)
// BlockByHashAlt gets a block and its transaction at the provided height
BlockByHash(ctx context.Context, hash string) (BlockResponse, error)
// BlockByHeightAlt gets a block given its height, if height is nil then last block is returned
BlockByHeight(ctx context.Context, height *int64) (BlockResponse, error)
// BlockTransactionsByHash gets the block, parent block and transactions
// given the block hash.
BlockTransactionsByHash(ctx context.Context, hash string) (BlockTransactionsResponse, error)
// BlockTransactionsByHash gets the block, parent block and transactions
// given the block hash.
BlockTransactionsByHeight(ctx context.Context, height *int64) (BlockTransactionsResponse, error)
// GetTx gets a transaction given its hash
GetTx(ctx context.Context, hash string) (*types.Transaction, error)
// GetUnconfirmedTx gets an unconfirmed Tx given its hash
// NOTE(fdymylja): NOT IMPLEMENTED YET!
GetUnconfirmedTx(ctx context.Context, hash string) (*types.Transaction, error)
// Mempool returns the list of the current non confirmed transactions
Mempool(ctx context.Context) ([]*types.TransactionIdentifier, error)
// Peers gets the peers currently connected to the node
Peers(ctx context.Context) ([]*types.Peer, error)
// Status returns the node status, such as sync data, version etc
Status(ctx context.Context) (*types.SyncStatus, error)
// Construction API
// PostTx posts txBytes to the node and returns the transaction identifier plus metadata related
// to the transaction itself.
PostTx(txBytes []byte) (res *types.TransactionIdentifier, meta map[string]interface{}, err error)
// ConstructionMetadataFromOptions
ConstructionMetadataFromOptions(ctx context.Context, options map[string]interface{}) (meta map[string]interface{}, err error)
OfflineClient
}
// OfflineClient defines the functionalities supported without having access to the node
type OfflineClient interface {
NetworkInformationProvider
// SignedTx returns the signed transaction given the tx bytes (msgs) plus the signatures
SignedTx(ctx context.Context, txBytes []byte, sigs []*types.Signature) (signedTxBytes []byte, err error)
// TxOperationsAndSignersAccountIdentifiers returns the operations related to a transaction and the account
// identifiers if the transaction is signed
TxOperationsAndSignersAccountIdentifiers(signed bool, hexBytes []byte) (ops []*types.Operation, signers []*types.AccountIdentifier, err error)
// ConstructionPayload returns the construction payload given the request
ConstructionPayload(ctx context.Context, req *types.ConstructionPayloadsRequest) (resp *types.ConstructionPayloadsResponse, err error)
// PreprocessOperationsToOptions returns the options given the preprocess operations
PreprocessOperationsToOptions(ctx context.Context, req *types.ConstructionPreprocessRequest) (options map[string]interface{}, err error)
// AccountIdentifierFromPublicKey returns the account identifier given the public key
AccountIdentifierFromPublicKey(pubKey *types.PublicKey) (*types.AccountIdentifier, error)
}
```
### 2. Cosmos SDK Implementation
The cosmos sdk implementation, based on version, takes care of satisfying the `Client` interface.
In Stargate, Launchpad and 0.37, we have introduced the concept of rosetta.Msg, this message is not in the shared repository as the sdk.Msg type differs between cosmos-sdk versions.
The rosetta.Msg interface follows:
```go
// Msg represents a cosmos-sdk message that can be converted from and to a rosetta operation.
type Msg interface {
sdk.Msg
ToOperations(withStatus, hasError bool) []*types.Operation
FromOperations(ops []*types.Operation) (sdk.Msg, error)
}
```
Hence developers who want to extend the rosetta set of supported operations just need to extend their module's sdk.Msgs with the `ToOperations` and `FromOperations` methods.
### 3. API service invocation
As stated at the start, application developers will have two methods for invocation of the Rosetta API service:
1. Shared process for both application and API
2. Standalone API service
#### Shared Process (Only Stargate)
Rosetta API service could run within the same execution process as the application. This would be enabled via app.toml settings, and if gRPC is not enabled the rosetta instance would be spinned in offline mode (tx building capabilities only).
#### Separate API service
Client application developers can write a new command to launch a Rosetta API server as a separate process too, using the rosetta command contained in the `/server/rosetta` package. Construction of the command depends on cosmos sdk version. Examples can be found inside `simd` for stargate, and `contrib/rosetta/simapp` for other release series.
## Status
Proposed
## Consequences
### Positive
- Out-of-the-box Rosetta API support within Cosmos SDK.
- Blockchain interface standardisation
## References
- https://www.rosetta-api.org/
- https://github.com/tendermint/cosmos-rosetta-gateway
docs/architecture/adr-037-gov-split-vote.md
+104
View File
@@ -0,0 +1,104 @@
# ADR 037: Governance split votes
## Changelog
- 2020/10/28: Intial draft
## Status
Proposed
## Abstract
This ADR defines a modification to the the governance module that would allow a staker to split their votes into several voting options. For example, it could use 70% of its voting power to vote Yes and 30% of its voting power to vote No.
## Context
Currently, an address can cast a vote with only one options (Yes/No/Abstain/NoWithVeto) and use their full voting power behind that choice.
However, often times the entity owning that address might not be a single individual. For example, a company might have different stakeholders who want to vote differently, and so it makes sense to allow them to split their voting power. Another example use case is exchanges. Many centralized exchanges often stake a portion of their users' tokens in their custody. Currently, it is not possible for them to do "passthrough voting" and giving their users voting rights over their tokens. However, with this system, exchanges can poll their users for voting preferences, and then vote on-chain proportionally to the results of the poll.
## Decision
We modify the vote structs to be
```
type WeightedVoteOption struct {
Option string
Weight sdk.Dec
}
type Vote struct {
ProposalID int64
Voter sdk.Address
Options []WeightedVoteOption
}
```
And for backwards compatibility, we introduce `MsgWeightedVote` while keeping `MsgVote`.
```
type MsgVote struct {
ProposalID int64
Voter sdk.Address
Option Option
}
type MsgWeightedVote struct {
ProposalID int64
Voter sdk.Address
Options []WeightedVoteOption
}
```
The `ValidateBasic` of a `MsgWeightedVote` struct would require that
1. The sum of all the Rates is equal to 1.0
2. No Option is repeated
The governance tally function will iterate over all the options in a vote and add to the tally the result of the voter's voting power * the rate for that option.
```
tally() {
results := map[types.VoteOption]sdk.Dec
for _, vote := range votes {
for i, weightedOption := range vote.Options {
results[weightedOption.Option] += getVotingPower(vote.voter) * weightedOption.Weight
}
}
}
```
The CLI command for creating a multi-option vote would be as such:
```sh
simd tx gov vote 1 "yes=0.6,no=0.3,abstain=0.05,no_with_veto=0.05" --from mykey
```
To create a single-option vote a user can do either
```
simd tx gov vote 1 "yes=1" --from mykey
```
or
```sh
simd tx gov vote 1 yes --from mykey
```
to maintain backwards compatibility.
## Consequences
### Backwards Compatibility
- Previous VoteMsg types will remain the same and so clients will not have to update their procedure unless they want to support the WeightedVoteMsg feature.
- When querying a Vote struct from state, its structure will be different, and so clients wanting to display all voters and their respective votes will have to handle the new format and the fact that a single voter can have split votes.
- The result of querying the tally function should have the same API for clients.
### Positive
- Can make the voting process more accurate for addresses representing multiple stakeholders, often some of the largest addresses.
### Negative
- Is more complex than simple voting, and so may be harder to explain to users. However, this is mostly mitigated because the feature is opt-in.
### Neutral
- Relatively minor change to governance tally function.
docs/architecture/adr-038-state-listening.md
+612
View File
@@ -0,0 +1,612 @@
# ADR 038: KVStore state listening
## Changelog
- 11/23/2020: Initial draft
## Status
Proposed
## Abstract
This ADR defines a set of changes to enable listening to state changes of individual KVStores and exposing these data to consumers.
## Context
Currently, KVStore data can be remotely accessed through [Queries](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules/messages-and-queries.md#queries)
which proceed either through Tendermint and the ABCI, or through the gRPC server.
In addition to these request/response queries, it would be beneficial to have a means of listening to state changes as they occur in real time.
## Decision
We will modify the `MultiStore` interface and its concrete (`rootmulti` and `cachemulti`) implementations and introduce a new `listenkv.Store` to allow listening to state changes in underlying KVStores.
We will also introduce the tooling for writing these state changes out to files and configuring this service.
### Listening interface
In a new file, `store/types/listening.go`, we will create a `WriteListener` interface for streaming out state changes from a KVStore.
```go
// WriteListener interface for streaming data out from a listenkv.Store
type WriteListener interface {
// if value is nil then it was deleted
// storeKey indicates the source KVStore, to facilitate using the the same WriteListener across separate KVStores
// set bool indicates if it was a set; true: set, false: delete
OnWrite(storeKey types.StoreKey, set bool, key []byte, value []byte)
}
```
### Listener type
We will create a concrete implementation of the `WriteListener` interface in `store/types/listening.go`, that writes out protobuf
encoded KV pairs to an underlying `io.Writer`.
This will include defining a simple protobuf type for the KV pairs. In addition to the key and value fields this message
will include the StoreKey for the originating KVStore so that we can write out from separate KVStores to the same stream/file
and determine the source of each KV pair.
```protobuf
message StoreKVPair {
optional string store_key = 1; // the store key for the KVStore this pair originates from
required bool set = 2; // true indicates a set operation, false indicates a delete operation
required bytes key = 3;
required bytes value = 4;
}
```
```go
// StoreKVPairWriteListener is used to configure listening to a KVStore by writing out length-prefixed
// protobuf encoded StoreKVPairs to an underlying io.Writer
type StoreKVPairWriteListener struct {
writer io.Writer
marshaller codec.BinaryMarshaler
}
// NewStoreKVPairWriteListener wraps creates a StoreKVPairWriteListener with a provdied io.Writer and codec.BinaryMarshaler
func NewStoreKVPairWriteListener(w io.Writer, m codec.BinaryMarshaler) *StoreKVPairWriteListener {
return &StoreKVPairWriteListener{
writer: w,
marshaller: m,
}
}
// OnWrite satisfies the WriteListener interface by writing length-prefixed protobuf encoded StoreKVPairs
func (wl *StoreKVPairWriteListener) OnWrite(storeKey types.StoreKey, set bool, key []byte, value []byte) {
kvPair := new(types.StoreKVPair)
kvPair.StoreKey = storeKey.Name()
kvPair.Set = set
kvPair.Key = key
kvPair.Value = value
if by, err := wl.marshaller.MarshalBinaryLengthPrefixed(kvPair); err == nil {
wl.writer.Write(by)
}
}
```
### ListenKVStore
We will create a new `Store` type `listenkv.Store` that the `MultiStore` wraps around a `KVStore` to enable state listening.
We can configure the `Store` with a set of `WriteListener`s which stream the output to specific destinations.
```go
// Store implements the KVStore interface with listening enabled.
// Operations are traced on each core KVStore call and written to any of the
// underlying listeners with the proper key and operation permissions
type Store struct {
parent types.KVStore
listeners []types.WriteListener
parentStoreKey types.StoreKey
}
// NewStore returns a reference to a new traceKVStore given a parent
// KVStore implementation and a buffered writer.
func NewStore(parent types.KVStore, psk types.StoreKey, listeners []types.WriteListener) *Store {
return &Store{parent: parent, listeners: listeners, parentStoreKey: psk}
}
// Set implements the KVStore interface. It traces a write operation and
// delegates the Set call to the parent KVStore.
func (s *Store) Set(key []byte, value []byte) {
types.AssertValidKey(key)
s.parent.Set(key, value)
s.onWrite(true, key, value)
}
// Delete implements the KVStore interface. It traces a write operation and
// delegates the Delete call to the parent KVStore.
func (s *Store) Delete(key []byte) {
s.parent.Delete(key)
s.onWrite(false, key, nil)
}
// onWrite writes a KVStore operation to all of the WriteListeners
func (s *Store) onWrite(set bool, key, value []byte) {
for _, l := range s.listeners {
l.OnWrite(s.parentStoreKey, set, key, value)
}
}
```
### MultiStore interface updates
We will update the `MultiStore` interface to allow us to wrap a set of listeners around a specific `KVStore`.
Additionally, we will update the `CacheWrap` and `CacheWrapper` interfaces to enable listening in the caching layer.
```go
type MultiStore interface {
...
// ListeningEnabled returns if listening is enabled for the KVStore belonging the provided StoreKey
ListeningEnabled(key StoreKey) bool
// SetListeners sets the WriteListeners for the KVStore belonging to the provided StoreKey
// It appends the listeners to a current set, if one already exists
SetListeners(key StoreKey, listeners []WriteListener)
}
```
```go
type CacheWrap interface {
...
// CacheWrapWithListeners recursively wraps again with listening enabled
CacheWrapWithListeners(storeKey types.StoreKey, listeners []WriteListener) CacheWrap
}
type CacheWrapper interface {
...
// CacheWrapWithListeners recursively wraps again with listening enabled
CacheWrapWithListeners(storeKey types.StoreKey, listeners []WriteListener) CacheWrap
}
```
### MultiStore implementation updates
We will modify all of the `Store` and `MultiStore` implementations to satisfy these new interfaces, and adjust the `rootmulti` `GetKVStore` method
to wrap the returned `KVStore` with a `listenkv.Store` if listening is turned on for that `Store`.
```go
func (rs *Store) GetKVStore(key types.StoreKey) types.KVStore {
store := rs.stores[key].(types.KVStore)
if rs.TracingEnabled() {
store = tracekv.NewStore(store, rs.traceWriter, rs.traceContext)
}
if rs.ListeningEnabled(key) {
store = listenkv.NewStore(key, store, rs.listeners[key])
}
return store
}
```
We will also adjust the `cachemulti` constructor methods and the `rootmulti` `CacheMultiStore` method to forward the listeners
to and enable listening in the cache layer.
```go
func (rs *Store) CacheMultiStore() types.CacheMultiStore {
stores := make(map[types.StoreKey]types.CacheWrapper)
for k, v := range rs.stores {
stores[k] = v
}
return cachemulti.NewStore(rs.db, stores, rs.keysByName, rs.traceWriter, rs.traceContext, rs.listeners)
}
```
### Exposing the data
We will introduce a new `StreamingService` interface for exposing `WriteListener` data streams to external consumers.
```go
// Hook interface used to hook into the ABCI message processing of the BaseApp
type Hook interface {
ListenBeginBlock(ctx sdk.Context, req abci.RequestBeginBlock, res abci.ResponseBeginBlock) // update the streaming service with the latest BeginBlock messages
ListenEndBlock(ctx sdk.Context, req abci.RequestEndBlock, res abci.ResponseEndBlock) // update the steaming service with the latest EndBlock messages
ListenDeliverTx(ctx sdk.Context, req abci.RequestDeliverTx, res abci.ResponseDeliverTx) // update the steaming service with the latest DeliverTx messages
}
// StreamingService interface for registering WriteListeners with the BaseApp and updating the service with the ABCI messages using the hooks
type StreamingService interface {
Stream(wg *sync.WaitGroup, quitChan <-chan struct{}) // streaming service loop, awaits kv pairs and writes them to some destination stream or file
Listeners() map[sdk.StoreKey][]storeTypes.WriteListener // returns the streaming service's listeners for the BaseApp to register
Hook
}
```
#### Writing state changes to files
We will introduce an implementation of `StreamingService` which writes state changes out to files as length-prefixed protobuf encoded `StoreKVPair`s.
This service uses the same `StoreKVPairWriteListener` for every KVStore, writing all the KV pairs from every KVStore
out to the same files, relying on the `StoreKey` field in the `StoreKVPair` protobuf message to later distinguish the source for each pair.
The file naming schema is as such:
* After every `BeginBlock` request a new file is created with the name `block-{N}-begin`, where N is the block number. All
subsequent state changes are written out to this file until the first `DeliverTx` request is received. At the head of these files,
the length-prefixed protobuf encoded `BeginBlock` request is written, and the response is written at the tail.
* After every `DeliverTx` request a new file is created with the name `block-{N}-tx-{M}` where N is the block number and M
is the tx number in the block (i.e. 0, 1, 2...). All subsequent state changes are written out to this file until the next
`DeliverTx` request is received or an `EndBlock` request is received. At the head of these files, the length-prefixed protobuf
encoded `DeliverTx` request is written, and the response is written at the tail.
* After every `EndBlock` request a new file is created with the name `block-{N}-end`, where N is the block number. All
subsequent state changes are written out to this file until the next `BeginBlock` request is received. At the head of these files,
the length-prefixed protobuf encoded `EndBlock` request is written, and the response is written at the tail.
```go
// FileStreamingService is a concrete implementation of StreamingService that writes state changes out to a file
type FileStreamingService struct {
listeners map[sdk.StoreKey][]storeTypes.WriteListener // the listeners that will be initialized with BaseApp
srcChan <-chan []byte // the channel that all of the WriteListeners write their data out to
filePrefix string // optional prefix for each of the generated files
writeDir string // directory to write files into
dstFile *os.File // the current write output file
marshaller codec.BinaryMarshaler // marshaller used for re-marshalling the ABCI messages to write them out to the destination files
stateCache [][]byte // cache the protobuf binary encoded StoreKVPairs in the order they are received
}
```
This streaming service uses a single instance of a simple intermediate `io.Writer` as the underlying `io.Writer` for its single `StoreKVPairWriteListener`,
It collects KV pairs from every KVStore synchronously off of the same channel, caching them in the order they are received, and then writing
them out to a file generated in response to an ABCI message hook. Files are named as outlined above, with optional prefixes to avoid potential naming collisions
across separate instances.
```go
// intermediateWriter is used so that we do not need to update the underlying io.Writer inside the StoreKVPairWriteListener
// everytime we begin writing to a new file
type intermediateWriter struct {
outChan chan <-[]byte
}
// NewIntermediateWriter create an instance of an intermediateWriter that sends to the provided channel
func NewIntermediateWriter(outChan chan <-[]byte) *intermediateWriter {
return &intermediateWriter{
outChan: outChan,
}
}
// Write satisfies io.Writer
func (iw *intermediateWriter) Write(b []byte) (int, error) {
iw.outChan <- b
return len(b), nil
}
// NewFileStreamingService creates a new FileStreamingService for the provided writeDir, (optional) filePrefix, and storeKeys
func NewFileStreamingService(writeDir, filePrefix string, storeKeys []sdk.StoreKey, m codec.BinaryMarshaler) (*FileStreamingService, error) {
listenChan := make(chan []byte, 0)
iw := NewIntermediateWriter(listenChan)
listener := listen.NewStoreKVPairWriteListener(iw, m)
listners := make(map[sdk.StoreKey][]storeTypes.WriteListener, len(storeKeys))
// in this case, we are using the same listener for each Store
for _, key := range storeKeys {
listeners[key] = listener
}
// check that the writeDir exists and is writeable so that we can catch the error here at initialization if it is not
// we don't open a dstFile until we receive our first ABCI message
if err := fileutil.IsDirWriteable(writeDir); err != nil {
return nil, err
}
return &FileStreamingService{
listeners: listeners,
srcChan: listenChan,
filePrefix: filePrefix,
writeDir: writeDir,
marshaller: m,
stateCache: make([][]byte, 0),
}, nil
}
// Listeners returns the StreamingService's underlying WriteListeners, use for registering them with the BaseApp
func (fss *FileStreamingService) Listeners() map[sdk.StoreKey][]storeTypes.WriteListener {
return fss.listeners
}
func (fss *FileStreamingService) ListenBeginBlock(ctx sdk.Context, req abci.RequestBeginBlock, res abci.ResponseBeginBlock) {
// NOTE: this could either be done synchronously or asynchronously
// create a new file with the req info according to naming schema
// write req to file
// write all state changes cached for this stage to file
// reset cache
// write res to file
// close file
}
func (fss *FileStreamingService) ListenEndBlock(ctx sdk.Context, req abci.RequestBeginBlock, res abci.ResponseBeginBlock) {
// NOTE: this could either be done synchronously or asynchronously
// create a new file with the req info according to naming schema
// write req to file
// write all state changes cached for this stage to file
// reset cache
// write res to file
// close file
}
func (fss *FileStreamingService) ListenDeliverTx(ctx sdk.Context, req abci.RequestDeliverTx, res abci.ResponseDeliverTx) {
// NOTE: this could either be done synchronously or asynchronously
// create a new file with the req info according to naming schema
// NOTE: if the tx failed, handle accordingly
// write req to file
// write all state changes cached for this stage to file
// reset cache
// write res to file
// close file
}
// Stream spins up a goroutine select loop which awaits length-prefixed binary encoded KV pairs and caches them in the order they were received
func (fss *FileStreamingService) Stream(wg *sync.WaitGroup, quitChan <-chan struct{}) {
wg.Add(1)
go func() {
defer wg.Done()
for {
select {
case <-quitChan:
return
case by := <-fss.srcChan:
append(fss.stateCache, by)
}
}
}()
}
```
Writing to a file is the simplest approach for streaming the data out to consumers.
This approach also provides the advantages of being persistent and durable, and the files can be read directly,
or an auxiliary streaming services can read from the files and serve the data over a remote interface.
#### Auxiliary streaming service
We will create a separate standalone process that reads and internally queues the state as it is written out to these files
and serves the data over a gRPC API. This API will allow filtering of requested data, e.g. by block number, block/tx hash, ABCI message type,
whether a DeliverTx message failed or succeeded, etc. In addition to unary RPC endpoints this service will expose `stream` RPC endpoints for realtime subscriptions.
#### File pruning
Without pruning the number of files can grow indefinitely, this may need to be managed by
the developer in an application or even module-specific manner (e.g. log rotation).
The file naming schema facilitates pruning by block number and/or ABCI message.
The gRPC auxiliary streaming service introduced above will include an option to remove the files as it consumes their data.
### Configuration
We will provide detailed documentation on how to configure a `FileStreamingService` from within an app's `AppCreator`,
using the provided `AppOptions` and TOML configuration fields.
#### BaseApp registration
We will add a new method to the `BaseApp` to enable the registration of `StreamingService`s:
```go
// RegisterStreamingService is used to register a streaming service with the BaseApp
func (app *BaseApp) RegisterHooks(s StreamingService) {
// set the listeners for each StoreKey
for key, lis := range s.Listeners() {
app.cms.SetListeners(key, lis)
}
// register the streaming service hooks within the BaseApp
// BaseApp will pass BeginBlock, DeliverTx, and EndBlock requests and responses to the streaming services to update their ABCI context using these hooks
app.hooks = append(app.hooks, s)
}
```
We will also modify the `BeginBlock`, `EndBlock`, and `DeliverTx` methods to pass ABCI requests and responses to any streaming service hooks registered
with the `BaseApp`.
```go
func (app *BaseApp) BeginBlock(req abci.RequestBeginBlock) (res abci.ResponseBeginBlock) {
...
// Call the streaming service hooks with the BeginBlock messages
for _ hook := range app.hooks {
hook.ListenBeginBlock(app.deliverState.ctx, req, res)
}
return res
}
```
```go
func (app *BaseApp) EndBlock(req abci.RequestEndBlock) (res abci.ResponseEndBlock) {
...
// Call the streaming service hooks with the EndBlock messages
for _, hook := range app.hooks {
hook.ListenEndBlock(app.deliverState.ctx, req, res)
}
return res
}
```
```go
func (app *BaseApp) DeliverTx(req abci.RequestDeliverTx) abci.ResponseDeliverTx {
...
gInfo, result, err := app.runTx(runTxModeDeliver, req.Tx)
if err != nil {
resultStr = "failed"
res := sdkerrors.ResponseDeliverTx(err, gInfo.GasWanted, gInfo.GasUsed, app.trace)
// If we throw and error, be sure to still call the streaming service's hook
for _, hook := range app.hooks {
hook.ListenDeliverTx(app.deliverState.ctx, req, res)
}
return res
}
res := abci.ResponseDeliverTx{
GasWanted: int64(gInfo.GasWanted), // TODO: Should type accept unsigned ints?
GasUsed: int64(gInfo.GasUsed), // TODO: Should type accept unsigned ints?
Log: result.Log,
Data: result.Data,
Events: sdk.MarkEventsToIndex(result.Events, app.indexEvents),
}
// Call the streaming service hooks with the DeliverTx messages
for _, hook := range app.hook {
hook.ListenDeliverTx(app.deliverState.ctx, req, res)
}
return res
}
```
#### TOML Configuration
We will provide standard TOML configuration options for configuring a `FileStreamingService` for specific `Store`s.
Note: the actual namespace is TBD.
```toml
[store]
streamers = [ # if len(streamers) > 0 we are streaming
"file",
]
[streamers]
[streamers.file]
keys = ["list", "of", "store", "keys", "we", "want", "to", "expose", "for", "this", "streaming", "service"]
writeDir = "path to the write directory"
prefix = "optional prefix to prepend to the generated file names"
```
We will also provide a mapping of the TOML `store.streamers` "file" configuration option to a helper functions for constructing the specified
streaming service. In the future, as other streaming services are added, their constructors will be added here as well.
```go
// StreamingServiceConstructor is used to construct a streaming service
type StreamingServiceConstructor func(opts servertypes.AppOptions, keys []sdk.StoreKey) (StreamingService, error)
// StreamingServiceType enum for specifying the type of StreamingService
type StreamingServiceType int
const (
Unknown StreamingServiceType = iota
File
// add more in the future
)
// NewStreamingServiceType returns the StreamingServiceType corresponding to the provided name
func NewStreamingServiceType(name string) StreamingServiceType {
switch strings.ToLower(name) {
case "file", "f":
return File
default:
return Unknown
}
}
// String returns the string name of a StreamingServiceType
func (sst StreamingServiceType) String() string {
switch sst {
case File:
return "file"
default:
return ""
}
}
// StreamingServiceConstructorLookupTable is a mapping of StreamingServiceTypes to StreamingServiceConstructors
var StreamingServiceConstructorLookupTable = map[StreamingServiceType]StreamingServiceConstructor{
File: FileStreamingConstructor,
}
// NewStreamingServiceConstructor returns the StreamingServiceConstructor corresponding to the provided name
func NewStreamingServiceConstructor(name string) (StreamingServiceConstructor, error) {
ssType := NewStreamingServiceType(name)
if ssType == Unknown {
return nil, fmt.Errorf("unrecognized streaming service name %s", name)
}
if constructor, ok := StreamingServiceConstructorLookupTable[ssType]; ok {
return constructor, nil
}
return nil, fmt.Errorf("streaming service constructor of type %s not found", ssType.String())
}
// FileStreamingConstructor is the StreamingServiceConstructor function for creating a FileStreamingService
func FileStreamingConstructor(opts servertypes.AppOptions, keys []sdk.StoreKey) (StreamingService, error) {
filePrefix := cast.ToString(opts.Get("streamers.file.prefix"))
fileDir := cast.ToString(opts.Get("streamers.file.writeDir"))
return streaming.NewFileStreamingService(fileDir, filePrefix, keys), nil
}
```
#### Example configuration
As a demonstration, we will implement the state watching features as part of SimApp.
For example, the below is a very rudimentary integration of the state listening features into the SimApp `AppCreator` function:
```go
func NewSimApp(
logger log.Logger, db dbm.DB, traceStore io.Writer, loadLatest bool, skipUpgradeHeights map[int64]bool,
homePath string, invCheckPeriod uint, encodingConfig simappparams.EncodingConfig,
appOpts servertypes.AppOptions, baseAppOptions ...func(*baseapp.BaseApp),
) *SimApp {
...
keys := sdk.NewKVStoreKeys(
authtypes.StoreKey, banktypes.StoreKey, stakingtypes.StoreKey,
minttypes.StoreKey, distrtypes.StoreKey, slashingtypes.StoreKey,
govtypes.StoreKey, paramstypes.StoreKey, ibchost.StoreKey, upgradetypes.StoreKey,
evidencetypes.StoreKey, ibctransfertypes.StoreKey, capabilitytypes.StoreKey,
)
// configure state listening capabilities using AppOptions
listeners := cast.ToStringSlice(appOpts.Get("store.streamers"))
for _, listenerName := range listeners {
// get the store keys allowed to be exposed for this streaming service/state listeners
exposeKeyStrs := cast.ToStringSlice(appOpts.Get(fmt.Sprintf("streamers.%s.keys", listenerName))
exposeStoreKeys = make([]storeTypes.StoreKey, 0, len(exposeKeyStrs))
for _, keyStr := range exposeKeyStrs {
if storeKey, ok := keys[keyStr]; ok {
exposeStoreKeys = append(exposeStoreKeys, storeKey)
}
}
// get the constructor for this listener name
constructor, err := baseapp.NewStreamingServiceConstructor(listenerName)
if err != nil {
tmos.Exit(err.Error()) // or continue?
}
// generate the streaming service using the constructor, appOptions, and the StoreKeys we want to expose
streamingService, err := constructor(appOpts, exposeStoreKeys)
if err != nil {
tmos.Exit(err.Error())
}
// register the streaming service with the BaseApp
bApp.RegisterStreamingService(streamingService)
// waitgroup and quit channel for optional shutdown coordination of the streaming service
wg := new(sync.WaitGroup)
quitChan := new(chan struct{}))
// kick off the background streaming service loop
streamingService.Stream(wg, quitChan) // maybe this should be done from inside BaseApp instead?
}
...
return app
}
```
## Consequences
These changes will provide a means of subscribing to KVStore state changes in real time.
### Backwards Compatibility
- This ADR changes the `MultiStore`, `CacheWrap`, and `CacheWrapper` interfaces, implementations supporting the previous version of these interfaces will not support the new ones
### Positive
- Ability to listen to KVStore state changes in real time and expose these events to external consumers
### Negative
- Changes `MultiStore`, `CacheWrap`, and `CacheWrapper` interfaces
### Neutral
- Introduces additional- but optional- complexity to configuring and running a cosmos application
- If an application developer opts to use these features to expose data, they need to be aware of the ramifications/risks of that data exposure as it pertains to the specifics of their application
docs/architecture/adr-039-epoched-staking.md
+110
View File
@@ -0,0 +1,110 @@
# ADR 039: Epoched Staking
## Changelog
- 10-Feb-2021: Initial Draft
## Authors
- Dev Ojha (@valardragon)
- Sunny Aggarwall (@sunnya97)
## Status
Proposed
## Abstract
This ADR updates the proof of stake module to buffer the staking weight updates for a number of blocks before updating the consensus' staking weights. The length of the buffer is dubbed an epoch. The prior functionality of the staking module is then a special case of the abstracted module, with the epoch being set to 1 block.
## Context
The current proof of stake module takes the design decision to apply staking weight changes to the consensus engine immediately. This means that delegations and unbonds get applied immediately to the validator set. This decision was primarily done as it was implementationally simplest, and because we at the time believed that this would lead to better UX for clients.
An alternative design choice is to allow buffering staking updates (delegations, unbonds, validators joining) for a number of blocks. This 'epoch'd proof of stake consensus provides the guarantee that the consensus weights for validators will not change mid-epoch, except in the event of a slash condition.
The decision to have immediate execution of staking changes was primarily done as it was implementationally simplest, and because we at the time believed that this would lead to better UX for clients. The UX hurdle may not be as significant as was previously thought, since it is possible to provide users acknowledgement that their bond was recorded and will be executed.
Furthermore, it has become clearer over time that immediate execution of staking events comes with limitations, such as:
* Threshold based cryptography. One of the main limitations is that because the validator set can change so regularly, it makes the running of multiparty computation by a fixed validator set difficult. Many threshold-based cryptographic features for blockchains such as randomness beacons and threshold decryption require a computationally-expensive DKG process (will take much longer than 1 block to create). To productively use these, we need to guarantee that the result of the DKG will be used for a reasonably long time. It wouldn't be feasible to rerun the DKG every block. By epoching staking, it guarantees we'll only need to run a new DKG once every epoch.
* Light client efficiency. This would lessen the overhead for IBC. Because of the lite client bisection algorithm, the number of headers you need to verify is related to bounding the validator set diffs between two successively verified headers. By limiting the frequency of validator set changes, we can reduce the size of IBC lite client proofs.
* Fairness of deterministic leader election. Currently we have no ways of reasoning of fairness of deterministic leader election in the presence of staking changes without epochs (tendermint/spec#217). Adding epochs at least makes it easier for our deterministic leader election to match something we can prove secure. (Albeit, we still haven’t proven if our current algorithm is fair with > 2 validators)
* Staking derivative design. Currently, reward distribution is done lazily using the F1 fee distribution. While saving computational complexity, lazy accounting increases “statefulness of staking”. Right now, each delegation entry has to track the time of last withdrawal. Handling this can be a challenge for some staking derivatives designs (see example). Force-withdrawing rewards to users can help solve this, however it is infeasible to force-withdraw rewards to users on a per block basis. With epoching, a chain could more easily alter the design to have rewards be forcefully withdrawn (iterating over delegator accounts only once per-epoch), and thus remove the time of delegation from state. This preliminarily seems like it may be of utility in certain staking derivative designs.
## Design considerations
### Slashing
There is a design consideration for whether to apply a slash immediately or at the end of an epoch. A slash event should apply to only members who are actually staked during the time of the infraction, namely during the epoch the slash event occured.
Applying it immediately can be viewed as offering greater consensus layer security, at potential costs to the aforementioned usecases. The benefits of immediate slashing for consensus layer security can be all be obtained by executing the validator jailing immediately (thus removing it from the validator set), and delaying the actual slash change to the validator's weight until the epoch boundary. For the use cases mentioned above, workarounds can be integrated to avoid problems, as follows:
- For threshold based cryptography, it can keep using the original keep epoch weights for the cryptography thresholds, but allow the underlying finality to benefit from extra security more quickly.
- For light client efficiency, there can be a bit included in the header indicating an intra-epoch slash (ala https://github.com/tendermint/spec/issues/199).
- For fairness of deterministic leader election, this will cause problems with the formalization of it / proximity of implementation to formally provable spec. However, a less formal claim can be made that the amount lost due to the slash should hopefully outweigh slight biases into the leader election process. This claim is dubious with the presence of MEV, but potentially formal upperbounds on MEV for fairness here could be derived.
- For staking derivative design, this will not cause problems with the suggested design there, nor does it increase the stateful of staking. (As whether a slash has occured is fully queryable given the validator address)
However, for achieving consensus layer security, it suffices to apply the validator jailing immediately, but still delay the actual slash changes to waiting until the end of the epoch. This largely mitigates the concern for the fairness of deterministic leader election as well, since that validator is removed the set being rotated from immediately.
### Token lockup
When someone makes a transaction to delegate, even though they are not immediately staked, their tokens should be moved into a pool managed by the staking module which will then be used at the end of an epoch. This prevents concerns where they stake, and then spend those tokens not realizing they were already allocated for staking, and thus having their staking tx fail.
### Pipelining the epochs
For threshold based cryptography in particular, we need a pipeline for epoch changes. This is because when we are in epoch N, we want the epoch N+1 weights to be fixed so that the validator set can do the DKG accordingly. So if we are currently in epoch N, the stake weights for epoch N+1 should already be fixed, and new stake changes should be getting applied to epoch N + 2.
This can be handled by making a parameter for the epoch pipeline. This parameter should not be alterable except during hard forks, to mitigate implementation complexity of switching the pipeline length.
### Rewards
Even though all staking updates are applied at epoch boundaries, rewards can still be distributed immediately when they are claimed. This is because they do not affect the current stake weights, as we do not implement auto-bonding of rewards. If such a feature were to be implemented, it would have to be setup so that rewards are auto-bonded at the epoch boundary.
## Decision
__Step-1__: Implement buffering of all staking and slashing messages.
First we create a pool for storing tokens that are being bonded, but should be applied at the epoch boundary called the `EpochDelegationPool`. Then, we have two separate queues, one for staking, one for slashing. We describe what happens on each message being delivered below:
### Staking messages
- **MsgCreateValidator**: Move user's self-bond to `EpochDelegationPool` immediately. Queue a message for the epoch boundary to handle the self-bond, taking the funds from the `EpochDelegationPool`. If Epoch execution fail, return back funds from `EpochDelegationPool` to user's account.
- **MsgEditValidator**: Validate message and if valid queue the message for execution at the end of the Epoch.
- **MsgDelegate**: Move user's funds to `EpochDelegationPool` immediately. Queue a message for the epoch boundary to handle the delegation, taking the funds from the `EpochDelegationPool`. If Epoch execution fail, return back funds from `EpochDelegationPool` to user's account.
- **MsgBeginRedelegate**: Validate message and if valid queue the message for execution at the end of the Epoch.
- **MsgUndelegate**: Validate message and if valid queue the message for execution at the end of the Epoch.
### Slashing messages
- **MsgUnjail**: Validate message and if valid queue the message for execution at the end of the Epoch.
- **Slash Event**: Whenever a slash event is created, it gets queued in the slashing module to apply at the end of the epoch. The queues should be setup such that this slash applies immediately.
### Evidence Messages
- **MsgSubmitEvidence**: This gets executed immediately, and the validator gets jailed immediately. However in slashing, the actual slash event gets queued.
Then we add methods to the end blockers, to ensure that at the epoch boundary the queues are cleared and delegation updates are applied.
__Step-2__: Implement querying of queued staking txs.
When querying the staking activity of a given address, the status should return not only the amount of tokens staked, but also if there are any queued stake events for that address. This will require nodes supporting querying to either do some more indexing to have this be efficiently queryable, or to have transactions
__Step-3__: Adjust gas
Currently gas represents the cost of executing a transaction when its done immediately. (Merging together costs of p2p overhead, state access overhead, and computational overhead) However, now a transaction can cause computation in a future block, namely at the epoch boundary.
To handle this, we should initially include parameters for estimating the amount of future computation (denominated in gas), and add that as a flat charge needed for the message.
We leave it as out of scope for how to weight future computation versus current computation in gas pricing, and have it set such that the are weighted equally for now.
## Consequences
### Positive
* Abstracts the proof of stake module that allows retaining the existing functionality
* Enables new features such as validator-set based threshold cryptography
### Negative
* Increases complexity of integrating more complex gas pricing mechanisms, as they now have to consider future execution costs as well.
docs/architecture/adr-template.md
+6
View File
@@ -53,6 +53,12 @@
{neutral consequences}
## Further Discussions
While an ADR is in the DRAFT or PROPOSED stage, this section should contain a summary of issues to be solved in future iterations (usually referencing comments from a pull-request discussion).
Later, this section can optionally list ideas or improvements the author or reviewers found during the analysis of this ADR.
## Test Cases [optional]
Test cases for an implementation are mandatory for ADRs that are affecting consensus changes. Other ADRs can choose to include links to test cases if applicable.