From 09998ef86e28ef73077d61ade73392a428fe009a Mon Sep 17 00:00:00 2001 From: Robert Zaremba Date: Thu, 24 Sep 2020 01:06:55 +0200 Subject: [PATCH] adr-27: adding (#7232) * adr-27: adding Each ADR should have an abstract and an example use case to quickly present the goal. Here, adding this to DR 027: Deterministic Protobuf Serialization * review update * added link to TxRaw * added list of major use-cases * updated wording in context and abstract * updated the adr-020 part * add consequences and motivation for TxRaw * Update docs/architecture/adr-027-deterministic-protobuf-serialization.md Co-authored-by: Amaury Martiny * Update docs/architecture/adr-027-deterministic-protobuf-serialization.md Co-authored-by: Amaury Martiny * Update docs/architecture/adr-027-deterministic-protobuf-serialization.md Co-authored-by: Amaury Martiny * Update docs/architecture/adr-027-deterministic-protobuf-serialization.md Co-authored-by: Amaury Martiny * Update docs/architecture/adr-027-deterministic-protobuf-serialization.md Co-authored-by: Amaury Martiny * rewording * Update docs/architecture/adr-027-deterministic-protobuf-serialization.md Co-authored-by: Simon Warta <2603011+webmaster128@users.noreply.github.com> * Update docs/architecture/adr-027-deterministic-protobuf-serialization.md Co-authored-by: Simon Warta <2603011+webmaster128@users.noreply.github.com> * Update docs/architecture/adr-027-deterministic-protobuf-serialization.md Co-authored-by: Simon Warta <2603011+webmaster128@users.noreply.github.com> Co-authored-by: Amaury Martiny Co-authored-by: Simon Warta <2603011+webmaster128@users.noreply.github.com> Co-authored-by: Cory Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com> --- ...27-deterministic-protobuf-serialization.md | 66 ++++++++++++++----- 1 file changed, 48 insertions(+), 18 deletions(-) diff --git a/docs/architecture/adr-027-deterministic-protobuf-serialization.md b/docs/architecture/adr-027-deterministic-protobuf-serialization.md index 7012a6dc50..974dc0c660 100644 --- a/docs/architecture/adr-027-deterministic-protobuf-serialization.md +++ b/docs/architecture/adr-027-deterministic-protobuf-serialization.md @@ -9,18 +9,49 @@ Proposed -## Context +## Abstract +Fully deterministic structure serialization, which works across many languages and clients, +is needed for structure signature use-case. We need to be sure that whenever we serialize +a data structure, no matter in which supported language, the raw bytes +will stay the same. [Protobuf](https://developers.google.com/protocol-buffers/docs/proto3) serialization is not bijective (i.e. there exist a practically unlimited number of -valid binary representations for a protobuf document)1. For signature -verification in Cosmos SDK, signer and verifier need to agree on the same -serialization of a SignDoc as defined in -[ADR-020](./adr-020-protobuf-transaction-encoding.md) without transmitting the -serialization. This document describes a bijective serialization scheme for +valid binary representations for a protobuf document)1. + +This document describes a deterministic serialization scheme for a subset of protobuf documents, that covers this use case but can be reused in other cases as well. +### Context + +For signature verification in Cosmos SDK, signer and verifier need to agree on +the same serialization of a `SignDoc` as defined in +[ADR-020](./adr-020-protobuf-transaction-encoding.md) without transmitting the +serialization. + +Currently, for block signatures we are using a workaround: we create a new [TxRaw](https://github.com/cosmos/cosmos-sdk/blob/9e85e81e0e8140067dd893421290c191529c148c/proto/cosmos/tx/v1beta1/tx.proto#L30) +instance (as defined in [adr-020-protobuf-transaction-encoding](https://github.com/cosmos/cosmos-sdk/blob/master/docs/architecture/adr-020-protobuf-transaction-encoding.md#transactions)) +by converting all [Tx](https://github.com/cosmos/cosmos-sdk/blob/9e85e81e0e8140067dd893421290c191529c148c/proto/cosmos/tx/v1beta1/tx.proto#L13) +fields to bytes on the client side. This adds an additional manual +step when sending and signing transactions. + +### Decision + +The following encoding scheme is proposed to be used by other ADRs. +Currently we are using the this ADR to for `SignDoc` serialization. + +## Specification + +### Scope + +This ADR defines a protobuf3 serializer. The output is a valid protobuf +serialization, such that every protobuf parser can parse it. + +No maps are supported in version 1 due to the complexity of defining a +deterministic serialization. This might change in future. Implementations must +reject documents containing maps as invalid input. + ### Background - Protobuf3 Encoding Most numeric types in protobuf3 are encoded as @@ -40,18 +71,6 @@ malleability. Among other sources of non-determinism, this ADR eliminates the possibility of encoding malleability. -## Decision - -The following encoding scheme is proposed to be used by other ADRs. - -### Scope - -This ADR defines a protobuf3 serializer. The output is a valid protobuf -serialization, such that every protobuf parser can parse it. - -No maps are supported in version 1 due to the complexity of defining a -deterministic serialization. This might change in future. Implementations must -reject documents containing maps as invalid input. ### Serialization rules @@ -250,9 +269,20 @@ for all protobuf documents we need in the context of Cosmos SDK signing. - When implementing transaction signing, the encoding rules above must be understood and implemented. - The need for rule number 3. adds some complexity to implementations. +- Some data structures may require custom code for serialization. Thus + the code is not very portable - it will require additional work for each + client implementing serialization for not compatible data structures. ### Neutral + +### Usage in SDK + +For the reasons mentioned above ("Negative" section) we prefer to keep workarounds +for shared data structure. Example: the aforementioned `TxRaw` is using raw bytes +as a workaround. This allows them to use any valid Protobuf library without +a need of implementing this standard (and related risks of bugs). + ## References - 1 _When a message is serialized, there is no guaranteed order for