From 424e6e60ce9699f65a73538714677298a5963884 Mon Sep 17 00:00:00 2001
From: testinginprod <98415576+testinginprod@users.noreply.github.com>
Date: Tue, 6 Dec 2022 15:45:23 +0100
Subject: [PATCH] feat(ADR): ADR 062: Collections -- a new storage layer for
 cosmos-sdk modules. (#14107)

---
 docs/architecture/README.md                   |   1 +
 .../adr-062-collections-state-layer.md        | 117 ++++++++++++++++++
 2 files changed, 118 insertions(+)
 create mode 100644 docs/architecture/adr-062-collections-state-layer.md

diff --git a/docs/architecture/README.md b/docs/architecture/README.md
index 0658208960..aa8eecea46 100644
--- a/docs/architecture/README.md
+++ b/docs/architecture/README.md
@@ -82,6 +82,7 @@ When writing ADRs, follow the same best practices for writing RFCs. When writing
 * [ADR 046: Module Params](./adr-046-module-params.md)
 * [ADR 057: App Wiring](./adr-057-app-wiring.md)
 * [ADR 059: Test Scopes](./adr-059-test-scopes.md)
+* [ADR 062: Collections State Layer](./adr-062-collections-state-layer.md)
 
 ### Draft
 
diff --git a/docs/architecture/adr-062-collections-state-layer.md b/docs/architecture/adr-062-collections-state-layer.md
new file mode 100644
index 0000000000..8ebaddda77
--- /dev/null
+++ b/docs/architecture/adr-062-collections-state-layer.md
@@ -0,0 +1,117 @@
+# ADR 062: Collections, a simplified storage layer for cosmos-sdk modules.
+
+## Changelog
+
+* 30/11/2022: PROPOSED
+
+## Status
+
+PROPOSED - Implemented
+
+## Abstract
+
+We propose a simplified module storage layer which leverages golang generics to allow module developers to handle module
+storage in a simple and straightforward manner, whilst offering safety, extensibility and standardisation.
+
+## Context
+
+Module developers are forced into manually implementing storage functionalities in their modules, those functionalities include
+but are not limited to:
+
+- Defining key to bytes formats.
+- Defining value to bytes formats.
+- Defining secondary indexes.
+- Defining query methods to expose outside to deal with storage.
+- Defining local methods to deal with storage writing.
+- Dealing with genesis imports and exports.
+- Writing tests for all the above.
+
+
+This brings in a lot of problems:
+- It blocks developers from focusing on the most important part: writing business logic.
+- Key to bytes formats are complex and their definition is error-prone, for example:
+  - how do I format time to bytes in such a way that bytes are sorted?
+  - how do I ensure when I don't have namespace collisions when dealing with secondary indexes?
+- The lack of standardisation makes life hard for clients, and the problem is exacerbated when it comes to providing proofs for objects present in state. Clients are forced to maintain a list of object paths to gather proofs.
+
+### Current Solution: ORM
+
+The current SDK proposed solution to this problem is [ORM](https://github.com/cosmos/cosmos-sdk/blob/main/docs/architecture/adr-055-orm.md).
+Whilst ORM offers a lot of good functionality aimed at solving these specific problems, it has some downsides:
+- It requires migrations.
+- It uses the newest protobuf golang API, whilst the SDK still mainly uses gogoproto. 
+- Integrating ORM into a module would require the developer to deal with two different golang frameworks (golang protobuf + gogoproto) representing the same API objects.
+- It has a high learning curve, even for simple storage layers as it requires developers to have knowledge around protobuf options, custom cosmos-sdk storage extensions, and tooling download. Then after this they still need to learn the code-generated API.
+
+### CosmWasm Solution: cw-storage-plus
+
+The collections API takes inspiration from [cw-storage-plus](https://docs.cosmwasm.com/docs/1.0/smart-contracts/state/cw-plus/),
+which has demonstrated to be a powerful tool for dealing with storage in CosmWasm contracts.
+It's simple, does not require extra tooling, it makes it easy to deal with complex storage structures (indexes, snapshot, etc).
+The API is straightforward and explicit.
+
+## Decision
+
+We propose to port the `collections` API, whose implementation lives in [NibiruChain/collections](https://github.com/NibiruChain/collections) to cosmos-sdk.
+
+Collections implements four different storage handlers types:
+
+- `Map`: which deals with simple `key=>object` mappings.
+- `KeySet`: which acts as a `Set` and only retains keys and no object (usecase: allow-lists).
+- `Item`: which always contains only one object (usecase: Params)
+- `Sequence`: which implements a simple always increasing number (usecase: Nonces)
+- `IndexedMap`: builds on top of `Map` and `KeySet` and allows to create relationships with `Objects` and `Objects` secondary keys.
+
+All the collection APIs build on top of the simple `Map` type.
+
+Collections is fully generic, meaning that anything can be used as `Key` and `Value`. It can be a protobuf object or not.
+
+Collections types, in fact, delegate the duty of serialisation of keys and values to a secondary collections API component called `ValueEncoders` and `KeyEncoders`.
+
+`ValueEncoders` take care of converting a value to bytes (relevant only for `Map`). And offers a plug and play layer which allows us to change how we encode objects, 
+which is relevant for swapping serialisation frameworks and enhancing performance.
+`Collections` already comes in with default `ValueEncoders`, specifically for: protobuf objects, special SDK types (sdk.Int, sdk.Dec).
+
+`KeyEncoders` take care of converting keys to bytes, `collections` already comes in with some default `KeyEncoders` for some privimite golang types
+(uint64, string, time.Time, ...) and some widely used sdk types (sdk.Acc/Val/ConsAddress, sdk.Int/Dec, ...).
+These default implementations also offer safety around proper lexicographic ordering and namespace-collision.
+
+Examples of the collections API can be found here:
+- introduction: https://github.com/NibiruChain/collections/tree/main/examples
+- usage in nibiru: [x/oracle](https://github.com/NibiruChain/nibiru/blob/master/x/oracle/keeper/keeper.go#L32), [x/perp](https://github.com/NibiruChain/nibiru/blob/master/x/perp/keeper/keeper.go#L31)
+- cosmos-sdk's x/staking migrated: https://github.com/testinginprod/cosmos-sdk/pull/22
+
+
+## Consequences
+
+### Backwards Compatibility
+
+The design of `ValueEncoders` and `KeyEncoders` allows modules to retain the same `byte(key)=>byte(value)` mappings, making
+the upgrade to the new storage layer non-state breaking.
+
+
+### Positive
+
+- ADR aimed at removing code from the SDK rather than adding it. Migrating just `x/staking` to collections would yield to a net decrease in LOC (even considering the addition of collections itself).
+- Simplifies and standardises storage layers across modules in the SDK.
+- Does not require to have to deal with protobuf.
+- It's pure golang code.
+- Generalisation over `KeyEncoders` and `ValueEncoders` allows us to not tie ourself to the data serialisation framework.
+- `KeyEncoders` and `ValueEncoders` can be extended to provide schema reflection.
+
+### Negative
+
+- Golang generics are not as battle-tested as other Golang features, despite being used in production right now.
+- Collection types instantiation needs to be improved.
+
+### Neutral
+
+{neutral consequences}
+
+## Further Discussions
+
+- Automatic genesis import/export (not implemented because of API breakage)
+- Schema reflection
+
+
+## References