[ADR: 013] Observability (#5188)

* [ADR: 013] Metrics

- Inital ADR for metrics with in the SDK
- extending metrics to be module specific as well

Signed-off-by: Marko Baricevic <marbar3778@yahoo.com>

* add some metrics

* update adr-013 to Observability

* Update docs/architecture/README.MD

Co-Authored-By: Jack Zampolin <jack.zampolin@gmail.com>

* change wording

* minor wording change

* add more things

* undo module work

* Fix formatting and added more info

* Address comments

* Address comments

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com>
Co-authored-by: Timothy Chen <tnachen@gmail.com>
Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>

docs/architecture/README.md

@@ -39,6 +39,7 @@ Please add a entry below in your Pull Request for an ADR.
 - [ADR 010: Modular AnteHandler](./adr-010-modular-antehandler.md)
 - [ADR 011: Generalize Genesis Accounts](./adr-011-generalize-genesis-accounts.md)
 - [ADR 012: State Accessors](./adr-012-state-accessors.md)
+- [ADR 013: Observability](./adr-013-observability.md)
 - [ADR 015: IBC Packet Receiver](./adr-015-ibc-packet-receiver.md)
 - [ADR 016: Validator Consensus Key Rotation](./adr-016-validator-consensus-key-rotation.md)
 - [ADR 017: Historical Header Module](./adr-017-historical-header-module.md)

docs/architecture/adr-013-metrics.md

@@ -0,0 +1,97 @@
+# ADR 013: Observability
+
+## Changelog
+
+- 20-01-2020: Initial Draft
+
+## Status
+
+Proposed
+
+## Context
+
+There has been discussion around exposing more metrics to users and node operators about the application. Currently there is only a way to expose metrics from Tendermint and not the application itself. To bring more visibility into applications, I would like to propose reporting of metrics through [Prometheus](https://prometheus.io/).
+
+Extending `AppModuleBasic` to support registering of metrics would enable developers to see more information about individual modules.
+
+```go
+type AppModuleBasic interface {
+  Name() string
+  RegisterCodec(*codec.Codec)
+  RegisterMetrics(namespace string, labelsAndValues... string) *Metrics
+
+  // genesis
+  DefaultGenesis() json.RawMessage
+  ValidateGenesis(json.RawMessage) error
+
+  // client functionality
+  RegisterRESTRoutes(context.CLIContext, *mux.Router)
+  GetTxCmd(*codec.Codec) *cobra.Command
+  GetQueryCmd(*codec.Codec) *cobra.Command
+}
+// .....
+
+func (bm BasicManager) RegisterMetrics(appName string, labelsAndValues... string) MetricsProvider {
+	for _, b := range bm {
+		b.CreateMetrics(appName, labelsAndValues)
+	}
+}
+```
+
+Each module can define its own `Metrics` type and`CreateMetrics` function in the x/<module>/observability/metrics.go file:
+
+```go
+type Metrics struct {
+  Size metrics.Guage
+
+  Transactions metrics.Counter
+}
+
+func CreateMetrics(namespace string, labelsAndValues... string) *Metrics {
+  labels := make([]string, len(labelsAndValues/2))
+  for i := 0; i < len(labelsAndValues); i += 2 {
+      labels[i/2] = labelsAndValues[i]
+  }
+  return &Metrics{
+    Size: prometheus.NewGaugeFrom(stdprometheus.GaugeOpts{
+		Namespace: namespace,
+		Subsystem: "subsystem",
+		Name:      "size",
+		Help:      "Size of the custom metric",
+	}, labels).With(labelsAndValues...),
+    Transactions: prometheus.NewCounterFrom(stdprometheus.CounterOpts{
+                Namespace: namespace,
+                Subsystem: "subsystem",
+                Name:      "transactions",
+                Help:      "Number of transactions processed",
+        }, labels).With(labelsAndValues...),
+  }
+
+```
+
+To get the correct namespace for the modules changing `BasicManager` to consist of the app name is needed.
+
+```go
+type BasicManager struct {
+	appName string
+	modules map[string]AppModuleBasic
+}
+```
+
+## Decision
+
+- Use Prometheus for metric gathering.
+- Add a method to register metrics to the `AppModuleBasic` interface
+- Modules create a observability/metrics.go that defines the metrics and create the metrics object.
+
+## Consequences
+
+### Positive
+
+- Add more visibility into SDK based application and modules
+
+### Negative
+
+### Neutral
+
+## References