LaconicNetwork/cosmos-sdk
7
0
Fork 0
Code Issues Pull Requests Packages Projects Releases 2 Wiki Activity

feat: integration test helpers (#15556)

Browse Source
This commit is contained in:
Julien Robert
2023-03-31 18:02:15 +00:00
committed by GitHub
parent 43a4db5a9e
commit aeaa301506
8 changed files with 372 additions and 158 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/architecture/adr-059-test-scopes.md
+16 -14
View File
@@ -3,6 +3,8 @@
## Changelog
* 2022-08-02: Initial Draft
* 2023-03-02: Add precision for integration tests
* 2023-03-23: Add precision for E2E tests
## Status
@@ -55,13 +57,11 @@ These are almost like integration tests in that they exercise many things togeth
use mocks.
Example 1 journey vs illustrative tests - [depinject's BDD style tests](https://github.com/cosmos/cosmos-sdk/blob/main/depinject/features/bindings.feature), show how we can
rapidly build up many illustrative cases demonstrating behavioral rules without [very much
code](https://github.com/cosmos/cosmos-sdk/blob/main/depinject/binding_test.go) while maintaining high level readability.
rapidly build up many illustrative cases demonstrating behavioral rules without [very much code](https://github.com/cosmos/cosmos-sdk/blob/main/depinject/binding_test.go) while maintaining high level readability.
Example 2 [depinject table driven tests](https://github.com/cosmos/cosmos-sdk/blob/main/depinject/provider_desc_test.go)
Example 3 [Bank keeper tests](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/x/bank/keeper/keeper_test.go#L94-L105) - A mock implementation of `AccountKeeper` is
supplied to the keeper constructor.
Example 3 [Bank keeper tests](https://github.com/cosmos/cosmos-sdk/blob/2bec9d2021918650d3938c3ab242f84289daef80/x/bank/keeper/keeper_test.go#L94-L105) - A mock implementation of `AccountKeeper` is supplied to the keeper constructor.
#### Limitations
@@ -147,6 +147,9 @@ End to end tests exercise the entire system as we understand it in as close an a
to a production environment as is practical. Presently these tests are located at
[tests/e2e](https://github.com/cosmos/cosmos-sdk/tree/main/tests/e2e) and rely on [testutil/network](https://github.com/cosmos/cosmos-sdk/tree/main/testutil/network) to start up an in-process Tendermint node.
An application should be built as minimally as possible to exercise the desired functionality.
The SDK uses an application will only the required modules for the tests. The application developer is adviced to use its own application for e2e tests.
#### Limitations
In general the limitations of end to end tests are orchestration and compute cost.
@@ -162,12 +165,14 @@ The scope of e2e tests has been complected with command line interface testing.
We accept these test scopes and identify the following decisions points for each.
| Scope | App Fixture | Mocks? |
| ----------- | ----------- | ------ |
| Unit | None | Yes |
| Integration | depinject | Some |
| Simulation | depinject | No |
| E2E | simapp | No |
| Scope | App Type | Mocks? |
| ----------- | ------------------- | ------ |
| Unit | None | Yes |
| Integration | integration helpers | Some |
| Simulation | minimal app | No |
| E2E | minimal app | No |
The decision above is valid for the SDK. An application developer should test their application with their full application instead of the minimal app.
### Unit Tests
@@ -175,8 +180,6 @@ All modules must have mocked unit test coverage.
Illustrative tests should outnumber journeys in unit tests.
~BDD feature tests are recommended when building up illustrative and journey scenarios.~
Unit tests should outnumber integration tests.
Unit tests must not introduce additional dependencies beyond those already present in
@@ -200,7 +203,7 @@ Integration tests should outnumber e2e tests.
### Simulations
Simulations shall use `depinject`. They are located under `/x/{moduleName}/simulation`.
Simulations shall use a minimal application (usually via app wiring). They are located under `/x/{moduleName}/simulation`.
### E2E Tests
@@ -233,7 +236,6 @@ demonstrated in [PR#12706](https://github.com/cosmos/cosmos-sdk/pull/12706).
### Neutral
* learning curve for BDD style tests
* some discovery required for e2e transition to dockertest
## Further Discussions
docs/docs/building-modules/11-structure.md
+1 -1
View File
@@ -75,7 +75,7 @@ x/{module_name}
└── README.md
```
* `client/`: The module's CLI client functionality implementation and the module's integration testing suite.
* `client/`: The module's CLI client functionality implementation and the module's CLI testing suite.
* `exported/`: The module's exported types - typically interface types. If a module relies on keepers from another module, it is expected to receive the keepers as interface contracts through the `expected_keepers.go` file (see below) in order to avoid a direct dependency on the module implementing the keepers. However, these interface contracts can define methods that operate on and/or return types that are specific to the module that is implementing the keepers and this is where `exported/` comes into play. The interface types that are defined in `exported/` use canonical types, allowing for the module to receive the keepers as interface contracts through the `expected_keepers.go` file. This pattern allows for code to remain DRY and also alleviates import cycle chaos.
* `keeper/`: The module's `Keeper` and `MsgServer` implementation.
* `module/`: The module's `AppModule` and `AppModuleBasic` implementation.
docs/docs/building-modules/16-testing.md
+13 -31
View File
@@ -57,34 +57,16 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/gov/keeper/keeper_test.g
Integration tests are at the second level of the [test pyramid](https://martinfowler.com/articles/practical-test-pyramid.html).
In the SDK, we locate our integration tests under [`/tests/integrations`](https://github.com/cosmos/cosmos-sdk/tree/main/tests/integration).
The goal of these integration tests is to test a component with a minimal application (i.e. not `simapp`). The minimal application is defined with the help of [`depinject`](../tooling/02-depinject.md) – the SDK dependency injection framework, and includes all necessary modules to test the component. With the helps of the SDK testing package, we can easily create a minimal application and start the application with a set of genesis transactions: <https://github.com/cosmos/cosmos-sdk/blob/main/testutil/sims/app_helpers.go>.
The goal of these integration tests is to test how a component interacts with other dependencies. Compared to unit tests, integration tests do not mock dependencies. Instead, they use the direct dependencies of the component. This differs as well from end-to-end tests, which test the component with a full application.
Integration tests interact with the tested module via the defined `Msg` and `Query` services. The result of the test can be verified by checking the state of the application, by checking the emitted events or the response. It is adviced to combine two of these methods to verify the result of the test.
The SDK provides small helpers for quickly setting up an integration tests. These helpers can be found at <https://github.com/cosmos/cosmos-sdk/blob/main/testutil/integration>.
### Example
Here, we will walkthrough the integration tests of the `x/distribution` module. The `x/distribution` module has, in addition to keeper unit tests, integration tests that test the `x/distribution` module with a minimal application. This is expected as you may want to test the `x/distribution` module with actual application logic, instead of only mocked dependencies.
For creating a minimal application, we use [`simtestutil.Setup`](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/testutil/sims/app_helpers.go#L95-L99) and an [`AppConfig`](../tooling/02-depinject.md) of the `x/distribution` minimal dependencies.
For instance, the `AppConfig` of `x/distribution` is defined as:
* https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/distribution/testutil/app_config.go
This is a stripped down version of the `simapp` `AppConfig`:
* https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/simapp/app_config.go
:::note
You can as well use the `AppConfig` `configurator` for creating an `AppConfig` [inline](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/slashing/app_test.go#L54-L62). There no difference between those two ways, use whichever you prefer.
:::
```go reference
https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/tests/integration/distribution/keeper/keeper_test.go#L28-L33
```
Now the types are injected and we can use them for our tests:
```go reference
https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/tests/integration/distribution/keeper/keeper_test.go#L21-L53
https://github.com/cosmos/cosmos-sdk/blob/29e22b3bdb05353555c8e0b269311bbff7b8deca/testutil/integration/example_test.go#L22-L89
```
## Deterministic and Regression tests
@@ -106,6 +88,10 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/tests/integration/bank/kee
Simulations uses as well a minimal application, built with [`depinject`](../tooling/02-depinject.md):
:::note
You can as well use the `AppConfig` `configurator` for creating an `AppConfig` [inline](https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/slashing/app_test.go#L54-L62). There is no difference between those two ways, use whichever you prefer.
:::
Following is an example for `x/gov/` simulations:
```go reference
@@ -121,6 +107,7 @@ https://github.com/cosmos/cosmos-sdk/blob/v0.47.0-rc1/x/gov/simulation/operation
End-to-end tests are at the top of the [test pyramid](https://martinfowler.com/articles/practical-test-pyramid.html).
They must test the whole application flow, from the user perspective (for instance, CLI tests). They are located under [`/tests/e2e`](https://github.com/cosmos/cosmos-sdk/tree/main/tests/e2e).
<!-- @julienrbrt: makes more sense to use an app wired app to have 0 simapp dependencies -->
For that, the SDK is using `simapp` but you should use your own application (`appd`).
Here are some examples:
@@ -132,11 +119,6 @@ Here are some examples:
The SDK is in the process of creating its E2E tests, as defined in [ADR-59](https://docs.cosmos.network/main/architecture/adr-059-test-scopes.html). This page will eventually be updated with better examples.
:::
## Summary
## Learn More
| Scope | App Fixture | Mocks? |
| ----------- | ----------- | ------ |
| Unit | None | Yes |
| Integration | `depinject` | Some |
| Simulation | `depinject` | No |
| E2E | `appd` | No |
Learn more about testing scope in [ADR-59](https://docs.cosmos.network/main/architecture/adr-059-test-scopes.html).