Commit Graph
283 Commits
Author SHA1 Message Date
Tomas TauberandRobert Zaremba f757c90f61 fix: removed potential sources of non-determinism in upgrades (#10189)
forced deterministic iteration order in upgrade migrations, x/upgrade and store during upgrades

Co-authored-by: Robert Zaremba <robert@zaremba.ch>
2021-09-28 17:20:43 +02:00
likhita-809 4c3aa4dbac fix: Charge gas even when there are no entries in gaskv (#10218)
<!--
The default pull request template is for types feat, fix, or refactor.
For other templates, add one of the following parameters to the url:
- template=docs.md
- template=other.md
-->

## Description

Closes: #10127 

Charge gas for seeks with empty ranges

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed 
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)
2021-09-27 06:53:49 +00:00
Andrei Ivasko 6cbbd6da75 refactor!: Keyring migration (#9695)
<!--
The default pull request template is for types feat, fix, or refactor.
For other templates, add one of the following parameters to the url:
- template=docs.md
- template=other.md
-->

## Description

The draft PR #9222 
Closes: #7108

<!-- Add a description of the changes that this PR introduces and the files that
are the most critical to review. -->

- implement proto definition for `Record` 
- rename `Info.go` to `legacyInfo.go` within `keyring` package
- implement CLI `migrate` command that migrates all keys from  legacyInfo to proto according to @robert-zaremba migration [algorithm](https://github.com/cosmos/cosmos-sdk/pull/9222/#discussion_r624683839)
- remove legacy keybase entirely.
- add `Migrate` and `MigrateAll` functions  in `keyring.go` for single key and all keys migration
- add tests
- fix tests

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules)
- [x] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [x] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed 
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)
2021-09-20 12:02:15 +00:00
Dev Ojha 28bf2c124b perf: Make CacheKV store interleaved iterator and insertion not O(n^2) (#10026) 2021-09-10 15:42:56 -04:00
Dev Ojha 78b151dd97 perf: Remove telemetry from wrappings of store (#10077)
## Description

Closes: #10072 

Significantly speeds up all the GasKvStore and CacheKVStore operations. Now Get/Set for these stores no longer even appears on my Osmosis benchmarks, saving ~8% of those benchmark's times. (Only one of the internal methods for setCacheValue appear -- will try to separately get a PR for reducing those memory allocations if #10026 is merged)

Talked to @alexanderbez about this on Discord, and he seemed in agreement with the approach of removing telemetry from these stores.

This does technically change telemetry, but I don't know if this is / should be considered a breaking change?

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [x] provided a link to the relevant issue or specification
- [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules)
- [x] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [x] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification - is there something I should be updating?
- [x] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed 
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)
2021-09-05 20:07:04 +00:00
Cuong Manh Le a87d4dcda4 perf: convert to string once in setCacheValue (#9992)
## Description

So we don't burn un-necessary CPU in case of dirty store/delete.

```
name                          old time/op    new time/op    delta
CacheKVStoreIterator500-8       23.3µs ± 2%    23.4µs ± 1%    ~     (p=0.151 n=5+5)
CacheKVStoreIterator1000-8      47.0µs ± 2%    46.3µs ± 0%  -1.50%  (p=0.008 n=5+5)
CacheKVStoreIterator10000-8      462µs ± 3%     458µs ± 1%    ~     (p=0.690 n=5+5)
CacheKVStoreIterator50000-8     2.63ms ± 5%    2.51ms ± 2%  -4.63%  (p=0.032 n=5+5)
CacheKVStoreIterator100000-8    8.09ms ±21%    6.98ms ± 4%    ~     (p=0.151 n=5+5)
CacheKVStoreGetNoKeyFound-8      393ns ± 2%     397ns ± 2%    ~     (p=0.421 n=5+5)
CacheKVStoreGetKeyFound-8        269ns ± 5%     268ns ± 3%    ~     (p=1.000 n=5+5)
```

Fixes #9991

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [x] provided a link to the relevant issue or specification
- [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed 
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)
2021-08-24 08:59:38 +00:00
Robert Zaremba 13559f9132 feat: file watcher for cosmovisor (#8590)
Adding upgrade file watcher for cosmovisor.

Currently the comswisor upgrade mechanism relays on parsing log messages. This is not reliable:
+ depends on the log level output (x/upgrade uses INFO)
+ can be hacked by accidentally logging user user content
+ can be broken by using upgrade name which will break the regex pattern.

closes: #7703
closes: #8523
closes: #8651
closes: #8793
closes: #8964 

**Depends on**: 
- #9652

---

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

- [ ] Targeted PR against correct branch (see [CONTRIBUTING.md](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [ ] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [ ] Code follows the [module structure standards](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules/structure.md).
- [ ] Wrote unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [ ] Updated relevant documentation (`docs/`) or specification (`x/<module>/spec/`)
- [ ] Added relevant `godoc` [comments](https://blog.golang.org/godoc-documenting-go-code).
- [ ] Added a relevant changelog entry to the `Unreleased` section in `CHANGELOG.md`
- [ ] Re-reviewed `Files changed` in the Github PR explorer
- [ ] Review `Codecov Report` in the comment section below once CI passes
2021-08-11 15:03:48 +00:00
Aleksandr Bezobchuk 6b24ada743 refactor: update default pruning strategy (#9859)
The `default` pruning strategy, specifically the total number of recent blocks to keep (currently 100), was decided on completely arbitrary. 

I propose we change it to be enough blocks given that the typical block time is 5s and typical unbonding period is 21 days:
(3600*(24/5)) * 21 = 362880.

I propose this because anyone wishing to run an IBC relayer, which will be most chains, will have to be required to use `custom` to set it to this value anyway. So why not just make it the default and make it easier for operators.

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed 
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)
2021-08-06 14:23:13 +00:00
likhita-809 6997a3ce43 docs: Add godocs to GasMeter methods (#9665)
<!--
The default pull request template is for types feat, fix, or refactor.
For other templates, add one of the following parameters to the url:
- template=docs.md
- template=other.md
-->

## Description

ref: #9651 

This is a follow-up PR of adding godocs to GasMeter methods

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [ ] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [ ] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [ ] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [ ] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [ ] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed 
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)
2021-07-12 23:17:01 +00:00
Tyler d4d25f5e18 fix: remove stores from renamed/deleted store upgrades (#9409)
<!-- < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < ☺
v                               ✰  Thanks for creating a PR! ✰
v    Before smashing the submit button please review the checkboxes.
v    If a checkbox is n/a - please still include it but + a little note why
☺ > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >  -->

## Description

<!-- Add a description of the changes that this PR introduces and the files that
are the most critical to review.
-->

- stores that were renamed are now properly deleted
- deleted/renamed and renamed stores are no longer added to `CommitInfo`
- deleted/renamed stores are now properly removed from rootmulti store's memory

ref: #7991 
closes: N/A

---

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

- [x] Targeted PR against correct branch (see [CONTRIBUTING.md](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [x] Linked to Github issue with discussion and accepted design OR link to spec that describes this work.
- [x] Code follows the [module structure standards](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules/structure.md).
- [ ] Wrote unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [ ] Updated relevant documentation (`docs/`) or specification (`x/<module>/spec/`)
- [ ] Added relevant `godoc` [comments](https://blog.golang.org/godoc-documenting-go-code).
- [ ] Added a relevant changelog entry to the `Unreleased` section in `CHANGELOG.md`
- [x] Re-reviewed `Files changed` in the Github PR explorer
- [x] Review `Codecov Report` in the comment section below once CI passes
2021-07-12 16:54:07 +00:00
likhita-809 43e0598560 types: Inconsistent limit on InfiniteGasMeter and add GasLeft func to GasMeter (#9651) 2021-07-09 10:38:25 -04:00
yihuang e4f2fa0423 feat!: support debug trace QueryResult (#9576)
<!--
The default pull request template is for types feat, fix, or refactor.
For other templates, add one of the following parameters to the url:
- template=docs.md
- template=other.md
-->

## Description

To let abci query response include more detailed error message with node started with `--trace`.

<!-- Add a description of the changes that this PR introduces and the files that
are the most critical to review. -->

---

### Author Checklist

*All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.*

I have...

- [x] included the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] added `!` to the type prefix if API or client breaking change
- [x] targeted the correct branch (see [PR Targeting](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#pr-targeting))
- [ ] provided a link to the relevant issue or specification
- [x] followed the guidelines for [building modules](https://github.com/cosmos/cosmos-sdk/blob/master/docs/building-modules)
- [ ] included the necessary unit and integration [tests](https://github.com/cosmos/cosmos-sdk/blob/master/CONTRIBUTING.md#testing)
- [x] added a changelog entry to `CHANGELOG.md`
- [ ] included comments for [documenting Go code](https://blog.golang.org/godoc)
- [ ] updated the relevant documentation or specification
- [x] reviewed "Files changed" and left comments if necessary
- [ ] confirmed all CI checks have passed

### Reviewers Checklist

*All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.*

I have...

- [ ] confirmed the correct [type prefix](https://github.com/commitizen/conventional-commit-types/blob/v3.0.0/index.json) in the PR title
- [ ] confirmed `!` in the type prefix if API or client breaking change
- [ ] confirmed all author checklist items have been addressed 
- [ ] reviewed state machine logic
- [ ] reviewed API design and naming
- [ ] reviewed documentation is accurate
- [ ] reviewed tests and test coverage
- [ ] manually tested (if applicable)
2021-07-08 09:25:40 +00:00
90edeb67e2 feat: add RefundGas function to GasMeter (#9403)
* feat: add RefundGas function to GasMeter

* changelog

* add comment about use case

* Apply suggestions from code review

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>
2021-06-02 15:14:15 +00:00
Ryan Christoffersenandryanchrypto cb66c99eab chore: add markdownlint to lint commands (#9353)
* add markdownlint config

* update make lint commands

* update markdownlint config

* run make lint-fix

* fix empty link

* resuse docker container

* run lint-fix

* do not echo commands

Co-authored-by: ryanchrypto <12519942+ryanchrypto@users.noreply.github.com>
2021-05-27 15:31:04 +00:00
Robert Zaremba b065e20f2b fix: add missing nil check in store.GetStore (#9354)
* fix: add missing nil check in store.GetStore

* check nil in rootmulti as well
2021-05-19 08:17:46 +00:00
Robert Zaremba 3e4d81c3d3 Fix proto format (#9279)
* makefile: fix proto-format

* reformat all proto files

* regenerating proto

* remove dangling semicolon
2021-05-07 21:33:17 +00:00
59810f3286 x/authz: audit updates (#9042)
* x/authz: audit updates

* audit with Aaron

* authz: Update Authorization.Accept method

* authz: add event proto definitions

* update query service

* authz: use typed events

* refactore and rename query authorizations

* remve Authorization infix from proto services

* renames wip

* refactoring

* update tests

* fix compilation

* fixing gRPC query tests

* fix simulation tests

* few renames

* more refactore

* add missing file

* moving export genesis to keeper

* Update docs

* update tests

* rename event Msg attribute to MsgTypeURL

* Upate Authorization interface

* rollback Makefile changes

* fix tests

* Apply suggestions from code review

Co-authored-by: Aaron Craelius <aaron@regen.network>

* renames

* refactore authz/exported

* lint fix

* authz/types refactore

* comment update

* conflict updates

* Apply suggestions from code review

Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>

* authz: move storage keys to keeper

* review updates

* docs update

* Update x/authz/client/cli/query.go

Co-authored-by: Aaron Craelius <aaron@regen.network>

* move codec to the root package

* authz CMD info update

* comment update

* update imports and build flags

* fix functional tests

* update proto comment

* fix tests

* fix test

Co-authored-by: Aaron Craelius <aaron@regen.network>
Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>
2021-05-06 18:23:48 +00:00
Cuong Manh Le 4f306fca4d store/cachekv: reduce allocation with []byte -> string in map keys (#9275)
Uses internal/conv throughout store/kv which shows performance gains.

Benchmark for store/cachekv:

name                          old time/op    new time/op    delta
CacheKVStoreIterator500-8       23.4µs ± 1%    23.3µs ± 1%     ~     (p=0.095 n=5+5)
CacheKVStoreIterator1000-8      46.7µs ± 1%    46.2µs ± 0%   -0.96%  (p=0.008 n=5+5)
CacheKVStoreIterator10000-8      457µs ± 1%     455µs ± 1%     ~     (p=1.000 n=5+5)
CacheKVStoreIterator50000-8     2.59ms ± 2%    2.47ms ± 1%   -4.64%  (p=0.008 n=5+5)
CacheKVStoreIterator100000-8    7.33ms ± 3%    6.91ms ± 1%   -5.75%  (p=0.008 n=5+5)
CacheKVStoreGetNoKeyFound-8      423ns ± 1%     391ns ± 2%   -7.41%  (p=0.008 n=5+5)
CacheKVStoreGetKeyFound-8        267ns ± 3%     264ns ± 2%     ~     (p=0.595 n=5+5)

name                          old alloc/op   new alloc/op   delta
CacheKVStoreIterator500-8       5.18kB ± 0%    5.18kB ± 0%     ~     (all equal)
CacheKVStoreIterator1000-8      9.29kB ± 0%    9.29kB ± 0%     ~     (p=0.079 n=4+5)
CacheKVStoreIterator10000-8     85.2kB ± 0%    84.9kB ± 0%   -0.30%  (p=0.008 n=5+5)
CacheKVStoreIterator50000-8      468kB ± 1%     458kB ± 0%   -2.17%  (p=0.008 n=5+5)
CacheKVStoreIterator100000-8    1.16MB ± 1%    1.10MB ± 0%   -5.34%  (p=0.008 n=5+5)
CacheKVStoreGetNoKeyFound-8       222B ± 1%      214B ± 0%   -3.78%  (p=0.008 n=5+5)
CacheKVStoreGetKeyFound-8        51.0B ± 0%     51.0B ± 0%     ~     (all equal)

name                          old allocs/op  new allocs/op  delta
CacheKVStoreIterator500-8         13.0 ± 0%      13.0 ± 0%     ~     (all equal)
CacheKVStoreIterator1000-8        13.0 ± 0%      13.0 ± 0%     ~     (all equal)
CacheKVStoreIterator10000-8       51.0 ± 0%      43.0 ± 0%  -15.69%  (p=0.008 n=5+5)
CacheKVStoreIterator50000-8      1.22k ± 4%     0.94k ± 1%  -23.04%  (p=0.008 n=5+5)
CacheKVStoreIterator100000-8     6.48k ± 4%     4.85k ± 1%  -25.12%  (p=0.008 n=5+5)
CacheKVStoreGetNoKeyFound-8       5.00 ± 0%      4.00 ± 0%  -20.00%  (p=0.008 n=5+5)
CacheKVStoreGetKeyFound-8         2.00 ± 0%      2.00 ± 0%     ~     (all equal)

Benchmark for x/auth/keeper:

name                            old time/op    new time/op    delta
AccountMapperGetAccountFound-8    1.27µs ± 3%    1.26µs ± 1%    ~     (p=0.270 n=5+5)
AccountMapperSetAccount-8         3.53µs ± 0%    3.44µs ± 1%  -2.59%  (p=0.008 n=5+5)

name                            old alloc/op   new alloc/op   delta
AccountMapperGetAccountFound-8      440B ± 0%      440B ± 0%    ~     (all equal)
AccountMapperSetAccount-8         2.13kB ± 0%    2.08kB ± 0%  -2.31%  (p=0.008 n=5+5)

name                            old allocs/op  new allocs/op  delta
AccountMapperGetAccountFound-8      10.0 ± 0%      10.0 ± 0%    ~     (all equal)
AccountMapperSetAccount-8           42.0 ± 0%      38.0 ± 0%  -9.52%  (p=0.008 n=5+5)

Fixes #9274
2021-05-06 06:33:01 -07:00
Cuong Manh Le 711976efc1 store/internal: validate keys before calling ProofsFromMap (#9235)
Otherwise, an empty key as input or present in data can cause a panic at
runtime.

Caught by oss-fuzz: https://oss-fuzz.com/testcase-detail/4647668077953024

Fixes #9233
2021-05-02 15:53:59 -07:00
be4a965599 codec: Rename codec and marshaler interfaces (#9226)
* codec: Rename codec and marshaler interfaces, ref: 8413

* codec: remove BinaryBare

* changelog update

* Update comments and documentation

* adding doc string comments

* Update CHANGELOG.md

Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>

* Update codec/codec.go

Co-authored-by: Marko <marbar3778@yahoo.com>

Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
Co-authored-by: Amaury <1293565+amaurym@users.noreply.github.com>
Co-authored-by: Marko <marbar3778@yahoo.com>
2021-04-29 10:46:22 +00:00
Alessio Treglia 6e26ee9805 bump tendermint core (#9081) 2021-04-08 20:22:00 +01:00
Ian Norden feed37dc56 ADR-038 Part 1: WriteListener, listen.KVStore, MultiStore and KVStore updates (#8551)
* StoreKVPair protobuf message definition and generated go types

* store WriteListener

* update MultiStore, CacheWrap, CacheWrapper interfaces

* adjust KVStores to fit new CacheWrapper interface

* new ListenKVStore

* adjust multistores to fit new MultiStore interface and enable wrapping returned KVStores with the new ListenKVStore

* typo fixes in adr

* ListenKV Store test

* update server mock KVStore and MultiStore

* multistore unit test; fix multistore constructor

* update changelog

* fix bug identified in CI

* improve codecov, minor fixes/adjustments

* review fixes

* review updates; flip set to delete in KVStorePair, updated proto-docs from running 'make proto-gen'
2021-03-30 16:13:51 -04:00
b9f3db1be8 all: skip noisy/faulty benchmarks + add b.ReportAllocs for every benchmark (#8856)
* Skips very noisy benchmarks that end up running only for b.N=1 because
their entire time is spent in setup, and varying parameters doesn't change
much given that the number of stores is what dominates the expense. To
ensure we can provide reliable benchmarks, progressively for the project,
skip these until there is a proper re-work of what the benchmarks need to do

* Previously sub-benchmarks: b.Run(...) did not b.ReportAllocs() due to a faulty
assumption that invoking b.ReportAllocs() at the top would be inherited by
all sub-benchmarks. This change fixes that

Fixes #8779
Fixes #8855

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-03-11 15:59:13 +00:00
Emmanuel T Odeke c2d5b24f58 store/cachekv: use typed types/kv.List instead of container/list.List (#8811)
Reduces CPU burn by using a typed List to avoid the expensive type
assertions from using an interface. This is the only option for now
until Go makes generics generally available.

The change brings time spent on the time assertion cummulatively to:
    580ms down from 6.88s

Explanation:
The type assertions were showing up heavily in profiles:
* Before this commit
```shell
Total: 42.18s
ROUTINE ======================== github.com/cosmos/cosmos-sdk/store/cachekv.newMemIterator
in /Users/emmanuelodeke/go/src/github.com/cosmos/cosmos-sdk/store/cachekv/memiterator.go
    14.01s     18.87s (flat, cum) 44.74% of Total
         .          .     17:	items      []*kv.Pair
         .          .     18:	ascending  bool
         .          .     19:}
         .          .     20:
         .          .     21:func newMemIterator(start, end []byte, items *list.List, ascending bool) *memIterator {
         .      620ms     22:	itemsInDomain := make([]*kv.Pair, 0, items.Len())
         .          .     23:
         .          .     24:	var entered bool
         .          .     25:
     510ms      870ms     26:	for e := items.Front(); e != nil; e = e.Next() {
     6.85s      6.88s     27:		item := e.Value.(*kv.Pair)
     5.71s      8.19s     28:		if !dbm.IsKeyInDomain(item.Key, start, end) {
     120ms      120ms     29:			if entered {
         .          .     30:				break
         .          .     31:			}
         .          .     32:
         .          .     33:			continue
         .          .     34:		}
         .          .     35:
     820ms      980ms     36:		itemsInDomain = append(itemsInDomain, item)
         .          .     37:		entered = true
         .          .     38:	}
         .          .     39:
         .      1.21s     40:	return &memIterator{
         .          .     41:		start:     start,
         .          .     42:		end:       end,
         .          .     43:		items:     itemsInDomain,
         .          .     44:		ascending: ascending,
         .          .     45:	}
```

and given that the list only uses that type, it is only right to lift the
code from container/list.List, and only modify Element.Value's type.

For emphasis, the code is basically just a retrofit of
container/list/list.go but with a typing, and we'll keep it as is
until perhaps Go1.17 or Go1.18 or when everyone uses Go1.17+ after
generics have landed.

* After this commit
```shell
Total: 45.25s
ROUTINE ======================== github.com/cosmos/cosmos-sdk/store/cachekv.newMemIterator
in /Users/emmanuelodeke/go/src/github.com/cosmos/cosmos-sdk/store/cachekv/memiterator.go
     4.84s      6.77s (flat, cum) 14.96% of Total
         .          .     16:	items      []*kv.Pair
         .          .     17:	ascending  bool
         .          .     18:}
         .          .     19:
         .          .     20:func newMemIterator(start, end []byte, items *kv.List, ascending bool) *memIterator {
         .      330ms     21:	itemsInDomain := make([]*kv.Pair, 0, items.Len())
         .          .     22:
         .          .     23:	var entered bool
         .          .     24:
      60ms      160ms     25:	for e := items.Front(); e != nil; e = e.Next() {
     580ms      580ms     26:		item := e.Value
     3.68s      4.78s     27:		if !dbm.IsKeyInDomain(item.Key, start, end) {
      80ms       80ms     28:			if entered {
         .          .     29:				break
         .          .     30:			}
         .          .     31:
         .          .     32:			continue
         .          .     33:		}
         .          .     34:
     440ms      580ms     35:		itemsInDomain = append(itemsInDomain, item)
         .          .     36:		entered = true
         .          .     37:	}
         .          .     38:
         .      260ms     39:	return &memIterator{
         .          .     40:		start:     start,
         .          .     41:		end:       end,
         .          .     42:		items:     itemsInDomain,
         .          .     43:		ascending: ascending,
         .          .     44:	}
```

Fixes #8810
2021-03-08 09:16:23 -08:00
Albert ChonandAlessio Treglia a65f838eb1 reduce gas costs by 10x for transient store operations (#8790)
* reduce gas costs by 10x for transient store operations

* fix TestTransientGasConfig for ReadCostFlat

* added changelog entry

* fix changelog

* fix changelog

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-03-05 16:25:19 +00:00
e17953ab11 perf change (#8796)
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-03-05 13:17:31 +00:00
Robert ZarembaandAlessio Treglia 5f2b90c3c7 internal: create package for unsafe bytes convertion (#8733)
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-03-01 15:10:22 +00:00
Emmanuel T OdekeandAlessio Treglia dbb9923917 store/cachekv, x/bank/types: algorithmically fix pathologically slow code (#8719)
After continuously profiling InitGensis with 100K accounts, it showed
pathologically slow code, that was the result of a couple of patterns:
* Unconditional and not always necessary map lookups
* O(n^2) sdk.AccAddressFromBech32 retrievals when the code is expensive,
during a quicksort

The remedy involved 4 parts:
* O(n) sdk.AccAddressFromBech32 invocations, down from O(n^2) in the quicksort
* Only doing map lookups when the domain key check has passed
* Using a black magic compiler technique of the map clearing idiom
* Zero allocation []byte<->string conversion

With 100K accounts, this brings InitGenesis down to ~6min, instead of
20+min, it reduces the sort code from ~7sec down to 50ms.

Also some simple benchmark reflect the change:
```shell
name                    old time/op    new time/op    delta
SanitizeBalances500-8     19.3ms ±10%     1.5ms ± 5%  -92.46%  (p=0.000 n=20+20)
SanitizeBalances1000-8    41.9ms ± 8%     3.0ms ±12%  -92.92%  (p=0.000 n=20+20)

name                    old alloc/op   new alloc/op   delta
SanitizeBalances500-8     9.05MB ± 6%    0.56MB ± 0%  -93.76%  (p=0.000 n=20+18)
SanitizeBalances1000-8    20.2MB ± 3%     1.1MB ± 0%  -94.37%  (p=0.000 n=20+19)

name                    old allocs/op  new allocs/op  delta
SanitizeBalances500-8      72.4k ± 6%      4.5k ± 0%  -93.76%  (p=0.000 n=20+20)
SanitizeBalances1000-8      162k ± 3%        9k ± 0%  -94.40%  (p=0.000 n=20+20)
```

The CPU profiles show the radical change as per
https://github.com/cosmos/cosmos-sdk/issues/7766#issuecomment-786671734

Later on, we shall do more profiling and fixes but for now this brings
down the run-time for InitGenesis.

Fixes #7766

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-02-27 07:26:22 -08:00
Alessio Treglia f2ee972e31 various linter fixes (#8666) 2021-02-23 08:46:01 +00:00
Alessio Tregliaandmergify[bot] 3b8e0f9387 store/multistore: revert a height limit increase from #8396 (#8466)
See https://github.com/cosmos/cosmos-sdk/pull/8396#discussion_r565654207

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2021-01-29 18:46:23 +00:00
Emmanuel T OdekeandAlessio Treglia 784a9a69a1 all: ensure b.ReportAllocs() in all the benchmarks (#8460)
With this change, we'll get details on the number of
allocations performed by code. Later on when we have
continuous benchmarking infrastructure, this change
will prove useful to flag regressions.

Fixes #8459

Co-authored-by: Alessio Treglia <alessio@tendermint.com>
2021-01-27 23:52:08 -08:00
e2f510afcc Compatibility with the ARM architecture (#8396)
Co-authored-by: Alessio Treglia <alessio@tendermint.com>
Co-authored-by: Jonathan Gimeno <jgimeno@gmail.com>
2021-01-27 11:07:45 +00:00
Aaron Craelius 2521964d50 Update gogo proto deps with v1.3.2 security fixes (#8350)
* Update gogo proto deps with v1.3.2 security fixes

* Regenerate proto files
2021-01-15 19:45:34 +00:00
e481f13ff3 docs: cache-wrapping and virtual store (#8102)
* docs: cache-wrapping and virtual store

* Finish the docs update

* Update docs/building-modules/msg-services.md

Co-authored-by: Amaury <amaury.martiny@protonmail.com>

* Update docs/building-modules/msg-services.md

Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>

Co-authored-by: Amaury <amaury.martiny@protonmail.com>
Co-authored-by: Aleksandr Bezobchuk <alexanderbez@users.noreply.github.com>
2021-01-05 15:57:33 +00:00
Robert Zarembaandmergify[bot] 6de685805d Snapshot sync: use hasher for chunk hashes (#7215) (#7259)
* snapshot sync: use blake3 for chunk hashes (7215)

Blake3 improves on security and speed compared to sha256.
https://blake3.io/

In this PR:
+ use optimized blake3 hashes (with dedicated SIMD code)
  instead of sha256
+ reuse resources on hashing chunks.
+ cleaned few error return statements.

* linter issues fixes

* revert blake2 hashing to sha256

* revert go.mod

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-12-10 19:02:17 +00:00
Anil Kumar KammariandRobert Zaremba 4a233b8dcf Rename GRPCRouter (#8079)
* rename GRPCRouter

* Update store/types/store.go

Co-authored-by: Robert Zaremba <robert@zaremba.ch>

* Fix gofmt

Co-authored-by: Robert Zaremba <robert@zaremba.ch>
2020-12-04 15:06:50 +00:00
Aleksandr Bezobchuk 6476b09b64 multistore: fix SetInitialVersion (#8048) 2020-11-30 12:22:05 -05:00
6e3fab854f Fix upgrade store loader (#7817)
* fix upgrade store loader

* fix test

* resolve comment

Co-authored-by: billy rennekamp <billy.rennekamp@gmail.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-11-13 15:27:52 +00:00
Coryandmergify[bot] 505d424a3b [IAVL]: Bump to v0.15.0-rc4 (#7549)
* iavl 0.15.0-rc4 version bump

* update comments on error modes for store.LoadStore

* update CONFIO_URL in makefile

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-10-19 18:38:17 +00:00
Riccardo Montagnin bbbc0e15f0 Merge PR #7415: Return empty store tree on non-existing version 2020-10-13 08:39:33 -04:00
Alexander Bezobchuk 7ae84898de Merge PR #7265: Tendermint Block Pruning 2020-09-14 10:12:49 -04:00
Alessio Treglia 46a8e94740 run make format && go mod tidy (#7298) 2020-09-14 12:42:09 +01:00
Erik Grinaker 4faeefebd2 Add state sync support (#7166)
* Add state sync support

* fix incorrect test tempdir

* proto: move and update Protobuf schemas

* proto: lint fixes

* comment tweaks

* don't use type aliasing

* don't call .Error() when logging errors

* use create terminology instead of take for snapshots

* reuse chunk hasher

* simplify key encoding code

* track chunk index in Manager

* add restoreDone message for Manager

* add a ready channel to Snapshotter.Restore()

* add comment on streaming IO API

* use sdkerrors for error handling

* fix incorrect error

* tweak changelog

* syntax fix

* update test code after merge
2020-09-08 09:05:44 +00:00
3b9b58c931 Add height in exported genesis (#7089)
* Add height in exported genesis

* +1

* Add test

* Refactor ctx in setupApp

* Use amino in export

* Use tmjson

* Add custom initialVersion (set to 0 for now)

* Add comment

* Add mount in initChainer

* app.LastBlockheight

* InitializeAndSeal in InitChain?

* Revert create store with initial version

* Update to latest iavl

* Check height in test

* Make it work

* Add more tests

* Rename interface

* Use struct isntead of 6 args

* Fix lint

* Remove stray fmt

* Revert go mod/sum

* Install iavl rc3

* Update comments

* Add fee in network

* Typo

* Fix logic in commit

* Fix tests

* Only set initial version on > 1

* Genesis block num = 1

* Fresh chain, genesis block = 0

* Add comments

* Revert Mutable/ImmutableTree

* Allow for zero height

* Fix restart

* Add comments

* Add comments, fix test

* Fix remaining one test

* Add panic test

* Update comment

* Add test for --height

* No cast

* Add check that genesis file exists

* Remove duplicate imports

* Fail early

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Jack Zampolin <jack.zampolin@gmail.com>
Co-authored-by: Cory <cjlevinson@gmail.com>
2020-09-03 10:11:46 +00:00
d2de7a0441 Bump github.com/cosmos/iavl from 0.15.0-rc1 to 0.15.0-rc2 (#7064)
* Bump github.com/cosmos/iavl from 0.15.0-rc1 to 0.15.0-rc2

Bumps [github.com/cosmos/iavl](https://github.com/cosmos/iavl) from 0.15.0-rc1 to 0.15.0-rc2.
- [Release notes](https://github.com/cosmos/iavl/releases)
- [Changelog](https://github.com/cosmos/iavl/blob/master/CHANGELOG.md)
- [Commits](https://github.com/cosmos/iavl/compare/v0.15.0-rc1...v0.15.0-rc2)

Signed-off-by: dependabot[bot] <support@github.com>

* fix test

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Marko <marbar3778@yahoo.com>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
Co-authored-by: Aleksandr Bezobchuk <aleks.bezobchuk@gmail.com>
2020-08-17 17:10:24 +00:00
Neeraj Murarkaandmergify[bot] 2079576cbe Replaced hardcoded 'cosmos' and 'cosmosvaloper' help references (#7056)
* Replaced hardcoded 'cosmos' and 'cosmosvaloper' help references

* Ran make format target

Co-authored-by: mergify[bot] <37929162+mergify[bot]@users.noreply.github.com>
2020-08-17 09:04:50 +00:00
8de96d16f9 tendermint: update to rc3 (#6892)
* modify light imports

* change abci.header to tmproto.header

* use rc

* rc

* fix import

* Merge PR #6893: fix key imports

* fix rc2

* tendermint: update 3 (#6899)

* tendermint: update 4 (#6919)

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>

* tendermint: update 5 (#6923)

Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>

* bump to latest master

* tendermint: update (#6972)

Co-authored-by: Aleksandr Bezobchuk <aleks.bezobchuk@gmail.com>
Co-authored-by: Cory Levinson <cjlevinson@gmail.com>

* Update x/ibc/07-tendermint/types/test_utils.go

Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>

* address comment

* go mod

* bring back things

* fix test

* update tm proto files

Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
Co-authored-by: Aleksandr Bezobchuk <aleks.bezobchuk@gmail.com>
Co-authored-by: Cory Levinson <cjlevinson@gmail.com>
Co-authored-by: Federico Kunze <federico.kunze94@gmail.com>
2020-08-14 13:58:53 -04:00
Amaury MartinyandFederico Kunze 5559af8b97 Proto Auditing: Replace ID with Id (#7032)
* Proto Auditing: Replace ID with Id

* make proto-gen

* Update docs

Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
2020-08-14 08:55:54 +00:00
61a97ef347 Audit / Refactor proto packages layout (#6905)
* create base proto package, move ValAddresses to staking

* move tx, crypto, query proto packages into base; mark module level packages as beta

* new cosmos/base/abci.proto file, refactor some message names

* add comments to TxResponse

* refactor RegisterInterface calls to new proto package names

* fix bug in merge commit

* move missing module proto packages to beta, move cosmos.kv to cosmos.base.kv

* add tx.proto files, holding proto messages for module Msgs

* rm old generated pb.go files

* cosmos/base/base.proto -> cosmos/base/coin.proto

* mark genutil proto package as beta

* Fix conflicts

* Restructure proto files

* Re put signing

* Fix test

* Update slashing genesis

* Ignore confio proto files

* Fix interface register

* Make proto-gen

* Fix lint

* Put tx service into tx.v1beta1

* Use v1beta1 in interface registry

* Remove to cosmos.base.store

* Remove extra confio in buf

Co-authored-by: Amaury Martiny <amaury.martiny@protonmail.com>
Co-authored-by: Alexander Bezobchuk <alexanderbez@users.noreply.github.com>
Co-authored-by: Federico Kunze <31522760+fedekunze@users.noreply.github.com>
2020-08-11 11:49:45 +00:00
Erik Grinaker 7a0d5ff718 store: add test cases for hashing of empty inputs (#7004) 2020-08-11 13:10:46 +02:00