Integrate a new custom ipld module into laconicd #24
Labels
No Label
bug
C:CLI
C:Crypto
C:Encoding
C:Proto
C:Types
dependencies
docker
documentation
duplicate
enhancement
go
good first issue
help wanted
high priority
in progress
invalid
javascript
low priority
medium priority
question
Status: Stale
Type: ADR
Type: Build
Type: CI
Type: Docs
Type: Tests
urgent
wontfix
Copied from Github
Kind/Breaking
Kind/Bug
Kind/Documentation
Kind/Enhancement
Kind/Feature
Kind/Security
Kind/Testing
Priority
Critical
Priority
High
Priority
Low
Priority
Medium
Reviewed
Confirmed
Reviewed
Duplicate
Reviewed
Invalid
Reviewed
Won't Fix
Status
Abandoned
Status
Blocked
Status
Need More Info
No Milestone
No project
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies
No dependencies set.
Reference: cerc-io/laconicd-deprecated#24
Loading…
Reference in New Issue
Block a user
No description provided.
Delete Branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
From Ashwin: We can build this as a library instead of a module (as it doesn't contribute to state)
ipfs
public.blocks
tableBugfixes then this feature please, it should be a light lift.
@0xmuralik @aleem1314
For some background:
As a Tendermint/CosmosSDK state machine, some of the main roles of Laconicd are
Watchers can be thought of as WASM bytes that describe a custom state transition function over custom subsets of native (e.g. Ethereum) chain state.
To support this arbitrary transforming and indexing of chain data (while also preserving its Merkle context), we first extract chain state from its "flat" Merkle representation on disc (e.g. levelDB for geth) into an initial relational representation defined here. This is done by a modified geth client and some other tools for historical state: https://github.com/cerc-io/ipld-eth-state-snapshot, https://github.com/cerc-io/eth-statediff-service. We refer to this whole stack as the "full-indexing node" or FIN.
The ipld-eth-db representation retains the native Merkle representation (aka hash => node mappings for every node in the entire ethereum DAG) but in an IPFS blockstore format. An IPFS blockstore (v1) is a flat key-value store that maps multihash keys to IPLD blocks (in the case of most Ethereum objects, these are RLP encoded bytes), the shim for treating a Postgres database as a v1 blockstore is https://github.com/ipfs/go-ds-sql, so we currently store our IPLD blocks in that format (except note that we have denormalized our table by block_number). Additionally the ipld-eth-db schema has other tables (e.g. "eth.header_cids", "eth.state_cids") which "blowout" certain ethereum object fields into searchable columns and reference back to the raw IPLD record.
I mention v1 blockstore because that is what we currently use and is what go-ds-sql and ipfs work with, but we are in the middle of upgrading ipld-eth-db to v5 and one of the main changes is that we revert to using a v0 blockstore format. That is, instead of using multihash keys we use fully fledged CIDs. Some additional context on that change: https://github.com/cerc-io/ipld-eth-db/issues/122.
The reason I am providing all this context here is because up until this issue laconicd has been developed in isolation from the FIN stack, but this issue is where the two finally begin to converge.
In order for laconic to perform watcher fraud proof evaluation (4 above), in order to evaluate a fraud proof from within the laconic Tendermint/Cosmos state machine, we need to be able to provide the state machine access to the raw chain inputs to the watcher (the fraud proof evaluation essentially involves repeating the challenged state transitions in the federated laconicd execution environment to check and see if you get the expected result or not).
That is, we need a module that exposes a
keeper with abasic CRUD interface on-top of ipld-eth-db (specifically, the “ipld.blocks” table) instead of on top of the Cosmos Multistore/KVStore/IAVL.Table we need to interface with:
key
is a CID, anddata
is an IPLD block bytes.To summarize, we need to write a module that exposes a interface that has
“Get”, “Set”, “Delete”, “Update” operations for the ipld.blocks table of ipld-eth-db. We can extend/refine this interface further as we begin to use it.
I am going to try to break this down into some finer actionable tasks today.
This won't need to be a x/module with a Keeper, as it isn't managing state committed to Tendermint, it can just be some package that other modules/keepers pull in.
I think we could actually get away with using https://github.com/cerc-io/ipfs-ethdb for now as it exposes the basic CRUD operations we need on-top of ipld-eth-db (to start experimenting with anyways), but it brings along some unnecessary stuff.
Steps to close this issue:
We could begin work on 1 before finishing the other lines of work, but those are the two places in laconicd where we will need to provide access to the IPLD blockstore so we won't be able to integrate the package further until that work is done.