How Laconic Network Uses IPLD

Accessing blockchain data off-chain is no longer a matter of trust

The mission of the Laconic Network is to ensure that decentralized off-chain caches can serve blockchain data without losing the ability to prove the integrity of that data across data transformations. This post shows how the Laconic Stack maintains the verifiability of Ethereum data for its decentralized caching and querying solution.

Querying blockchain data is difficult and expensive

DApp developers face a dilemma when it comes to working with blockchain data. Ethereum data is stored in various ways, including the state, storage, and transaction tries, as well as in event logs. The original concept for accessing Ethereum data was that DApps would run or have access to an Ethereum full node and query it directly.

However, DApp developers rarely want to run their own full node only to gain access to the data they need. Further, and more problematic, due to the way data is stored on Ethereum, there is often no direct way of querying for particular information. Because blockchains are optimized for writing new blocks and achieving consensus, they cannot be indexed like a classic database. As a result, to query for data, developers have to make multiple calls to the Ethereum API, replay transactions to read the events emitted, and store intermittent state in the application to join it with data from subsequent invocations.  This process is difficult and expensive.

The alternative to running a full node is to use a middleware indexing and caching service. While this alternative simplifies data acquisition for developers, it introduces some new and significant problems. In addition to high costs, long setup times, reliability issues, and the dependence on Web2-style centralized services, there is one problem that is especially pernicious: the data served by these services must be trusted. It is currently impossible for existing centralized data providers to offer cryptographic proofs for every byte of data served.

The Solution: Laconic Network and IPLD

Many people are familiar with the Interplanetary File System (IPFS), but fewer people know about the Interplanetary Linked Data (IPLD) protocol. IPLD is the underlying technology of IPFS, and it is the superpower technology behind many of IPFS’s advantages.

IPLD can represent arbitrary data in self-describing structured objects, and it can link those objects into Merkle DAGs. As such, IPLD offers the following advantages:

1. The data is content-addressable via a Content ID (CID), just as IPFS files are content-addressable.
2. The data is cryptographically provable, meaning you can verify that a set of data belongs in a larger data set even without needing to have either set on hand. You only need the cryptographic hashes.
3. The data undergoes natural deduplication, reducing storage and transport costs.
4. The data is tamper-proof. Any attempts to change the data that you have requested via content addressing can be foiled by validating the hashes.

IPLD codecs map data from its original form to the IPLD data model. So if a codec exists for that data, the data has a formal representation in the IPLD data model.

The Connection to Ethereum

In the case of Ethereum data, the DAG-ETH codecs can be used to map on-chain data to off-chain IPLD structures. This mapping to IPLD makes it possible to inspect, process, or reason about data in a uniform way. The codecs cover block headers, uncles, transactions, transaction receipts, and receipt logs, as well as all the different Merkle Patricia tries that are rooted in an Ethereum block. As a result, IPLD can accurately represent any Ethereum data.

Unlike the original data, the off-chain IPLD representation enables indexing for fast querying. The restrictions of the blockchain do not apply here. Indexes can be added that allow developers to query data in ways that are otherwise difficult to achieve with the native ETH JSON RPC API.

More importantly, the IPLD representation of blockchain data can be kept verifiable even after transforming the data into other models. Transformations are necessary to support complex DApps that aggregate and link data in app-specific ways or rely on a non-native representation of the data. The classic methods of slicing, dicing, and recombining the data would render it unverifiable. With Laconic Network data, thanks to our use of IPLD, the data remains as verifiable as it is on the blockchain.

Merkleized data is provable data

The IPLD representation of an Ethereum block includes the block header and the uncles, transactions, receipts, logs, receipt trie, transaction trie, state trie, and storage tries that Ethereum stores alongside the block header. A modified Merkle Patricia Trie (MMPT), which is relevant to Ethereum, has branch nodes, extension nodes, and leaf nodes. Non-leaf nodes store the content hashes of their child nodes. This hashing goes all the way up to the root node. Therefore, the hash value in the tree root represents all leaf data. If any part of the data in a Merkle tree is tampered with, the root hash would change and differ from the root hash stored in the block header. In other words, data like state, transactions, and more, is connected to the block header in a provable way.  

Storing all of the IPLD: Laconic Full Index Nodes

Traditionally, Ethereum Full Nodes are used to power DApps that need to access and query all of the data of the blockchain. The disadvantages of relying on a normal Full Node are well known:

1. Expensive to operate
2. Not optimized for data querying
3. Logs stored from events have to be replayed to access the data

Laconic Full Index Nodes (LFIN) overcome these limitations by generating additional derivative data based on that which is stored in the Ethereum Full Node, adding new indexes, relational mappings, and materialized views, and even merkleizing some data that was previously not merkleized. The result is an amount of data that far exceeds what is normally stored. The role of the LFIN is to track and store the complete data, including derived indexes and Merkle trees of that data, in IPLD block format, for eventual consumption by DApps.

The data representation inside of the LFIN optimizes many classes of queries by removing the need  to chain together data from multiple transactions. For example, listing all transactions for a particular wallet address could be achieved with a single query. Or gathering all of the node hashes needed to generate a cryptographic proof.

Laconic Watchers - interoperable and provable blockchain data

Because the volume of data in the Laconic Full Index Node is so vast, most DApps would not want to have to query that database directly. Instead, DApps work with a specialized caching layer that exists to transform LFIN data into the format needed by specific DApps. These are Laconic Watchers.

The data in the LFIN is still not ready for DApp developers at this point. Further transformation is needed to structure the data in the format required by your DApp. This is the role of the Laconic Watcher. Watchers will be the topic of a subsequent article, but within the scope of this article, you can understand them as being custom secondary or tertiary caches of data that directly fulfill the needs of specific applications.

Thanks to our use of IPLD, these transformations can occur without losing the cryptographic integrity of the data. Developers can build their Watchers to track, process, and cache the underlying blockchain data, and then query the watchers using GraphQL. The returned data includes the linked hash structure that can be followed all the way back to the underlying blockchain. In the Laconic Network, when any data is being transformed, the hash of the inputs to the transformation and the hash of the code that performs the transformation are preserved. Thus:

T(a+b) => c

The new model, c, will contain content-hash references to a, b, and the code performing T.

One intentional consequence of our adoption of structured IPLD models for data representation is that it greatly simplifies dealing with cross chain data. Our ability to transform data while maintaining linked hashing will be invaluable when we begin to index chains beyond Ethereum. Laconic Network is designed with blockchain interoperability in mind.

The source of proof

The Laconic Network improves the Web3/blockchain ecosystem greatly by combining the validation-preserving nature of IPLD with a new way of caching and transforming blockchain data in a decentralized manner. This ability frees DApp developers from relying on centralized, trust-based data providers and closes the cryptographic provability gap between the blockchain and DApps built to use blockchain data.

Stay up-to-date with Laconic news:

• Join our Discord server
• Subscribe to our Telegram channel
• Sign up for our mailing list below