Blockchain data is public and immutable. Smart contracts are also public, so everyone can see what they do. This transparency and immutability lead people to conclude that blockchain data can be trusted. It can and it can’t be. Almost nobody consumes the raw data, which is reliable. Instead, we consume interpreted data, and that interpretation is not reliable at all.

Web3 participants are unlikely to challenge the validity of the data they consume even though they know nothing about how it was produced. Data collection and interpretation methods are either obscure or unknown. Luckily, this will gradually change as the industry develops and the race to acquire higher quality data accelerates. More competition will inevitably lead to more stringent demands for reliable data.

Lack of Data Standardization

On-chain data is raw when it is first extracted from the blockchain. It has to be interpreted and indexed before it can be used. In DeFi, this data is produced by smart contracts, so the interpretation of the data depends on the contract (program) that produced it. There are millions of smart contracts deployed on the Ethereum chain alone. These multiple data sources complicate things further. A closer look will present various problems like how on-chain data timestamps and real-time data like oracles and API timestamps present different ways of expressing time.

Deriving insights from data require careful analysis and data normalization as well as consistent methodologies. All of these must be transparent and verifiable so users can now rely on them.

Lack of Data Transparency

Web3 and DeFi are touted as trustless because of their transparency, but to what extent is this true? Most providers of data or DeFi protocols do not fully disclose contextual details.

The truth is, it’s all a black box and the methodology isn’t transparent. Say you want to fetch the number of swaps that were conducted using the protocol’s API.

How do you assess the accuracy of the output?

• Are the times off
• Is the swap order correct?
• What does the price include?
• Is there any missing context?

These are all good questions and we can’t answer them without full transparency. In the absence of this transparency, we assume the data is correct.

A Case for Data Discrepancy

To showcase how discrepancy exists across multiple reliable data sources, we conducted an experiment where we queried liquidity data across three sources for the lending protocol, Aave V2.

The three sources were:

• Aave Native API
• Smart contract (on-chain) data extraction
• Subgraph API

On-chain analysis is conducted by measuring the amount of aTokens issued by Aave to depositors. The difference is between assets' balance fetched from Aave Native API and total supply of corresponding aTokens.

This is shocking. From the comparison, a major difference is present in the total number of assets (totaling up to several million in value).

Additional findings indicate:

• Aave was closest to real-time
• On-chain was point-of-time before block creation
• Subgraph data can be collected at any time of the data with no clear transparency on how the node operates and the variables it takes into account.

Different sync times with variance across outputs can be troubling. Not to mention, the outputs will be difficult to verify and reproduce. Check out the full analysis to see comparison across more assets.

Data Availability

Data availability is crucial when we consider the amount of value transacted every second in crypto. Imagine if a rebalancing application cannot rebalance swiftly because:

• the data isn't fresh,
• the source is slow or unresponsive, or worse,
• data is missing.

Users of more reliable data can react better and faster to market conditions. Reliable data is worth money. Unfortunately, these are common issues that persist in DeFi. Luckily, they are being addressed by a new generation of independent data providers.

Conclusion

These are a few of the problems we’ve encountered. It’s not to suggest that these are the only problems.

At Credmark we work publicly and transparently. All of the data we produce can be traced back to the original, trustworthy on-chain data. This data drives our finance and risk models that in turn produce data. All of this is available via our API.

Our engineering and modeling teams are always available. They respond to queries and process community feedback as quickly as possible.

To accelerate the development of decentralized finance, access to data with verifiable integrity is essential. Together, we can do this.

About Credmark

Credmark is a financial modeling platform for DeFi, powered by verifiable on-chain data. It provides the infrastructure necessary to build models. An API allows anyone to consume the output of these models.

Our modeling tools are the most flexible and robust available today. The Credmark Model Framework streamlines prototyping and deployment. Community members are incentivized to build and improve models. As a result, users benefit from verifiable data and best-in-class analyses.

Credmark is a decentralized project. Our community participates in governance, provides research, and develops models. Become a member of Credmark to advance the next-generation financial system