Introduction
The world of Technology is concentrated in the hands of a few big players. The introduction of AI and the need to play by their rules has come at a great cost to the essence and consciousness of us all, humans.
Web3 aims to cause a seismic shift of power from the hands of a few to the merits of us all, where there is no one centralized power. Decentralization of technology comes with challenges of its own, and if we narrow down the biggest challenge of all, it is the data.
In this post, the first in the series, we will try to look at the overview of Web3 Analytics and some core concepts that will help us get familiar with the terminology of Web3 and Blockchain.
Why is Etherion Talking about Web3?
I believe the world of decentralized internet and dapps will grow exponentially in the coming years, which brings about a greater need for data analytics in the decentralized world.
Web3 presents a unique challenge for the Data Analytics world, where the traditional definition of data is extended from readable, understandable, and user-centric to a wide range of information regarding network, wallet, addresses, and data assets. There is a need in the market for experienced Analytics professionals to make the world of decentralization better understandable, one dashboard at a time.
Let's jump into the world of Web3 Analytics.
Web3 & Core Concepts of Decentralization
Blockchain put the word decentralization on the map for us all to take into consideration, a world beyond traditional technology.
The world of Decentralization is far beyond Blockchain, and there are other such protocols with specific governance & security policies that together form Web3.
The core ideas of Web3 are -
- Decentralization
- Permissionless
- Trustless
- Monetization
In summary, Web3 aims to provide users with the ownership of their assets that are free of censorship, easily monetizable, and identifiable as part of the network.
What are the Use Cases of Web3 in the current market?
The use cases of Web3 are numerous, and as organizations and individual developers alike begin to move towards total ownership of their data and applications, we will see numerous more use cases.
In the present market, as far as my knowledge goes, we have the following use cases -
NFTs
AN NFT or Non-Fungible Token is a unique digital item stored on a blockchain that can represent anything, such as art form, photos, writing, etc., and serve as a digital record of ownership. It can very well be used as an identification for Intellectual Property.
Decentralized Finance (DeFi)
Decentralized Finance, or DeFi, is a financial system based on Blockchain Technology. A potential free and open market without the need for any regulatory bodies that allows for peer-to-peer transactions. Primarily based on Ethereum, one of the public blockchains that enables organizations to govern and secure these transactions with smart contracts.
CryptoCurrency
One of the famous use cases of Blockchain is probably the well-publicised alternative currency called Bitcoin. While it is not the only cryptocurrency, and many have popped up with the initial coin offering and it remains one of the most popular ways to monetize and transact.
dApps
dApps are short for decentralized applications. As the name suggests, dApps utilize the decentralized protocols and operate in a peer-to-peer network on the blockchain.
DAOs
DAOs refer to Decentralized Autonomous Organizations. As the name suggests, these are organizations governed by the principles of decentralization with a common mission where there is no sole ownership. Decisions are governed by proposals and voting on the chain for transparency.
While these are some broad categories of use cases for Web3, there are ongoing innovations both on the application and protocol layer that aim to make this technology more accessible and user friendly to general population while also making it easy for developers to build exciting stuff.
Overview of Web3 Analytics with Tools & Methodologies
Web3 Analytics is a process of understanding user behaviour and interactions on the decentralized web to gather insights about the platform and applications.
As it is with any analytics project, it is important to understand the objective of the project beforehand. If not, it is very easy to get stuck in the maze of information, creating dashboards that provide no real actionable insight.
Before we jump into different types of analysis that can be performed on a blockchain, let's try to understand how data really flows in a blockchain network.
In Blockchain, data is segregated mainly in two ways: On-Chain & Off-Chain.
Remember, data is distributed across multiple blockchains, and there is no easy way to identify the participants and the events in the data, as privacy is at the core of the decentralized technology. For a Data Analyst, the data that is available to be analysed from the blockchains is usually wallet addresses, smart contract events, token transfers, transaction addresses, transaction time, etc.
The type of analysis and tools needed to analyze this data vary, and a data professional should be able to differentiate between on-chain data and off-chain data.
On-Chain Data
- Transactions or Token Transfers
- Smart Contract Events or Logs
- Digital Assets or Wallet Swaps
- Network Consensus Data
Off-Chain Data
- User or Wallet Behavior Data
- Behavioral Insights
- Traditional Analytical Methods
In a decentralized world, privacy is a major consideration, which means users control the data that is shared, and that may have implications for data analytics.
Web3 Analytics Methodologies
The benefits of Blockchain and many other decentralized protocols have prompted traditional organizations to explore the use cases in mainstream applications.
As the adoption of decentralized technology increases, the challenge to extract actionable insights grows with it. Traditional analytical methodologies are fine, but there is a need for a better strategy to understand data from decentralized networks.
Let's look at some of the methodologies that can be used for analysis.
On-Chain Data Analysis
On-Chain Analysis involves analysing the blockchain data to gain insights into the network and identify trends. Key metrics such as transaction volume, smart contract events, and token transfers.
On-Chain Data Analysis helps in identifying risk, providing short-term and long-term investment insights, market timing, and patterns of price movements.
Tools like Dune Analytics, Flipside, The Graph, etc., can help in the analysis of data on the chain.
Cross-Chain Bridge Analytics
This involves monitoring asset flows and user behavior across multiple chains. The challenges with data analysis with cross-chain bridges are that the data can be fragmented, and the transaction paths can be obscure, with each chain having its own rules & security policies.
Tools such as Chainalysis & Wormhole, among others, help to make such cross-chain bridge analytics easy.
Data Indexing & Querying
Data in Web3 is more of a structured record that provides enough information about the transaction or the block. It is stored in a linear and immutable manner. To access data in a block, you may have to scan the entire chain to find the specific block.
Indexing involves creating a structured database with the information about the blocks in the chain that can be easily queried and retrieved without any computational overhead.
Tools like The Graph and SubQuery help in creating a schema-specific graph to easily access the data.
Wallet Clustering & Attribution
Blockchain thrives on anonymity and privacy of the activity on the chain. The problem is that it may enable bad entities to hide behind the anonymity layer.
Wallet Clustering offers a way to link the wallet addresses, transaction addresses, and network information in a chain to derive insights and leverage attribution platforms for heuristic analysis.
Tools like Nansen and Arkham offer wallet clustering tools to trace the transactions from one wallet to another.
Protocol-Specific Analytics
Protocol-specific Analytics involves analyzing data for the specific protocol to understand the various aspects of the network. It means the analysis is specific to the protocol and requires subject-matter expertise on the network.
For e.g., DeFi Analytics involves analyzing the data for the health of the network as per protocols. Metrics such as total borrowers, lenders, and volume can help to quantify the analysis.
Tools like Dune Analytics and DeFiLlama can help in creating protocol-specific quality checks on the network for real-time feedback.
Governance & DAO Analytics
DAO or Decentralized Autonomous Organization is an organization managed as a whole or in part by a decentralized protocol. It means the proposals are managed by voting and the finances are maintained in a distributed ledger technology like Blockchain.
The data in DAO is mainly centered around metrics like Treasury, Governance, and Holders. The data analysis provides insights into revenue streams, financial health, proposals created, voter power distribution, etc..
Tools like Dune, Tally Docs, and Boardroom allow organizations to be data-driven to make informed decisions.
Maximal Extractable Value & Transaction Ordering
Maximal Extractable Value means generating value by including, excluding, or reordering transactions in a block.
Methods like Frontrunning, Backrunning, and Sandwich Attacks can be used to understand the value generated from specific behaviors of the blocks.
Tools like EigenPhi and FlashBots help in achieving transaction ordering to study specific behaviors of the blocks.
Tools & Libraries for Web3 Analytics
Dune Analytics
Flipside Crypto
Nansen
Glassnode
The Graph
Etherscan
Ethereum ETL
DeFiLlama
While other tools and libraries can be used as required, some of the traditional libraries, like pandas/matplotlib, can be used for analysis.
In the next post, we will also have a look at some of the modern tools like Estuary, DuckDB, and Motherduck with Google BigQuery for off-chain analysis.
How To Get Started with Web3 Analytics?
It can be quite overwhelming to enter the world of Web3 and see that everything looks like the most sophisticated cryptocurrency ever created to replace real-world money, or matrix-like, like ever-changing, forever-scrolling numbers.
A basic strategy to get started with Web3 Analytics is to begin on a familiar path and then venture out into the big, bad world of Web3.
Here is a basic framework akin to the traditional Data Analytics from the old world.
Data Gathering
Everything starts with finding valuable data to analyse and extract insights that can be shared and used in a meaningful way. While the commercial tools like Dune Analytics, Nansen, and Flipside Crypto provide a ready-made platform to query data from multiple chains, some libraries can be used to extract data from Blockchains for heuristic analysis.
Libraries like Web3.py, Etherscan, The Graph, and Ethereum ETL can be used to extract transaction logs, smart contract events, and token information.
Data Cleaning
Privacy is at the core of Web3. Hence, the data is in a format that is not easily readable.
The main task here is to extract the hashed data and convert it into an understandable format that can make sense to the analysts and the end user. The structure of information on a blockchain is often nested, and as a data analyst this information needs to be flattened and only relevant information needs to be extracted.
Once the right data is identified and extracted, normalization can be performed across the entire dataset by labeling the data and correcting the format.
Libraries like Pandas, ETH-Utils, and Ethereum ETL can be used for this purpose, while storing the information in databases like DuckDB for quick exploratory analysis.
Exploratory Data Analysis
Once the Dataset is available in a database, using Jupyter Notebook or any other preferred platform can be used to perform the initial analysis.
Typical analytical methodologies, like descriptive statistics, are used to understand the transactions per wallet, token amounts, and gas fees paid can be extracted and visualized. Wallet Behaviour Analysis can be performed to detect in anomaly or outliers that can be highlighted.
Libraries like pandas, matplotlib, numpy, scikit-learn, plotly, and methods like isolation forest and graph analysis can be performed to make sense of the data.
Reporting & Dashboarding
The data is only insightful and useful for decision-making if presented in a proper context.
Tools like Tableau, PowerBI, Looker, and libraries like streamlit and observable can be used to create Dashboards to provide in-depth analysis and decision-making capability for the end user.
Where does Etherion stand in the Web3 Analytics World?
Etherion brings decades of experience into the field of Data Analytics and Governance.
While the world is rapidly moving towards AI adoption, and rightly so, with the current models solving so many of the problems that the users have been struggling with for a while, the team at Etherion appreciates the innovation in the Web3 space carried out by communities of developers looking to change the way we look at the internet.
Etherion wants to bring decades of data analytics and governance experience into the world of Web3 analytics to build products that look to solve real-world issues.
To Know More about Etherion - www.etherionconsulting.com
Key Takeaways
Here is a recap of the post -
- The world of Web3 is more than blockchain, which is expanding at a rapid pace and needs better actionable insight to build better products.
- Web3 Analytics is a growing field with established players trying to decomplicate the blockchain data for better insights.
- Overview of tools and strategy to get started in Web3 Analytics familiarly and easily.
- List of tools and libraries to use for the analysis of the blockchain data.
- Etherion Consulting is looking to dive into the world of Web3 Analytics by means of using commercial and open-source tools to develop managed solutions.
Next Steps
This post is the first in the series of Web3 Analytics, where I intend to cover different methodologies and tools in detail.
Implementing analytics in Web3 may seem like a challenging prospect, but Etherion aims to make it easier for individuals and organizations looking to venture into the world of Web3.
Please contact us for more information.
