Tokenguard User Segments & Conversion Insights - LTIPP Research Bounty Proposal

1. User Segments & Conversion Insights for LTIPP DeFi protocols

1.1 Why is it important?

:white_check_mark: STIP program was a great way to incentivize DeFi protocol users and allowed to increase the most important ecosystem & protocol factors such as DAU, MAU, fees & TVL. The growth was seen across the whole DeFi segment in the ARB ecosystem and all DeFi protocols that took part in the program.

:x: STIP uncovered however a number of challenges, both for DeFi protocols as well as the STIP Council & Advisors. Questions and challenges that were most commonly repeated are:

  • What types of incentives were delivering the best results? Which DeFi protocols were most successful in attracting new users? (404 DAO, GMX/Saurabh, Wintermute)
  • What was the wallet retention & activity after STIP? Did users stay in the ARB ecosystem or did they leave after receiving funds? (Vela, Karel, Furucombo)
  • What was the real (normalized) influence of STIP-rewarded wallets on crucial metrics such as DAU, fees & TVL and how it compares to other chains? (Wintermute, Karel, Jones)
  • How to discover new ways of incentivizing users to make LTIPP more efficient than STIP? Which user segments were most valuable? (SeriousPeople, Stryke, Wintermute, SEEDLatam)
  • How to report and draw insights from an overwhelming number of bi-weekly statuses and data infrastructure? (Vela, Umami, Savvy)

We believe that these challenges stem from two issues:

:x: DAO lacks one, universal source of decision-making insights based on easy-to-read comparable metrics for all supported DeFi protocols, enabling to compare them with DeFi segments in other ecosystems.

:x: DeFi protocols lack solution to independently discover most valuable user segments and improve their incentivisation strategies on their own. Dune and in-house data tools don’t allow for quick and comfortable exploration of user segments & their conversions.

1.2 Solution for the DAO

:x: With +100 DeFi protocols delivering status updates bi-weekly, it’s impossible for LTIPP Council & Advisors to make informed decisions. Dune is a great multi-tool but delivers raw data rather than insights, complicating decision-making even more.

:white_check_mark: Tokenguard delivers a simple growth leaderboard for all DeFi protocols in the LTIPP with all decision-making insights needed. It showcases the conversion of LTIPP funds into ARB ecosystem goals through a synthetic indicator comprising 6 predefined on-chain metrics, allowing for fast decision-making with regard to all protocols.

Decision-making metrics

  • New TVL acquisition per $1k ARB spent
  • Avg TVL retention
  • New wallet acquisition per $1k ARB spent
  • Avg wallet retention
  • Volume increase per $1k ARB spent
  • Avg volume

All metrics will be calculated in relative and absolute values and weighted in 50/50 ratio to accomodate for growth efficiency and ecosystem impact. Metrics can be modified and their number can be increased / decreased to meet LTIPP Council’s & Advisor’s requirements.

Current public version of the DAO interface:

https://app.tokenguard.io/crypto-growth
https://app.tokenguard.io/dapp-ranking?blockchain=moonbeam

The above metrics were suggested after multiple discussions with LTIPP stakeholders. These metrics will be displayed as an LTIPP leaderboard for each DeFi protocol in the program. A LTIPP Index (0-100) of all these averaged metrics will be calculated and:

  • Projects above average (> 50) demonstrate higher efficiency in converting LTIPP funds into ARB ecosystem goals
  • Projects below average (< 50) demonstrate lower efficiency in converting LTIPP funds and should receive help from Advisors & OpenBlock & Tokenguard to redesign their strategy

To accommodate for differences in DeFi team’s expertise and their brand’s awareness, Tokenguard will also display the information about protocol size to indicate whether it’s an early-stage or experienced project. This factor should be included by the LTIPP Council & Advisors while making decisions.

Exemplary insights & actions

  • Having a single source of comparable metrics from all DeFi protocols included
    • → Focusing Advisor’s & OpenBlock’s engagement on protocols that utilize LTIPP rewards less efficiently
  • Calculating the ROI and LTIPP influence on crucial metrics compared to other ecosystems
    • → Understanding if LTIPP should be continued in its existent form and observing the results of programs from other ecosystems
  • Measuring TVL increase excluding rewarded ARB and understanding which user segments leave the ecosystem after being incentivized
    • → Modification of LTIPP and future programs to focus on user segments that have have highest ROI long-term

1.3 DAO Report

[added on 06/13/2024 13:00 CET]

Taking into account the feedback regarding the goal of the DAO in connection with LTIPP, Tokenguard team will deliver 2 types of reports to the DAO:

:white_check_mark: Monthly Summary Reports: Alongside the real-time DAO Leaderboard insights, we will provide monthly summary reports highlighting key findings, trends, and recommendations to the DAO on how to improve wallet & TVL acquisition within LTIPP protocols by showcasing most and least efficient strategies used by protocols.

:white_check_mark: Final Summary Report: We will deliver a thorough research report that includes detailed analysis of different strategies that worked best or didn’t work during the LTIPP program, answering the most important questions:

  • Most valuable wallet segments for the ecosystem (Who to acquire?): An analysis with regard to wallet conversion and churn pathways. Such analysis can focus the scope of future programs to wallets that have the highest LTV for the ecosystem and will allow to discover new segments (eg. DeFi users moving to Gaming dApps; New wallets flowing to Arbitrum from Eth). Potential outcomes:

    • ➝ Discovering the power-user segment responsible for generating most of the LTV in the ARB ecosystem
    • ➝ Focusing the scope of LTIPP and future programs on users that bring in value not only in the DeFi sector but others as well (eg. gaming, NFT).
  • Acquisition strategies with highest New Wallets & TVL (How to acquire?): Analysing off and on-chain strategies used to acquire new wallets (including off-chain performance marketing and social media strategies) in order to answer how to acquire new wallets best. Potential outcomes:

    • ➝ Understanding if off-chain marketing CAC is higher or lower than on-chain marketing CAC
    • ➝ Discovering non-web3 user segments that could be acquired to ARB (eg. Forex traders, stock market traders, web2 gamers)
    • ➝ Moving part of the future LTIPP budget to trackable web2 / web3 performance marketing campaigns (using tools such as HypeLab, Addressable)
  • Rewarding strategies with highest Avg TVL & Wallet Retention (How to retain them?): Analyzing behavior patterns of rewarded wallets vs strategies used to understand how to keep these wallets in the ecosystem.

    • ➝ Implementing new incentivisation strategies (eg. long-term reward distribution, DeFi loyalty programs)
    • ➝ Modification of DeFi architectures to improve wallet & TVL retention (eg. implementing Protocol-owned-Liquidity & bonding,

Answering these 3 most important questions will allow designing a new user inflow funnel and significantly improve LTIPP but also generate ideas on new incentivisation programs that will focus on newly discovered user segments that haven’t been targeted so far.

1.4 Solution for DeFi & dApps

:white_check_mark: Tokenguard solves DeFi & dApp challenges by delivering a standalone no-code user segmentation & conversion tool. With cross-protocol and cross-chain tracking, it allows discovering the most valuable user segments and their behavior within seconds, making it possible for DeFi & dApps to independently improve their strategies to increase revenue and minimize churn.

Workflow

  1. Add your protocol smart contract ABI files.
  2. Let Tokenguard map the ABI conversions (30 minutes).
  3. Build your own user segments based on:
  • Date joined / last interacted / no. of interactions
  • dApp interacted with
  • Wallet size and source of funds (CeX, bridge, on-ramp)
  • Interaction with specific smart contract (eg. LTIPP reward distribution)
  1. Choose units to compare (no. of wallets, tokens transferred, no. of interactions)
  2. Choose on-chain conversions from ABI (eg. adding liquidity, trading)
  3. Compare segments population and metrics
  4. Compare segments behavior within protocol and flow between protocols

User Segments workflow walkthrough available here:
user_segments_loom_thumbnail

Current public version of the DeFi & dApp interface: https://app.tokenguard.io/stellaswap/users

Exemplary insights & actions

  • Identify which out of +10 common incentive types of $ARB in the LTIPP program provides the greatest value
    • → Channeling higher % of LTIPP rewards to wallets with the most valuable conversions, thus increasing revenues and fees per DeFi
  • Understanding the most common ways for wallets to churn and what % of awarded ARB is leaving the ecosystem after LTIPP
    • → Minimizing churn by rewarding wallets that keep their funds in the ecosystem after LTIPP
  • Identify the user segment that brings the highest TVL and shows the highest retention
    • → Channeling a higher % of LTIPP rewards to user segments with the biggest value, thereby increasing long-term TVL and wallet activity

2. Research Team information

2.1 Team background

Tokenguard is an experienced team of 8 data analysts and developers who, through cooperation with several Layer 1 teams and by providing solutions for communities and dApps, have gained knowledge about the solutions needed to support the growth of individual DeFi protocol and, consequently, the entire ecosystem.

Founding team members:

  • Kamil Gorski: Co-founder & CEO - Bootstrapped tech company to $1m revenue in less than a year; Former hacker & dev with 7+ yrs experience; Conducted +20 security audits of protocols. (https://www.linkedin.com/in/gorskikamil/)
  • Jakub Rojek: Co-founder & CTO - Co-created web3 user engagement platform OpenLoyalty. Created blockchain products before Ethereum even existed. (https://www.linkedin.com/in/jakub-rojek-dev/)
  • Damian Rolek: Co-founder & CDO - Implemented Al/ML conversion & churn algorithms for e-commerces as a Senior Data Scientist at PwC; Co-built a DeFi analytics platform for investors. (https://www.linkedin.com/in/damianrolek/)

Previous work includes i.a.:

  • Polkadot - Delivering dApp & DeFi analytics for measuring user insights
  • Astar - Delivering ecosystem analytics for tracking dApps activity and efficiency of grant programs
  • Aleph Zero - Delivering dApp & DeFi analytics for tracking on-chain conversions
  • Web3Foundation - Delivering open sourced, drag&drop analytics tool for dashboard creations
  • Sygnum Bank - Delivering on-chain analytic reports on the status of specific coins & tokens

Blog posts about ecosystem activity:

Accelerated by:
:medal_sports: Outlier Ventures & Synergis Capital 18/09/2023: $200k pre-seed round
:medal_sports: Finalist of AWS Global Fintech Accelerator 14/12/2023

Tokenguard App:

2.2 Why Tokenguard is the best fit to research this topic

Tokenguard is a specialized platform delivering actionable on-chain insights for Ecosystem Managers and DeFi protocol builders. Our experience in collaborating with teams in these areas streamlines the process of understanding what is important from both from the DAO perspective—allowing for more effective assessment of whether funds spent yield results—and supporting DeFi creators by providing a tool that helps increase the effectiveness of awarded rewards.

:x: Cooperation with DeFi & dApps uncovered an unanswered need from their builders - to quickly discover most valuable user segments. Dune Analytics, DappRadar & Defillama are great tools but don’t allow for analysis of conversions and relative segments.

:white_check_mark: With Tokenguard, DeFi builders can create and explore tens of potential user segments in a matter of minutes with no engagement from the development team.

3. Budget

3.1 Requested amount

Total requested amount: 124 542 ARB ($117 070)

3.2 Cost breakdown

Milestones Tasks Cost
1 DAO Leaderboard Building DAO custom metrics; Creation of DeFi protocol leaderboard; Leaderboard implementation $34,400
2 DAO Report Building DAO custom metrics; Creation of DeFi protocol leaderboard; Leaderboard implementation $12,820
3 DeFi & dApp Interface Arbitrum One data aggregation and transformation; Setting up and reservation of indexing infrastructure; Integrating 87 DeFi & dApps included in LTIPP; User Segments and Conversions Insights for DeFi & dApps $69,850
Total $117,070

:bulb: The request amount needs to be paid upfront for implementation and data indexation.

Detailed budget calculation available here.

2 Likes

Hi @kamilgorski,

Thanks for the Research Proposal! At first glance, the requested bounty amount seems rather large and this is primarily because the DAO is paying for a product. One of the main goals we would like to see come out from these research bounties is prescriptive and actionable insights into what went wrong/right with STIP/LTIP etc. and explanations as to why this might be the case and how the DAO can fix this for the future. With your proposal, there seems to be no research report deliverable that addresses the above, and while we do think the platform can provide value it will still require additional time to go through its insights and draw conclusions. Lastly, other bounties have proposed to do similar calculations with a research report at a lower cost.

Hey @kamilgorski, Thanks for submitting the proposal.

Team/Background

TokenGuard’s founding team is composed of a group that has experience building applications detecting churn rates for e-commerce businesses, user engagement in a web3 setting, and understanding on what contract-level data is emitted. They’ve also worked with protocols such as Polkadot, Astar, Aleph Zero, and Web3Foundation with tracking analytics.

TokenGuard’s Proposition

They’re looking to develop a dashboard solution that would resolve the challenging issue of the LTIPP Councils and its Advisors being informed with ongoing changes within the LTIPP recipient protocols. With a lense of focus centred on how effective was ARB spent within these protocols:

  • New TVL Acquisition (per $1K ARB spent)
  • Average TVL Retention
  • New Wallet Acquisition (per $1K ARB spent)
  • Average wallet retention
  • Volume increase per $1K ARB spent
  • Average Volume

Each protocol will be measured with a singular index, ranging from 0 to 100, where > 50 is highly efficient, < 50 is lower in efficiency, this index would demonstrate how effective ARB expended was and for its goals.

Our Concerns

Having read their very detailed proposal, watched the demo loom video, and observed their clear KPIs and grant breakdowns. We distinguished the following issues:

  1. The limitation for dashboard is that the scope is defined only for a specific protocol and its contracts, rather than the Arbitrum ecosystem. Given the inflow of interest into Arbitrum from the LTIPP, if users go on Arbitrum specific protocol leave for another on Arbitrum, this should be recognised as a net positive.
  2. Some protocols would have token incentives; they are likely to see more growth because of the incentives and token generation events. How do you plan to differentiate that?
  3. How would you justify the growth of smaller protocols compared to larger ones? For instance, if a smaller protocol increases its TVL from $1 million to $10 million, it demonstrates significant growth. However, for larger protocols like GMX, an increase in TVL from $200 million to $250 million, while substantial in absolute terms, may appear less impressive percentage-wise and might suggest slower relative growth.
  4. We also believe the Optional Advisory seems to be out of scope for the LTIPP, since the consultations seems more like upsell and eliminates user acquisition cost that is expended by the Arbitrum DAO. It’s also difficult to measure how meaingful these consultations are for these DeFi people.

Hey @WintermuteGovernance !

Thank you for your feedback on our proposal! It’s valuable and we want to address your concerns:

  1. Actionable insights:
  • We understand the need for prescriptive and actionable insights - Tokenguard is a platform built with these on mind. STIP produced +200 bi-weekly reports that were supervised by the DAO but many of them couldn’t be translated into actions because it’s impossible to go through all the forum data and make informed decisions having in mind different DeFi architectures. With 87 DeFi (twice as many as STIP) we decided to provide a solution to simplify the supervision process of LTIPP beneficiary in the form of comparable growth metrics in one leaderboard instead of bi-weekly posts.

  • At the same time, we’re delivering a standalone growth tool for all 87 DeFi protocols to allow them improve their incentivization strategies all by themselves in a comparable & measurable manner. A synergistic approach, where DeFi projects have real-time access to insights that produce the best growth and DAO has this aggregated growth information at hand.

  1. Report:
    Taking into account the feedback regarding the goal of the DAO in connection with LTIPP, we would like to propose the following changes to our proposal:
  • Monthly Summary Reports: Alongside the real-time platform insights, we will provide monthly summary reports highlighting key findings, trends, and recommendations on wallet segments & most / least effective strategies in terms of their ROI.

  • Final Summary Report: We will deliver a thorough research report that includes detailed analysis of different strategies that worked best or didn’t work during the LTIPP program, answering the most important questions:

    • Most valuable wallet segments for the ecosystem (Who to acquire?): An analysis with regard to wallet conversion and churn pathways. Such analysis can focus the scope of future programs to wallets that have the highest LTV for the ecosystem and will allow to discover new segments (eg. DeFi users moving to Gaming dApps; New wallets flowing to Arbitrum from Eth).
    • Acquisition strategies with highest New Wallets & TVL (How to acquire?): Analysing off and on-chain strategies used to acquire new wallets (including off-chain performance marketing and social media strategies) in order to answer how to acquire new wallets best. (eg. this may move a part of the future LTIPP budget to trackable web3 performance marketing campaigns).
    • Rewarding strategies with highest Avg TVL & Wallet Retention (How to retain them?): Analysing behavior patterns of rewarded wallets vs strategies used to understand how to keep these wallets in the ecosystem. Such analysis can increase the efficiency of LTIPP spending by implementing new product strategies or architectures within protocols (eg. Protocol-owned-Liquidity).
  1. Cost:
    While our budget may appear high in comparison to others, it’s important to highlight that Tokenguard delivers a tool that simultaneously serves DAO and 87 DeFi protocols. However, having in mind that the goal of LTIPP Research Bounty is to deliver insights to the DAO itself, let us rework the budget to minimize the cost thanks to smaller number of DeFi (87 instead of +100 we expected) and focus more on delivering insights to DAO.

Looking forward to engaging into the discussion!

Best,
Kamil

Hey @Saurabh !

Thanks for raising these questions on methodology - they’re extremely important. Our answers below:

  1. Great point. Our view is that for New TVL Acquisition and New Wallet Acquisition metrics will be calculated from the ecosystem perspective because that’s the goal of LTIPP (acquire new users and TVL). If the wallet adds new TVL to a different protocol - this is included as New TVL Acquisition although not as New Wallet Acquisition (which was already attributed to the first protocol). Such methodology protects from double-counting the same TVL / wallet acquisition. This can however be adjusted according to LTIPP Council & Advisors.

  2. Proposed metrics regard only activities that are incentivized with LTIPP rewards. As long as LTIPP rewards aren’t used as TGE incentives, this traffic will not be included in these metrics.

  3. That is correct. This is why we include all metrics in this synthetic indicator - with TVL and wallet retention being usually higher for established protocols compared to smaller ones. To make this indicator fair, both absolute (to measure ecosystem impact) and relative (to measure strategy efficiency) values are included in a 50/50 ratio - we’ve updated that in the proposal.

  4. We added it as an additional service and didn’t include it in the inital budget request - deleted to make the proposal clearer.

Really appreciated answering these, looking forward to discussing the methodology!

Best,
Kamil

Thank you for your application @kamilgorski. The application was very thoughtful in addressing some of the concerns of the council. We do have some concerns:

Budget
Our biggest concern here is the cost and budget requested. Your original budget requested for north of 90% of the allocated research budget. This revised budget is still asking for close to 80% of the allocated research budget. Additionally, since Tokenguard caters specifically to DeFi projects, the budget is a greater concern, as the research would not cover all projects receiving incentives.

Analysis and Reporting
Another issue we have is around the lack of a deliverable focused on deriving conclusive analysis and recommendations to the DAO so that we can improve upon our current programs.

As for the leaderboard, how do you normalize the scoring for vastly different protocols(size/sector/age)? We believe some kind of segmentation is needed to add necessary context for scoring and analysis.

Hey @404DAO , thank you for the feedback! Let us answer below:

  1. Budget
    We have modified the budget in the proposal by adapting to a real number of projects (86 instead of +100) and focusing the scope of work on the DAO report to gain insights that are important for improving LTIPP and future programs. The request ask was decreased by 26% to $117k (124k ARB) according to your and @WintermuteGovernance 's comments.

  2. DeFi / non-DeFi projects
    Tokenguard User Segmentation & Conversions Insights is applicable to any type of on-chain dApp or protocol as long as it’s based on smart contracts. So any type of protocol & dApp (NFT, Oracle or Gaming, such as DoG Protocol or Knights of Ether) in the LTIPP will be able to use the tool with the same benefits.

  3. Analysis & reporting
    We’ve added a DAO report to the proposal for delivering crucial growth insights to improve LTIPP and future programs.

  4. Normalization
    As previously discussed with @Saurabh - the synthetic indicator in the leaderboard includes both relative (for measuring growth efficiency) and absolute (for measuring ecosystem impact) metrics. Absolute metrics accommodate for differences in the size and age of protocols.
    Regarding sector differentiation - we will be following the tagging framework designed by OpenBlock to keep everything coherent. Since OpenBlock is focusing on many sector-dependent metrics, we decided to present high-level ROI insights for faster decision making within the DAO (and to not overlap OpenBlock’s work).

Looking forward to your thoughts on this!

Best,
Kamil