STIP Monitoring - Objective 1&2- Arbitrum Attribution by AlphaGrowth - [FINAL]

[bryancolligan]

STIP Monitoring - Objective 1&2- Arbitrum Attribution by AlphaGrowth - [FINAL]

Background

For the past 2 years, AlphaGrowth has served ecosystems. We help manage & grow their grant & incentives programs. We recognize billions of dollars of grants has been spent pursuing ecosystem growth. Understanding effective grants deployment is multi-billion dollar attribution opportunity. AlphaGrowth was made to take on the opportunity. With the forward thinking STIP Monitoring request the stars have aligned. A perfect opportunity for our team to serve the Arbitrum ecosystem.

Some key questions we will answer over the length of the engagement are:

Were the grants used as specified?

How does an ecosystem measure the efficacy of a grant?

Which contracts are grants deployed to? (pools, staking, perps, yield aggregators)

What happened by onchain metrics? (Fees, Transactions, Users, DAUs, MAUs, Trading Volume, Staking, Locking, TVL)

What happened by offchain metrics?(Twitter, Telegram, Discord, Reddit, Github, Website visits)?

Which assets recieved incentivizes and what happened? (Price, Holders, MarketCap, FDV)

Most important. Which incentives were effective and which were not?

AlphaGrowth’s combined experience in the data & grants is best in class. This is our exact expertise. AlphaGrowth will provide it’s expertise and build a comprehensive reporting. We under the importance of this responsibility. The impact of the incentives program is a serious matter. We will help Arbitrum and the entire crypto industry with the findings and we do not take this duty it lightly.

A sincere thank you from our team for taking the time to read through our proposal.

Scope of work

To put it in simple terms, the monitoring program just wants to answer one question

Arbitrum has given $x to a project. 6 Months later, what has the project given to Arbitrum?

The STIP monitoring program contains two broad areas. Impact measurement, and use of funds.

To set scope of our contribution as a service provider, we map a high level outline :

• Program Discovery
  • Understand individual grantee grant intention
  • Understand grantee use of funds
  • Methods of ARB incentive deployment
  • Collaborate with Plurality Labs and Stable Labs on use of funds definitions
  • Publicly state commitments of Grantees and which contract incentives are deployed
  • Categorization and segmentation of grantees based on specific expectations of KPIs
    • By grant amount: Beacon, Siren, Lighthouse, Pinnacle
    • By Sector: Yield, DEX, Lending, etc.
• Data Pipeline:
  • Collect data points necessary for program assessment: smart contract details
  • Routinely obtain timestamps of the start of incentive programs by individual grantees
  • Feed the data to AlphaGrowth’s Arbitrum Attribution system.
• Open Analytics Dashboard:
  • A single point of reference for the Arbitrum community for status report on the program
  • AlphaGrowth will create a dedicated platform for the dashboards, while a Dune template will be distributed to grantees to display data, certain benchmark metrics are better represented with a custom dashboard
  • Dashboard to contain at least 5 different interfaces for different metrics
• Grantee Communication:
  • Create communication channels with grantees to facilitate progress and tracking reports
  • Weekly/bi-weekly sync with grantees, ensuring proper flow of data and timely reporting
  • Report grantee activity, fund utilization and other updates to community
  • Leverage community expertise for any issue that the grantee team may require help with
• Documentation & Records:
  • Separate from dashboards, there will be a data repository for single point of access of every grantee data(CSVs, potentially an API)
  • Maintain jointly by AlphaGrowth and individual grantee teams
• Bi-weekly reports & updates:
  • The monitoring team will be the primary point of contact for STIP team to get any updates
  • Bi-weekly sync with the STIP team
• Impact Assessment:
  • Research on qualitative and quantitative aspects.
• Final Program Impact Evaluation:
  • A grand summary of the program will contain detailed reports on a diverse range of metrics
  • To be presented to the community by the Program end on April 24

Overview

Open Analytics dashboard

The dashboard will be a great source for community members to have a quick look at the ecosystem and protocol metrics. The dashboard is projected to have the following data:

• Ecosystem global Metrics
  • TVL
  • Staking
  • Transactions
  • Users
  • Fees Generated
  • Volume
• Ecosystem sector Metrics
  • DEX
  • Lending/Borrowing
  • Yield
  • Perpetuals
  • Other Categories
• Protocol level Analysis
  • Pools
  • Wallet/new users increase
  • Fees generated
  • Dune Dashboards
• ARB token
  • Current grantee wallet
  • Which contract ARB token in grantee wallet went
  • Remaining granted ARB in grantee wallet

Grantee Communications

Set up communication lines: Create a telegram group between SP team and individual grantee projects.

The telegram group to include,

• SP Manager (AlphaGrowth) and Program lead from AlphaGrowth
• Grantee team members (primary contact, team member who filled the forum application, technical team, etc)
• STIP members - Optional & Situational (Stablelab, Plurality labs, Arbitrum foundation, Off-chain labs)

Maintain Data for grantee documents

• Individual data entry page: Lists out all the necessary information about the grantee throughout the program
  • Relevant wallet addresses
  • Smart contract addresses
  • Dashboard and data links
  • Progress reports/links

Bi-weekly sync

• Meet with Grantee teams to retrieve updates (either virtual meetings or async on telegram)
• Ensure that the data pipeline on grantee updates stays full
• Update STIP team (Stable labs, plurality labs, Arbitrum foundation/offchain labs and more)
• Enumerate the issues faced by grantees and the possible help that can be managed from the broader Arbitrum community

Final Program Impact Evaluation

Qualitative and quantitative reports on the program, its impact, protocol initiatives and growth. Here are some of the key points that will be covered:

• Arbitrum Comparative ecosystem performance metrics
• Comparative sector-wise performance: DEX, lending, Yield, perps, etc.
• Grant usage statistics
• Spending metrics
• Underperforming stats, projects underutilizing funds, projects misusing funds, etc
• Program Findings and Recommendations
• Grantee ROI analysis
• Top Performers

Value Definition

Ecosystems grants programs should be run like an investment firm. Any grant from the firm should generate positive returns for the ecosystem or its participants. Grants are the key components of growth for ecosystems. They attract developers and incentivize particular desirable behaviors. Any program that is assigning money/resources will generate a certain degree of activity. There are 72+ confirmed grant programs currently active in the Web3 industry that we know of. But what makes a grant program successful? After all, resources are finite. Grant programs should not only have an impact on the ecosystems, but they should sustain the impact even after the initial acquisition grant has been spent.

Our focus of analysis will be to gauge the magnitude of impact generated by the incentive program, and how much of this impact will be sustainable even without follow-up grants.

Quantitative Impact

• Key metrics of impact of the program. Mapping of different aspects of impact in users, TVL, on-chain activity.
• Easy access to outcome data: The community should have access to the impact metrics, the more set of eyes, the better evaluation we get.

Qualitative impact

• Grant program outcomes roughly follow the Pareto principle. Our aim is to identify the relevance here.
• Grading: Segmenting grantees on starting positions (Grant size, ecosystem dominance, etc), Gauge whether the impact is proportional to their allocation
• What intrinsic and extrinsic factors help a grantee perform better

Behavior Incentivization & Continuity

• Which grantees were able to motivate users to take a desirable planned behavior on the app/project.
• How does the behavior continue over time when incentives no longer exist?

CAC

• Spending vs user acquisition
• New users over time

Profitability and Sustainability

• Is the program generating enough fees, revenue or other tangible benefits that match the overall cost?
• In the long term, is it forecasted to keep generating fees, etc even in the absence of an incentive program?

To sum it up, we will measure the value of the program using the AARRR Pirate metrics.
Acquisition: User growth, activity boost, CAC, etc.
Activation: Behavior incentivization + increase in stats like TVL
Retention: Continuity in user behavior
Revenue: Profitability and sustainability
Referral: N+1 Viral Co-efficient and user-2-user onboarding. Not within the scope with the current ask.

Execution strategy

There will be two main points of contact from AlphaGrowth, Program Manager and Program Lead along with the technical team that will create the analytics system. The program manager is the dedicated resource from AlphaGrowth to report to the STIP committee.
For better reporting, the program is divided into a number of cycles. Each cycle spans for a period of 2 weeks. Each month is divided into two cycles called Cycle 1 & Cycle 2. Thus updates will look like November Cycle 2 update, March Cycle 1 update, etc. This architecture ensures the simplicity and effectiveness of the operations. The STIP committee will be updated on the first Monday after the end of the ongoing cycle. This will be done in the form of a virtual meeting, followed by a public update to the community. The reports will go out on alternative Mondays.

Funding & Milestones

For building, managing, and communicating the data aggregation and impact analysis project for Arbitrum grants - To build and run this project over the next 6 months, our team is requesting a total budget of 85,000 ARB (~$68,000 at ARB cost on proposal submission). The reason we can price our ask so low is that we have invested over 6 figures into our development over the last 2 years to get to this spot and Arbitrum gets full benefit of our pre-existing infrastructure. The requests are divided into 3 main components

• Infrastructure costs: $8,000 - For hosting, Frontend Dev, Dune, APIs, UI/UX, computational resources, dashboard, reporting system, blockchain data providers, other third-party services
• Technical resources: $ 28,000 2 engineering resources
• Program resources: $ 32,000 3 resources, Product Manager, Operations manager and 1 Research Associates

Milestones and disbursements

Milestone	Description	Completion	Disbursement
Kickoff	Initial funding to kickstart the program and mobilize resources	Immediately	20K ARB
Milestone 1	Basic Dashboard for Arbitrum Chain	December Cycle 1
(Before 15 Dec 2023)	15K ARB
Milestone 2	Advanced Dashboard for protocols	January Cycle 1
(before 15 Jan 2024)	10K ARB
Service Period 2	Invoice for service of 4 cycles spanning the months of January & February 2024	Feb 26, 2024	20K ARB
Service Period 3 & Final report	Invoice for service in 4 cycles in March & April+ Final report	April 22, 2024	20K ARB
			85K ARB

Potential bottlenecks

As stated by the original RFP, there may be some friction points

• Variable start: Different KPIs and different timelines for the start of programs by grantees.
• Inadequate or no communication from grantees
• force majeure - Unforeseeable circumstances in any aspect of the program or external incidents

Potential learnings

• As the program proceeds, the reporting standards may get more insights.
• Learning that some processes are inefficient, we may adapt different set of technical processes, all of which will be conveyed to the STIP committee
• KPI and success measurement may differ for each grantee

Why AlphaGrowth

Operating since 2018, AlphaGrowth has years of experience in DeFi, Data and Crypto BD, we have worked with multiple protocols including Kava, Sommelier Finance, NEAR, Aurora, and many others.

A prerequisite for our operations is having a lot of data at our disposal. More data allows us to make better decisions and hence better impact. To better analyze the on-chain and off-chain data, the AlphaGrowth team developed internal systems to monitor the markets. We are ready to use this mountain of data to build the best system for the Arbitrum STIP program.

The dashboards will be developed as proper platforms. While we believe DUNE analytics provides good data, it might not be very well suited for data analysis and user journey for the monitoring program.

Below we have provided a few screenshots of our existing/WIP dashboards. Anyone should be able to visit the links and see the dashboards. We already have a functionality to filter Arbitrum projects based on TVL, Marketcap or Users. Further users can segment the data based on different categories like DEX, Yield, etc.

To back our proposal, we have linked some of our past and present works on Arbitrum analytics

• Dashboard Screenshots
• Dashboard and Analytics links

Arbitrum on-chain data | AlphaGrowth

[Arbitrum DEX ranking by TVL]

https://alphagrowth.io/projects?sort_by=tvl&sort_order=desc&sort_chain=arbitrum-one

[Yield projects on Arbitrum by TVL]

AlphaGrowth data on Chains

Arbitrum Ecosystem Map: Ecosystem by Category

Arbitrum Token Optionality

Resource	Link	Description
Arbitrum Crypto Analytics & User Data	Arbitrum Crypto Analytics & User Data | AlphaGrowth	General user data and Analytics on Arbitrum Chain
Arbitrum One Ecosystem and On-chain metrics	https://alphagrowth.io/arbitrum-one/ecosystem	Ecosystem apps on Arbitrum, TVL and transaction data on Arbitrum
Highest TVL Projects on Arbitrum	https://alphagrowth.io/highest-tvl-arbitrum-one	Projects ranked by TVL in Arbitrum
Arbitrum Ecosystem Saturation	How saturated is Arbitrum | AlphaGrowth	What are the most populated sectors on Arbitrum, what does it not have
Fastest Growing projects on Arbitrum by Twitter activity	Fastest Growing Project on Arbitrum by Twitter Activity | AlphaGrowth	Twitter activity and growth of Arbitrum related projects
Chain Comparison: Comparing Arbitrum to Optimism	Arbitrum vs Optimism | AlphaGrowth	Comparing general data about Optimism and Arbitrum
Arbitrum Detailed Dashboard	https://alphagrowth.io/projects?sort_by=tvl&sort_order=desc&query=&sort_category=dex&sort_chain=arbitrum-one	Ecosystem dashboard for Arbitrum

About Our Team

The folks that will be working on the Monitoring program. More about the members can be found on their linkedIn & Twitter profiles

• Bryan Colligan, Co-founder of AlphaGrowth.
  • Program Lead and Technical product manager for STIP Monitoring
  • Twitter, Linkedin
• Joe Bjornsen, Co-founder of AlphaGrowth
  • Project Manager for STIP Monitoring program
  • Linkedin
• Shardul Pilley, DAO & Governance Lead at AlphaGrowth
  • STIP Monitoring Program Operations Manager
  • LinkedIn

AlphaGrowth team is very excited to be a contributor the the Arbitrum Ecosystem’s growth. We are confident that our engagement with the Arbitrum community will help the community get a better understanding on the grant initiatives. Our working principle focuses on coordination, between us and different stakeholders within the ecosystem and we are excited to collaborate with other teams as well to get the best for the monitoring program.

For any queries or remarks, please feel free to write in this forum or collaborate with our team through telegram:
https://t.me/+CbjBLWJJ9N4xMjgx

Crypto Deal flow for Ecosystems and Investors | AlphaGrowth

[Burns]

Thank you for the well detailed proposal. Can you please confirm/answer the following:

1. Proposal does contain any of the Objective 1 inclusions except in the final report. Is this meant to be covered?
2. Is grant matching and ROI to be included and tracked or summarised at completion only? Define ROI if so.
3. Proposal mentions a bi-weekly sync and update, what is the intended format of this? Report/meeting/async comms? From the timeline, there appears to be 12 total.
4. Will dashboards be open source?
5. Financials don’t seem to add up (20 + 5 + 10 + 20 + 20 = 75k ARB, not 85k). Please confirm.
6. Can you confirm if there will be additional contributors to the team for this workload - are the engineering resources known? 7. What is the capacity for the team to deliver on these dashboards, reports and STIP recipient data coordination?
7. Does the team have documented experience advising protocols with data driven recommendations for incetive programs?

[sharp]

Hey @Burns , Shardul from Alphagrowth team here

1. Yes. In addition, AG team will monitor the grant and the flow of the grant to the deployment contract. We will communicate with the 29 teams which contracts will deploy the incentives. The entire scope of objective needs more discovery so this part is negotiable and depends on the desired solution.

2. For ROI we break down the ROI in the form of following these analytics:

   Ecosystem global Metrics

   • TVL

   • Staking

   • Transactions

   • Users

   • Fees Generated

   • Volume

   Protocol level Analysis

   • Pools

   • Wallet/new users increase

   • Fees generated

   • Dune Dashboards

   For grants matching not sure the exact question being asked. This requires a couple discovery calls. We can record grant matching data from the grantees for every cycle and the STIP team will be updated accordingly. It is understood that in most cases, grant matching will neither be regular (deployment of matching grant for every cycle) nor even (different volumes deployed over different cycles). Many grantees have not publicly committed to matching funds, we expect them to make some form of matching and will be continuously tracking in bi-weekly syncs (cycles). Another interesting phenomenon is a spending spree in many public organizations at the end of fiscal year due to the use-it-or-loose-it budget rules. where the public departments go on a spending spree at the end of fiscal year to fully utilize budget funds which will otherwise be returned back after end of fiscal year. There might be a similar application here that certain grantees may commit disproportionate amount of matching funds during the ending of monitoring program which may additionally skew results. For these reasons, the team thinks it is necessary to document Grant matching commitments and efforts by grantees for each cycle. Short term ROI as a change in metrics will be mentioned in bi-weekly sync. Full analysis of the program is only possible in the long term. A comprehensive ROI analysis will be included in the final report.

3. Bi-weekly virtual meetings (google meet) . The AG team will be available for async comms throughout the cycle (2 week time between reports) for the full program duration.

4. The Data will 100% be available via APIs and Download extract. Since we already have existing tooling and speed to market we aim to have a open source frontend site at the end of the engagement. The main factor will be timing, hosting and maintenance discussion.

5. Apologies for the confusion. Second entry (Milestone 1 - Basic Dashboard for Arbitrum Chain) was supposed to be quoted at 15K ARB, but was accidentally written as 5K ARB. This was an input mistake, we have adjusted the table accordingly

6. We mentioned the key program touchpoints in the proposal. The Alphagrowth team members that will be working on the program behind the scenes apart from the 3 members already mentioned are as below. Full list of team members can be found here: Team | AlphaGrowth

   Vitalii S: Partner Success @ Alphagrowth. Grantee relations for Monitoring program

   Kyril V: Product and Research @ Alphagrowth. Research Associate for Monitoring Program

   Jose A: BD @ Alphagrowth. Grantee communications lead for Monitoring Program

   Michael B: Tech lead @ Alphagrowth. Dashboard & Platform Lead for Monitoring program

   Oksana V: Research @ Alphagrowth. Research associate for Monitoring program

   As mentioned with our prior work experience in end of the proposal, the team has full capacity to deliver on its commitments. Should there be a requirement beyond the 9 members, the team can very well add members from the network of grant professionals.

7. Yes! Data is the first step in our interaction with Ecosystems. Our latest engagement with Kava, where AG team was tasked with sourcing Grant applicants for KAVA Rise program. Alphagrowth sourced a total of 2100+ applications, out of which we brought 89 applications to Kava. The impact was an increase in TVL on the relatively new chain by $80M (Around ~40% Chain TVL sourced through the grants program). This is where we analyzed the grant efficiency, the application of pareto principle (80/20 success) and more. Furthermore, the applicants were managed on the Alphagrowth platform itself, showcasing the robustness of the platform to easily handle more than 2100 applications. Each of these applications was analyzed by our research team before giving our recommendations to the KAVA team.

[bryancolligan]

@burns thank you for your questions. We want to be sure to be clear that we will be working on the Objective 1 for the STIP monitoring proposal. We have had a ton of internal discussions on how to quantify and qualify what how to meet and beat this objective.

The first item is that we will 100% build tooling and dashboards to track:

1. the grant deployement
2. the deployment from the protocol multsig
3. the deployment to incentive contracts on the different pools and protocols
4. the funding flows up to when the claimant receives incentive

Where we started to debate internally on was on the following request “claimant analysis, and any other signals”. We have 3 options for this and would love feedback.

The first option, is outsourcing this objective. Chainanalysis specializes in exactly this type of request wallet tracking. For the past 10 years Chainanalysis #1 priority is to track wallets, mixers, transactions and suspicious activity. Having understood a little under the hood what they built this is likely the best wholistic solution to detect suspicious activity around the incentives and token distribution. The amount data and resources needed for Circle node wash trade detection or Neighbor node detection is not something to take on lightly. That combined with trying to track and unwind distributions to privacy mixers is a specialist undertaking. Chainanalysis has an off the shelf product as well as post facto investigative services for hire.

The second option is that we can track claimant wallets and ARB token on a dune dashboard. The dashboard will contain stats on wallets that received ARB and how much across activities and protocols. This way, if needed, we can retro actively do an analysis on the on the least effective incentive programs. Which brings up option #3.

Third option. From our understanding, the goal is to track incentive effectiveness. Learning how protocols are using incentives to grow and sustain KPIs are more important than the 1000 ways not to use incentives. Tracking all the brilliant ways claimants and potentially protocols will try to game the system while fun may not be incredibly useful in truly understanding exactly the best way to deploy incentives. Things like team farming, when pooled into the base protocol, is beneficial if it helps make the protocol more capital efficient. The responsibility lies with the protocol on how to deploy incentives correctly. If the protocol growth is limited from misplaced incentives, whether malice or negligence does it make a difference? Ultimately the protocol reputation is on the line.

Looking forward to feedback on which path the STIP committee is interested in pursuing.

[sharp]

@Burns Here is our initial stip monitoring dashboard that we aim to push live