STIP Data Monitoring and Reporting - Objective 1&2] - Sixdegree - [FINAL]

Archive PL Data Monitoring Procurement
1

The entire Sixdegree team is super excited to present its proposal to RFP - Abitrum Short-Term Incentive Program (STIP) Data Monitoring and Reporting

We are looking forward to hear back from the community and from @DisruptionJoe @tnorm @Burns & @epowell101 who drafted this RFP.

Thank you.
0xetwas / Sixdegree team

Understanding of Scope

  • The purpose of this project is to establish a fair and equitable method to ensure the efficient functioning of the Short-Term Incentive Program (STIP)

  • Develop a data analysis framework to facilitate potential duplicates, benefiting the Arbitrum DAO in the long run

  • Demonstrate the effectiveness of the STIP and identify the most efficient incentive strategies in a transparent manner

The scope of the project can be divided into the following components:

  1. Monitoring and Reporting

    • Monitor the utilization of grant funds for successful STIP projects as per the proposed guidelines

    • Promptly report any instances of unethical behavior or fraud, e.g. Swapping Arb into other assets or transferring Arb in an illegitimate way

    • Analyze the claimant cohort to promptly detect and flag potential instances of incentive farming or wash trading

  2. Data analysis on the impact of STIP

    • Analyze the impact from the perspective of Arbitrum network and individual grantee projects, respectively

    • Track key metrics such as Total Value Locked (TVL), user count, transaction volume, etc…

    • Focus on growth metrics like retention rate and Customer Acquisition Cost (CAC), Lifetime Value (LTV)

    • Compare the status before, during and after the implementation of the incentive program

  3. Communication with the Arbitrum community and other stakeholders

    • Integrate individual data reports into a single comprehensive Dune dashboard

    • Provide bi-weekly updates to the Arbitrum community, ensuring the dissemination of the latest information

    • Utilize open-source dashboards and analytical tools to showcase the effectiveness of the STIP, thereby enhancing the transparency of the Arbitrum DAO

    • Address the concerns or uncertainties raised by the community

  4. Conduct a final comprehensive report on the STIP

    • Evaluate the effectiveness of the STIP

    • Identify key takeaways and lessons learned to inform future incentive programs

Key Signals and Metrics to Monitor

Screenshot 2023-10-14 at 08.31.13
Screenshot 2023-10-14 at 08.31.131254×634 57.9 KB

Design - Open Source

  • All the dashboards, analyzing methods, data sourcing methods, alert bot development will be fully open sourced

  • The OS design architecture will enable us to create long-term value for Arbitrum community

The details of work breakdown and the approaches

output
output1920×822 142 KB

Objective 1 - Reporting and Monitoring

  1. Track fund flow

    • Implement a customized monitor dashboard on Dune to track the flow of the funds

  2. ARB transfer/swap alert

    • Develop a Dune dashboard to monitor the grant funds usage

    • Funds inflow and outflow

    • Claiming process

    • Develop a discord alert bot to detect and report $ARB transfers/ swaps from grantees

    • Abnormal liquidity transfer

    • Abnormal rewards claim

  3. Flag claimant risk

    • Develop a method to identify suspicious claimant behaviour

    • Build a Dune dashboard to illustrate all claimant behaviour

    • Basic metrics about the claiming process

    • An overview and break down into protocols

    • identify rewards re-allocation

    • Identify wash trading behaviour

Objective 2 - Data Aggregation and Impact Analysis
Objective 2 - Data Aggregation and Impact Analysis1920×823 114 KB

Objective 2 - Data Aggregation and Impact Analysis

  1. Data Aggregation

    • Aggregate data / dashboards provided by grantee protocols

    • Build an all-in-one Dune dashboard to demonstrate all metrics

    • Organize the dashboard in a logical manner for ease of readability

  2. Impact Analysis

    • Build a Dune dashboard to track the key metrics such as TVL, number of transactions, gas consumption, user growth, volume, retention rate, etc…

    • From the view of Arbitrum network at a high level

    • Group by sector

    • Group by protocol

    • The impact analysis will compare the status before, during and after the incentive period

  3. Behaviour Analysis

    • Design a scoring system for grantee protocols

    • Criteria include user size, growth, economic value, activities, user stickiness, retention rate, etc…

    • Criteria will be adjusted depending on the specific sector the grantee project belongs to

    • Design a scoring system for users of grantee protocols

    • Criteria include maturity, active level, gas consumption, allocated assets, governance, etc…

    • Criteria will be adjusted depending on the specific sector the grantee project belongs to

    • This way we can have a benchmark for better understanding of the efficacy of STIP

    • Build a stand-alone website to showcase the results for better understanding and user experience

    • The data source for the website will be retrieved via the Graph, enriching the diversity of the data source and enhancing transparency

  4. Capital Utilisation Efficiency Analysis

    • Calculation of ROI in terms of Arbitrum as a whole and breaks down into sector and grantee protocols

    • Average cost of

      • Acquiring new users

      • Activating existing users

      • Adding TVL per unit

      • Long-term retained users (post STIP)

      • Long-term retained TVL (post STIP)

    • Comparison among all protocols / incentive methods

    • Compare all the costs mentioned above

    • Investigate the best incentive method in each sector

  5. Claimant Analysis

    • A. Build algorithms and to understand the user behaviour of claimants**

    • B. Build the Dune dashboard and conduct a report to elaborate on**

      • How claimants deal with the rewards

      • Hold

      • Transfer

      • Sell

      • Delegate

      • Cohort analysis

    • C. Build the Dune dashboard and conduct a report to elaborate on

      • Overlap of claimants among STIP protocols

      • Percentage of claimants holding $ARB

      • Incentive farmers

      • Cohort by volume

      • Cohort by balance

      • Cohort by activities

Value Definition (Priority 2 only)

In order to achieve the goals of the STIP, which include promoting ecosystem growth, experimenting with grant distribution models, gathering data for future programs, and discovering new strategies to boost network activity, it is important to focus on the following key concepts:

  • Ensuring proper utilization of funds

    • A crucial aspect of a successful program is the appropriate allocation of funds as outlined in the proposal, while also preventing any potential misuse

    • To accomplish this, we will closely monitor the fund flow in a real-time manner

  • Evaluating program success using key metrics

    • It is essential to gauge the success of the program based on measurable metrics

    • Key metrics to track include TVL, number of users, transaction, volume, gas usage, and generated fees

    • We will adjust for each protocol the right KPIs to correctly estimate the value they bring

  • Setting benchmark to gauge the efficacy

    • In order to understand the benefits brought by STIP, we have to set a benchmark for better comparison

    • Through comparing the status before, during and after STIP we can quantitatively evaluate the efficacy

    • The value is not only for protocol growth, but also for maturing users, i.e. Does STIP help users better dive into Arbitrum ecosystem?

  • Capital utilization efficiency

    • It is critical to calculate the ROI

    • Average cost of acquiring new users, activating existing users, adding TVL, etc…

    • Compare the CAC with other incentive campaigns such as liquidity mining and airdrop

    • Track the long-term value that STIP has brought to the ecosystem

  • User analysis

    • An important aspect of the program analysis is understanding the type of participants it attracts

    • Need to analyze their onchain behaviour to see which incentive method achieve huge success

    • Tracking how claimants deal with the rewards is also a way to evaluate the success of the program

    • Need to conduct cohort analysis to identify the quality of users

Project Timeline

  • Upon a successful vote, we will immediately commence work

  • Note, the project will last for 5 months (Oct.10th,2023 - April.1st, 2024)

  • We will set up the pipeline ASAP and iterate based on feedback from the community and stakeholders

  • The project would follow the following timeline:

Project Timeline
Project Timeline2089×1191 305 KB

output4
output41708×676 65 KB

  1. Project Design

    • Duration: 3 days

    • Timeline: Oct.30, 2023 - Nov.1, 2023

  2. Preparations

    • Duration: 22 days

    • Timeline: Oct.30, 2023 - Nov.20, 2023

  3. Realtime Fraud Detection / Misuse of Funds

    • Duration: 5 days

    • Timeline: Nov.20, 2023 - Nov.24, 2023

  4. Grant/Protocol Data and Impact Analysis

    • Duration: 30 days

    • Timeline: Nov.24, 2023 - Dec.24, 2023

  5. Data monitoring website development

    • Duration: 26 days

    • Timeline: Dec.24, 2023 - Jan.19, 2024

  6. Bi-weekly report

    • Duration: 42 days

    • Timeline: Nov.24, 2023 - Feb.2, 2024

  7. Summary report

    • Duration: 28 days

    • Timeline: March 1, 2024 - April 1, 2024

Milestone/Tranche amounts

  • Total funding requested

    • 230,000 USD

      • Note, our projection is estimated to need 215,000 USD

      • An admin fee and accounting fee altogether $15,000 or 6% of total grant will be needed to support the 6-month project. We will track this expense line item and can give it back based on actual vs expected

    • Paid in $ARB at 30d VWAP

    • 20% paid upfront each milestone and the rest 80% paid when milestone achieved

  • Milestones

    • Milestone 1

      • Set up the data monitoring and reporting pipeline

      • Accomplish Priority 1: Realtime Fraud Detection / Misuse of Funds

    • Milestone 2

      • Set up the analysis framework

      • Develop algorithms

      • Accomplish Priority 2: Grant/Protocol Data and Impact Analysis

    • Milestone 3

      • Develop a data monitoring website

    • Milestone 4

      • Finalise the final report

  • For more details please check:

cost
cost2322×1200 427 KB

output 6
output 62655×1585 322 KB

Prior experience and Proposed Team

Prior experience

  1. Onchain data analysis skills

    • Sixdegree is the leading onchain data research team

    • We are ranked number 1 on Dune Analytics and have worked with 25+ protocols on their data strategy, including Lens, ParaSwap, Gnosis Safe among others

    • Received grants from Dune Analytics, Lens Protocol as well as the Ethereum foundation for making onchain data tutorials, building customized onchain data analytic platform and conducting onchain data research, respectively

    • We run a community with 1000+ data analysts and at least 50+ top Dune wizards

  2. User analysis / growth analysis

    • We focus on user growth leveraging onchain data to drive retention / cac / cohort analysis and give actionable recommendations to increase onchain adoption

    • We helped Gnosis Safe identify airdrop farmers by analyzing onchain user behaviours

    • We have been working with Rabbithole to boostrap the growth of protocols

    • We helped ParaSwap understand the user type and developed a targeted incentive program

  3. Data engineering

    • We have built customized data analytic platform Lentics on Lens (stop running now)

    • Lentics won 1st prize in Lens Hackthon held by Hash Global

    • We have rich experience in building monitoring and alerting bots

Recent Research

Proposed Team

  • The entire team of Sixdegree will support this project as a core team

  • Sixdegree will be responsible for developing algorithms, building pipelines and auditing work

    • 0xetwas, co-founder, Bd & research

    • Louis, co-founder, Lead researcher

    • Spring, co-founder, Lead data wizards

    • Julyan, co-founder, Developer

    • 0xbi, co-founder, Senior Data wizards

  • We will have 5 more vetted top wizards to commit to the project, depending on the final scope of the project

    • We have a community of 1,000+ wizards that we have built over the past 12+ months thanks to working on our book called mastering onchain data that received a grant from Ethereum Foundation and Dune Analytics

  • The data analysis work will undergo cross-checks to ensure accuracy

  • Everyone on the team will dedicate up to 40 hours per week to build and then to maintain the dashboards and develop analysis methods

Team Links

Information/resources required

Wallet addresses of grantees and incentive related addresses (pool addresses, etc.)

  • Funding Address: [Enter the specific address where funds will be sent for grant recipients]
    • arb1:0x6C35179E09aD5EC7bCe17f62AD811F3a62b769fC
  • Funding Address Characteristics: [Enter details on the status of the address, eligible address must be a 2/3 multisig with private keys securely stored]
    • Multisig (SAFE)

Contact us at

By emails at
contact@sixdegree.xyz

cindy@sixdegree.xyz

By telegram
oxgummy

6 Likes
2

+1 to this

It’s essential that the ArbitrumDAO invests in accountability tools to make sure that grant recipients deliver on their promises. Historically in other DAOs, there have been problems with grant recipients not being able to deliver on their promises which go unchecked by the community. SixDegree has the team & tools available to make sure that teams are properly tracking their grant and delivering on results so the community can iterate on the grant process.

5 Likes
3

Thanks for this well thought out proposal. To further understand the offer, can you answer/update with the following:

  1. Will dashboard include tracking of ARB spending by project vs their budget?
  2. Can grant matching by project be included?
  3. Does the team have any experience in fraud/misuse monitoring?
  4. Will SP be responsible for chasing up projects for outstanding data/reports?
  5. Milestone amounts are in gsheet, please add to proposal for clarity.
  6. Confirm if there is ongoing maintenance fee for dashboards or 80% is paid upon completion to satisfaction of STIP committee then maintained until end of program.
  7. Confirm if 80% of weekly reports is paid upon delivery of final report or prorata per report.
  8. You have shared a number of good examples of previous work but none seem to be of the scale that this RFP asks for. Can you describe the teams capacity for this quantity of work and share any examples I may have missed that back up the Sixdegree team’s experience to deliver on this?
2 Likes
4

Thank you very much for your message

Please find below some our feedback to your questions.

We remain at your disposal if you have any questions @DisruptionJoe @tnorm @Burns & @epowell101

  1. Will dashboard include tracking of ARB spending by project vs their budget?

    • Yes, we confirm we will include ARB spending by project vs their budget

  2. Can grant matching by project be included?

    • Yes, it is listed in our task details

  3. Does the team have any experience in fraud/misuse monitoring?

    • Yes, our team has expertise in fraud and misuse monitoring

    • We have built a scoring system to help Gnosis Safe identify airdrop farmers. Please find information about our case study: User engagement analysis - by Sixdegree and Louis Wang

    • One of our team members has worked in the Gitcoin Fraud Detection & Defense (FDD) group, gaining valuable experience in detecting various fraudulent behaviors within Gitcoin Grant Rounds. This includes identifying airdrop farming, sybil attacks, donation recycling/mid-round grant forwarding, and collusion between grant owners and contributors.

    • We are confident that this experience can be effectively applied to the Arbitrum ecosystem.

  4. Will SP be responsible for chasing up projects for outstanding data/reports?

    • Yes, we will be responsible for chasing projects for outstanding data/reports

  5. Milestone amounts are in gsheet, please add to proposal for clarity.

  • Milestones

    • Milestone 1 （Oct.30, 2023 ~ Nov.24, 2023）

      • Set up the data monitoring and reporting pipeline

      • Accomplish Priority 1: Realtime Fraud Detection / Misuse of Funds

    • Milestone 2（Nov.24, 2023 ~ Dec.24, 2023）

      • Set up the analysis framework

      • Develop methods and algorithms

      • Accomplish Priority 2: Grant/Protocol Data and Impact Analysis

    • Milestone 3（Dec.24, 2023 ~ Jan.19, 2024）

      • Develop a data monitoring website

    • Milestone 4（Jan.19, 2024 ~ April 1, 2024）

      • Finalise the final report

  • Please match the milestone with subclass tasks.

1789×1258 310 KB

  1. Confirm if there is ongoing maintenance fee for dashboards or 80% is paid upon completion to satisfaction of STIP committee then maintained until end of program.

    • We confirm no ongoing maintenance fee for dashboards.

    • Milestones will be used for payment

  2. Confirm if 80% of weekly reports is paid upon delivery of final report or prorata per report.

    • Yes, we confirm, as discussed in the proposal, we will be paid in four tranches

  3. You have shared a number of good examples of previous work but none seem to be of the scale that this RFP asks for. Can you describe the teams capacity for this quantity of work and share any examples I may have missed that back up the Sixdegree team’s experience to deliver on this?

  4. Scale:

  • We have built 80+ dashboards over the past 12 months, all public on Dune Analytics

1840×1798 292 KB

All the work is done by the endeavour of our 1 FT within 4 weeks.
2. Team:

  1. Sixdegree will be responsible for developing algorithms and building pipelines

  2. Team achievements

1462×1224 193 KB

1567×928 121 KB

2874×1630 456 KB

1330×1024 140 KB

1280×926 229 KB

894×1344 307 KB

  1. Team members that are fully committed to this project:

    1) Kylin, Data Scientist

     - Key achievements:

    - PhD in Statistics

    - Research fellow at Sydney University

    - Ex- Gitcoin Fraud Detection & Defense

    2) Louis, Lead Researcher

     - Key achievements:

    - PhD in Laser Physics

    - Ex- DeFi researcher at Nethermind / smart contract developer at Connexion

    3) Spring, Lead Data Wizards

     - Key achievements:

    - 15+ years experience in SQL

    - Dune Top 3 wizard https://dune.com/springzhang

    - Built 100+ public Dune Dashboards

    4) Julyan, Full-stack Developer

     - Key achievements:

    - Ex- Senior AI Engineer at SenseTime ($50bn+ listed company)

    - Expertise in machine learning and modeling

    - Experience in AI x Crypto

    5) 0xbi, Senior Data Wizards

     - Key achievements:

    - Ex- Senior Data Analyst at DiDi / Filecoin/ Huobi

    - Rich experience in data monitoring bot building

    - Led multiple research including new breakthrough on airdrop analysis and retention for early airdrop holders

    6) 0xetwas, Bd & Research

     - Key achievements:

    - 10+ years in finance and quant

    7) Analysts

     - We have a number of analysts we work with and we can reach out to support us
5

Thank you! One remaining item is adding in $ or ARB amounts to be paid on each milestone and expected date for each. I see these in the gsheet but please edit your proposal and add these in or clearly state it.

This is what I have:
Total: $230k USD paid in ARB at 30d VWAP ($215k + $15k allowance)
Upfront - $13,690.91 USD
MS1 - $64,581.82 USD
MS2 - $42,490.91 USD
MS3 - $29,072.73 USD
MS4 - $64,800.00 USD
Allowance - $15k USD (confirm when it is to be paid)

1 Like
6

Hi @Burns - thank you very much four message.

Please find below our answers.

We are looking forward to hear back from the community and from @DisruptionJoe @tnorm @Burns & @epowell101 who drafted this RFP.

Tranches with milestones:

  • Tranche 1

    • 20% upfront for MS1

    • Amount: $13,690.91

    • Estimated date: Oct. 30, 2023

  • Milestone 1 （Oct.30, 2023 ~ Nov.24, 2023)

    • Set up the data monitoring and reporting pipeline

    • Accomplish Priority 1: Realtime Fraud Detection / Misuse of Funds

  • Tranche 2

    • 80% of MS1 + 20% upfront for MS2

    • Amount: $64,581.82

    • Estimated date: Nov.24, 2023

  • Milestone 2（Nov.24, 2023 ~ Dec.24, 2023)

    • Set up the analysis framework

    • Develop methods and algorithms

    • Accomplish Priority 2: Grant/Protocol Data and Impact Analysis

  • Tranche 3

    • 80% of MS2 + 20% upfront for MS3

    • Amount: $42,490.91

    • Estimated date: Dec.24, 2023

  • Milestone 3（Dec.24, 2023 ~ Jan.19, 2024)

    • Develop a data monitoring website

  • Tranche 4

    • 80% of MS3 + 20% upfront for MS4

    • Amount: $29,072.73

    • Estimated date: Jan.19, 2024

  • Milestone 4（Jan.19, 2024 ~ April 1, 2024)

    • Finalise the final report

  • Tranche 5

    • 80% of MS4 + allowance

    • Amount: $79,800

    • Estimated date: April 1, 2024

7

Hi @DisruptionJoe

After discussing internally, we are able to reduce our proposed budget to $195,000 equivalent to a decrease of $35,000 or about a 15% decrease from the initial budget.

We reduced the admin costs as well as the salaries by c. 15%.

Please find below additional information about the budget as well as an updated milestones schedule.

We are looking forward to hear back from the community and from @DisruptionJoe @tnorm @Burns & @epowell101 who drafted this RFP.

-Sixdegree team

Milestone/Tranche amounts

  • Total funding requested

    • 195,000 USD

      • The funding amount is calculated by multiplying the salaries of the senior researcher, senior data analyst, and developer by the number of working days

      • The basic salaries for the senior researcher, senior data analyst, and developer are 13,000, 11,000, 10,000, respectively

      • The number of working days is determined by the work breakdown outlined in the timeline

      • An administrative fee and accounting fee totaling $136 will be required to support the 6-month project

    • Payment will be made in $ARB at the 30-day volume-weighted average price in five installments

    • Each milestone will be paid in advance, with 20% paid upfront, and the remaining 80% paid upon completion of the milestone (4 milestones, described below)

Tranches with milestones:

  • Tranche 1

    • 20% upfront for MS1

    • Amount: $12,654.55

    • Estimated date: Oct. 30, 2023

  • Milestone 1 （Oct.30, 2023 ~ Nov.24, 2023）

    • Set up the data monitoring and reporting pipeline

    • Accomplish Priority 1: Realtime Fraud Detection / Misuse of Funds

  • Tranche 2

    • 80% of MS1 + 20% upfront for MS2

    • Amount: $59,127.27

    • Estimated date: Nov.24, 2023

  • Milestone 2（Nov.24, 2023 ~ Dec.24, 2023）

    • Set up the analysis framework

    • Develop methods and algorithms

    • Accomplish Priority 2: Grant/Protocol Data and Impact Analysis

  • Tranche 3

    • 80% of MS2 + 20% upfront for MS3

    • Amount: $37,254.55

    • Estimated date: Dec.24, 2023

  • Milestone 3（Dec.24, 2023 ~ Jan.19, 2024）

    • Develop a data monitoring website

  • Tranche 4

    • 80% of MS3 + 20% upfront for MS4

    • Amount: $27,463.64

    • Estimated date: Jan.19, 2024

  • Milestone 4（Jan.19, 2024 ~ April 1, 2024）

    • Finalise the final report

  • Tranche 5

    • 80% of MS4 + management fee

    • Amount: $58,499.64

    • Estimated date: April 1, 2024

  • For more details please check:

DLDLDLDL
DLDLDLDL1920×891 150 KB

1 Like
8

Hi @DisruptionJoe

We wanted to give more color about the different KPIs we will track for various protocols.

Please find below additional information.

We are at your disposal if you have any questions and are looking forward to hear back from the community and from @DisruptionJoe @tnorm @Burns & @epowell101 who drafted this RFP.

-Sixdegree team

0. Background

  • We will implement different KPIs for various protocols based on their respective categories.

  • Generally, we will monitor KPIs that reflect user engagement (active users, transactions), economic impact (tvl, volume), growth (new users), retention rate, and churn rate, among others.

  • Moreover, we aim to gain a comprehensive understanding of how the incentive program influences the Arbitrum ecosystem. For instance, we seek to assess metrics such as Customer Acquisition Cost (CAC), Lifetime Value (LTV), and Return on Investment (ROI) for each ARB token.

  • Additionally, we will analyze the user profiles of individuals who participate in the incentive program.

Please find further details outlined below.

1. Impact Analysis

  1. Build a Dune dashboard to track the key metrics such as TVL, number of transactions, gas consumption, user growth, volume, retention rate, etc…

    • From the view of Arbitrum network at a high level

    • Group by sector

    • Group by procotol

  2. The impact analysis will compare the status before, during and after the incentive period

2. Behaviour Analysis

  1. Design a scoring system for grantee protocols

    • Criteria include user size, growth, economic value, activities, user stickness, retention rate, etc…

    • Criteria will be adjusted depending on the specific sector the grantee project belongs to

  2. Design a scoring system for users of grantee protocols

    • Criteria include maturity, active level, gas consumption, allocated assets, governance, etc…

    • Criteria will be adjusted depending on the specific sector the grantee project belongs to

  3. This way we can have a benchmark for better understanding of the efficacy of STIP

  4. Build a stand-alone website to showcase the results for better understanding and user experience

  5. The data source for the website will be retrieved via the Graph, enriching the diversity of the data source and enhancing transparency

3. Capital Utilization Efficiency Analysis

  1. Calculation of ROI in terms of Arbitrum as a whole and breaks down into sector and grantee porotocols

  • Average cost of

    • Acquiring new users

    • Activating existing users

    • Adding TVL per unit

    • Long-term retained users (post STIP)

    • Long-term retained TVL (post STIP)

  1. Comparison among all protocols / incentive methods

    • Compare all the costs mentioned above

    • Investigate the best incentive method in each sector

4. Claimant Analysis

  1. Build algorithms and to understand the user behaviour of claimants

  2. Build the Dune dashboard and conduct a report to elaborate on

  • How claimants deal with the rewards

    • Hold

    • Transfer

    • Sell

    • Delegate

    • Cohort analysis

  1. Build the Dune dashboard and conduct a report to elaborate on

    • Overlap of claimants among STIP protocols

    • Percentage of claimants holding $ARB

    • Incentive farmers

    • Cohort by volume

    • Cohort by balance

    • Cohort by activities