Sharing the observations about the Questbook Performance in the last quarter based on observation and interaction with builders. Thank you @krst for the review and encouragement.
I would like to make it clear that my intention in exploring the details of this program is solely to encourage potential enhancements through open discussion. I hold no negative feelings or animosity towards the Questbook team. I hope to witness them manage this program to the utmost standard.
Following are four pain points that if addressed can lead to better clarity and long-term success of the program.
If not using Rubric was mentioned anywhere is public, please ignore this point. I am currently not aware of that and expect it as part of the proposal says so.
As mentioned in the proposals KPI & other discussions, rubrics were to be an important objective way of evaluating a project especially when approval is centralised.
These rubrics are missing for every project that got approved or rejected in the New Protocols & Ideas Domain.
Without using a rubric it is just a bureaucratic approach and not transparent.
Several projects have received their first comment from the domain allocator after a month.
Domain:- Arbitrum New Protocols and Ideas
Submitted on:- 06 Nov, 2023
First Public Comment:- 06 Dec, 2023
TAT:- 30 days
Final Status:- Rejected
Let’s increase Arbitrum’s exposure to the LATAM dev community
Submitted on:- 03 Nov, 2023
First public comment:- 25 Nov, 2023
TAT:- 22 days
Final Status:- Approved
In the Arbitrum Education, Community Growth and Events domain the TAT was always 24-48 hours for the first public comment. In the Gaming Domain, the few proposals that I checked had a TAT of 1-5 days. No complaints about these domains.
In fact, how can this be achieved in all domains? Most Allocators have mentioned in their nomination form to be able to keep TAT by 48 hours but did not implement it. Even a TAT of 5-7 days should be fine.
Rationale:
First public comment can give a lot of confidence to new applicants that the process is agile. And most importantly, it was said to be done considering that the benefits are far better understood by QuestBook than anybody else.
The current total paid out is ~$150k.
Total Admin cost for the quarter is ~$90k.
Paid out ratio to admin cost looks very expensive. I understand these will be paid out in future but if the period of DDA’s is over how will the follow-up on milestones be ongoing?
The above metric if provided will allow a better understanding of what needs to be addressed. For example, reaching out to the accepted teams to enable them to complete their milestone on time. Currently, there are no initiatives on those lines.
For most projects, the communication happens off the Questbook, which defeats the purpose of using a platform. In many proposals the Accepted reason is mentioned as the conversation happened over the call in the last few weeks/months.
This can be improved further by keeping most questions public(even sharing a recording of the conversation over call) or hosting an open office hour which can be attended by any community member to increase transparency.
Personal Opinion
With the performance in the last quarter in each domain, I agree with the suggestion earlier made of having at least two DDA or two DDA’s for two Domains and the rubric of both should be above a set benchmark for approval.
The counter provided earlier for not having a multi-domain allocator was not seen in action.
Having presented my observations, I strongly believe addressing these issues and accountability the QuestBook program can remain a vital part of ArbitrumDAO in the long run.