Hi @SEEDGov team!
Thank you again for all the operational effort and depth of analysis you bring to each monthly report. Compiling voting metrics, forum engagement, feedback scoring, and bonus allocations is a huge undertaking, and we truly appreciate how you keep the program running and the process transparent. However, we also recognize that any scoring methodology can introduce unintended incentives or edge-case behaviors, and your current Delegate Feedback average may sometimes discourage moderately-confident contributions. With that in mind, we’d like to offer a few ideas for adjustments that preserve quality incentives while still encouraging thoughtful participation.
Current challenge
- DF is the simple average of all scored comments. Even a few lower-scoring remarks pull down a delegate’s overall DF score, which can discourage sharing “good-but-not-perfect” comments/feedback/ideas.
- Fear of lowering one’s average may lead delegates to speak up only when absolutely sure, reducing the total volume and diversity of feedback.
This is also something @cp0x pointed out.
Design goals
After recognizing the challenges above, we believe any refinement to the DF calculation should be guided by clear objectives that both protect the integrity of high-impact insights and nurture a broad, confident level of participation.
- Ensure only a delegate’s strongest contributions drive their DF score, incentivizing high-impact insights.
- Allow delegates to share well-reasoned ideas, even if they aren’t guaranteed “perfect”, without fear of penalization.
- Foster a healthy volume and variety of perspectives by not unduly punishing occasional lower-scoring comments.
Comparison of score calculation methods
These are the potential methods for calculating the DF score, which fit the design goals provided.
| Method | Calculation | Pros | Cons |
|---|---|---|---|
| Current Average | Sum(scores) ÷ N |
Extremely simple; every comment counts; easy to audit | Low-quality comments directly lower average; discourages moderately confident posts |
| Top-k Mean | Sort scores descending, then average the best k (e.g. k = 4) |
Rewards only your top insights; other comments don’t hurt | Requires at least k scored comments; ignores above-average feedback beyond k comments |
| Upper-Quantile Mean | Sort scores descending, average the top X % of N (e.g. top 30 %) |
Captures a broader slice of above-average feedback; filters noise | Needs enough total comments to fill the quantile; adds threshold complexity |
Top-k Mean and Upper-Quantile Mean are essentially the same approach—both methods focus solely on averaging a delegate’s highest-scoring comments. The distinction between “best k” and “top X %” is secondary to the core goal: reward top-tier insights while filtering out lower-scoring noise.
In Top-k Mean, if a delegate posts 10 scored comments in a month and we set k = 4, we would simply average the highest four scores and ignore the other six. That way, everyone is encouraged to produce their best insights without worrying that a less polished comment will drag them down.
We hope this helps, and are happy to collaborate on a pilot in a future cycle!