-
Notifications
You must be signed in to change notification settings - Fork 603
Spec: Introduce Community Health Score panel #4367
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 15 commits
ed1e417
aed15b5
d7b4611
3415ffc
c2e4648
3bc62b6
40c7f17
c47bfc8
f03d991
48b75e5
9298f1f
8ccb308
f7e7622
70e7d27
aa8754c
d53c2c5
a7439fb
3223c60
bc6e7e8
0606367
a8abf6e
d72d4e5
5ccdbfc
14e869c
1178461
0c56a18
8cba2de
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| - Feature Name: Community health score (CHS) panel | ||
| - Created: 2026-08-04 | ||
| - Associated PR: [#4189](https://github.com/mozilla/pontoon/pull/4189) | ||
|
|
||
| # Summary | ||
|
|
||
| Introduce an admin-only Community health score panel within the global Insights dashboard that provides a monthly breakdown of the Community health score and its underlying metrics, along with historical score tracking. Also make the historical tracking available on locale Insights dashboards. | ||
|
|
||
| # Motivation | ||
|
|
||
| Currently, the Community health score is calculated via a series of moz-l10n scripts and Google Sheets formulas, collected manually on a monthly basis. Other than the tedious nature of needing to do this manually, PMs need better options to analyze CHS data directly from Pontoon. | ||
|
|
||
| # Feature explanation | ||
|
|
||
| ## Community health score aggregation | ||
|
|
||
| On the first of each month, an automation is run that collects telemetry on the state of each locale for the **previous month**. The data collected for **each locale** is compiled into the following datapoints: | ||
| - `active_managers`: managers with **reviews performed** + **approved translations** > 500 in the past 12 months | ||
|
functionzz marked this conversation as resolved.
Outdated
|
||
| - `active_translators`: translators with **reviews performed** + **approved translations** > 400 in the past 12 months | ||
| - `active_contributors`: contributors with **approved translations** >= 200 in the past 12 months | ||
|
functionzz marked this conversation as resolved.
Outdated
|
||
| - `all_contributors`: contributors with **approved translations** + **rejected translations** + **pending suggestions** >= 200 in the past 12 months | ||
| - `new_signups`: new signups with **approved translations** >= 100 in the past 12 months | ||
| - `key_projects_enabled`: number of projects that are classified as key projects via the `is_chs_project` flag | ||
| - `completion`: locale completion, as a percentage of (**approved_strings** + **strings_with_warnings**) out of **total strings** | ||
|
|
||
| The data used is then computed into the following scores based on predetermined thresholds: | ||
|
|
||
| - `active_managers_score`: score representation of `active_managers` | ||
| - `active_translators_score` score representation of `active_translators` | ||
| - `active_contributors_score` score representation of `active_contributors` | ||
| - `all_contributors_score` score representation of `all_contributors` | ||
| - `new_signups_score` score representation of `new_signups` | ||
| - `key_projects_enabled_score` score representation of `key_projects_enabled` | ||
| - `completion_score` score representation of `completion` | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. This paragraphs doesn't add much value. Instead, we should explain the formula for calculating each of these scores. Think of the spec as the containing sufficient information to develop the feature. It will also be helpul in the future when we'll need to understand how the score works, so we don't need to decypher the code. |
||
|
|
||
| Finally the scores are combined into a **Community health score** for that locale on that month which is used for measuring locale health. | ||
|
Collaborator
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. How excatly (mathematically) are they combined? |
||
|
|
||
| ## Community health score panel | ||
|
|
||
| The new **Community health score** panel located at `/insights` consists of a widget, a table of locales and a graph. | ||
|
|
||
| The widget at the top of the panel contains **locale search** and buttons titled `Edit Locales` and `Show scores`. `Edit Locales` swaps the panel contents for a selector that enables admins to change locale availability on the CHS panel. `Show scores` toggles default and score view of components that make up the CHS. | ||
|
|
||
| The table of locales represents the locales the admin wishes to observe based on the most recent month's CHS data. Each table row contains the following information: | ||
| - **Locale code** linking to individual locale pages | ||
| - Number of **active managers** | ||
| - Number of **active translators** | ||
| - Number of **active contributors** | ||
| - Number of **all contributors** | ||
| - Number of **new signups** | ||
| - Number of **enabled key projects** (%) | ||
|
functionzz marked this conversation as resolved.
Outdated
|
||
| - Overall locale **completion** (%) | ||
| - CHS | ||
|
|
||
| Beside each statistic, an up/down arrow combined with a number represents the delta over the previous month's statistic. Data that has no change is not represented with a delta. For counts (contributor, translator, manager counts etc.), each value is represented as an integer. For the scores and percentages, each value is represented with 2 decimal places. | ||
|
functionzz marked this conversation as resolved.
Outdated
|
||
|
|
||
| The graph consists of a tooltip, legend and display. The tooltip describes basic functionality. The legend contains locales and `Average (all locales)`, which can be selected/deselected to display corresponding CHS datapoints over time. The display shows CHS scores for selected locales in the past 12 months. | ||
|
functionzz marked this conversation as resolved.
Outdated
|
||
|
|
||
| ## Locale Insights changes | ||
|
|
||
| The [locale insights](https://github.com/mozilla/pontoon/blob/main/specs/0108-community-health-dashboard.md) page currently includes statistics and useful information regarding locale health and performance. | ||
|
|
||
|
|
||
| The main change introduces a new graph called `Community health score` which contains a tooltip that describes basic functionality, which is to show the CHS of the current locale at a particular point in time. It displays CHS scores for current locale trailing 12 months. As soon as the CHS calculation automation on Pontoon runs on the first of each month, the previous month's scores will be automatically available for display. | ||
|
functionzz marked this conversation as resolved.
Outdated
|
||
Uh oh!
There was an error while loading. Please reload this page.