Skip to content

Add SMP eScience Center indicator configurations (low/medium/high) #105

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
vuillaut wants to merge 1 commit into
base: main
Choose a base branch
from
Open

Add SMP eScience Center indicator configurations (low/medium/high) #105

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
66 changes: 66 additions & 0 deletions configurations/practical-smp-escience-center/readme.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Software Management Plan indicator profiles — v3 (reconciled)

Source: [Practical guide to Software Management Plans](https://zenodo.org/records/7589725) (Netherlands eScience Center / NWO).

The Practical guide to SMP provides a list of core requirements for research software, divided into 3 categories (low, medium, high).

Here, we map the EVERSE indicators to these core requirements and provide the corresponding configurations.

Last update: 2026-07-17.
Author: Thomas Vuillaume

## 1. Management levels (corrected)

This is Table 4 of the guide, corrected for the three missing practices (User documentation, Software licensing and compatibility, and Deployment documentation) at low level (see Table 1 of the same document).


| Core requirement (Section 5.1) | Low (6.1.1) | Medium (6.1.2) | High (6.1.3) |
|--------------------------------------|:-----------:|:--------------:|:------------:|
| Purpose | X | X | X |
| Version control | X | X | X |
| Repository | | X | X |
| User documentation | X | X | X |
| Software licensing and compatibility | X | X | X |
| Deployment documentation | X | X | X |
| Citation | | X | X |
| Developer documentation | | X | X |
| Testing | | X | X |
| Software Engineering quality | | X | X |
| Packaging | | X | X |
| Maintenance | | X | X |
| Support | | | X |
| Risk analysis | | | X |


## 2. Indicator mapping

Two columns, deliberately not one bucket:

- **Validates** — if this indicator passes, the core requirement is met. One per row, chosen as the strongest single sufficient proxy.
- **Participates in validation** — contributes evidence but isn't independently sufficient (partial coverage, a maturity add-on, or one of several alternative routes to the same outcome).

| Core requirement | Validates without ambiguity | Tool (wired up in this repo) | Participates in validation |
|--------------------------------------|------------------------------------------|------------------------------|------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
| Purpose | `descriptive_metadata` | OEBFAIR, RSFC | `codemeta_completeness`, `metadata_is_up_to_date` |
| Version control | `version_control_use` | RSFC | `versioning_standards_use` |
| Repository | `persistent_and_unique_identifier` | OEBFAIR, RSFC | `archived_in_scholarly_repository`, `archived_in_software_heritage`, `listed_in_registry` |
| User documentation | `software_has_documentation` | OEBFAIR, RSFC | — |
| Software licensing and compatibility | `software_has_license` | HowFairIs, OEBFAIR, RSFC | `software_has_license_for_file_types` |
| Deployment documentation | `requirements_specified` | RSFC | `dependency_management` |
| Citation | `software_has_citation` | CFFConvert, OEBFAIR, RSFC | — |
| Developer documentation | `has_contribution_guidelines` | RSFC | `human_code_review_requirement`, `code_documentation_coverage_ok` |
| Testing | `software_has_tests` | RSFC | `passed_tests_ok`, `software_test_coverage`, `has_ci-tests`, `functional_correctness`, `repository_workflows` |
| Software Engineering quality | `uses_tool_for_warnings_and_mistakes` | **none** | `has_no_linting_issues`, `code_smells_ok`, `cyclomatic_complexity_ok`, `code_duplication_ok`, `maintainability_index_ok`, `internal_cohesion_ok`, `coupling_between_objects_ok`, `lines_of_code_ok`, `dependency_management` |
| Packaging | `has_published_package` | OEBFAIR, OpenSSF Scorecard | `software_is_containerized`, `has_releases` |
| Maintenance | `project_is_active` | OpenSSF Scorecard | `has_active_contributors`, `code_churn_ok` |
| Support | `support_issue_tracking` | **none** | `has_active_communication_channels`, `response_timeframe_ok` |
| Risk analysis | `static_analysis_common_vulnerabilities` | OpenSSF Scorecard | `no_critical_vulnerability`, `no_leaked_credentials`, `has_no_binary_artifacts`, `uses_fuzzing` |

All 47 catalog indicators are accounted for: 14 as unambiguous validators (one per requirement) + 33 distinct indicators as participating evidence (`dependency_management` counted once despite the double listing).


## Open items

- **Table numbering inconsistency in the sources.** v1 cites "table 6.1.4," v2 cites "Table 4." I could not fetch the PDF directly this session (network restriction on the sandbox) to confirm which numbering is correct in the published guide — worth a quick manual check before this goes external.
- **Tools-that-measure-indicator column dropped.** v1 listed tools (RSFC, SOMEF, howfairis, etc.) per indicator. I did not verify those against current tool capabilities this session, so I left the column out rather than propagate unverified claims. If you want it back, that's a separate verification pass — tool coverage changes faster than indicator definitions.
- **Unambiguous/participating calls are judgment, not derived from the catalog.** The EVERSE metadata doesn't label indicators as "sufficient" vs. "supporting" — that split is my read of each indicator's description against the guide's practice text. Worth a second pair of eyes on the borderline ones: `archived_in_scholarly_repository` (Repository) and `has_no_linting_issues` (SE quality) are the two I'd flag as closest calls, since both could arguably swap into the unambiguous column.
104 changes: 104 additions & 0 deletions configurations/practical-smp-escience-center/smp-escience-high.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
{
"indicators": [
{
"name": "descriptive_metadata",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/descriptive_metadata"
},
{
"name": "descriptive_metadata",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/descriptive_metadata"
},
{
"name": "version_control_use",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/version_control_use"
},
{
"name": "unique_identifier",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/persistent_and_unique_identifier"
},
{
"name": "persistent_and_unique_identifier",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/persistent_and_unique_identifier"
},
{
"name": "has_documentation",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/software_has_documentation"
},
{
"name": "software_has_documentation",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/software_has_documentation"
},
{
"name": "has_license",
"plugin": "HowFairIs",
"@id": "https://w3id.org/everse/i/indicators/software_has_license"
},
{
"name": "has_license",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/software_has_license"
},
{
"name": "software_has_license",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/software_has_license"
},
{
"name": "requirements_specified",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/requirements_specified"
},
{
"name": "has_citation",
"plugin": "CFFConvert",
"@id": "https://w3id.org/everse/i/indicators/software_has_citation"
},
{
"name": "has_citation",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/software_has_citation"
},
{
"name": "software_has_citation",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/software_has_citation"
},
{
"name": "has_contribution_guidelines",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/has_contribution_guidelines"
},
{
"name": "software_has_tests",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/software_has_tests"
},
{
"name": "has_package",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/has_published_package"
},
{
"name": "has_published_package",
"plugin": "OpenSSFScorecard",
"@id": "https://w3id.org/everse/i/indicators/has_published_package"
},
{
"name": "project_is_active",
"plugin": "OpenSSFScorecard",
"@id": "https://w3id.org/everse/i/indicators/project_is_active"
},
{
"name": "static_analysis_common_vulnerabilities",
"plugin": "OpenSSFScorecard",
"@id": "https://w3id.org/everse/i/indicators/static_analysis_common_vulnerabilities"
}
]
}
49 changes: 49 additions & 0 deletions configurations/practical-smp-escience-center/smp-escience-low.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
{
"indicators": [
{
"name": "descriptive_metadata",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/descriptive_metadata"
},
{
"name": "descriptive_metadata",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/descriptive_metadata"
},
{
"name": "version_control_use",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/version_control_use"
},
{
"name": "has_documentation",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/software_has_documentation"
},
{
"name": "software_has_documentation",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/software_has_documentation"
},
{
"name": "has_license",
"plugin": "HowFairIs",
"@id": "https://w3id.org/everse/i/indicators/software_has_license"
},
{
"name": "has_license",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/software_has_license"
},
{
"name": "software_has_license",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/software_has_license"
},
{
"name": "requirements_specified",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/requirements_specified"
}
]
}
99 changes: 99 additions & 0 deletions configurations/practical-smp-escience-center/smp-escience-medium.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
{
"indicators": [
{
"name": "descriptive_metadata",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/descriptive_metadata"
},
{
"name": "descriptive_metadata",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/descriptive_metadata"
},
{
"name": "version_control_use",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/version_control_use"
},
{
"name": "unique_identifier",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/persistent_and_unique_identifier"
},
{
"name": "persistent_and_unique_identifier",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/persistent_and_unique_identifier"
},
{
"name": "has_documentation",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/software_has_documentation"
},
{
"name": "software_has_documentation",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/software_has_documentation"
},
{
"name": "has_license",
"plugin": "HowFairIs",
"@id": "https://w3id.org/everse/i/indicators/software_has_license"
},
{
"name": "has_license",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/software_has_license"
},
{
"name": "software_has_license",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/software_has_license"
},
{
"name": "requirements_specified",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/requirements_specified"
},
{
"name": "has_citation",
"plugin": "CFFConvert",
"@id": "https://w3id.org/everse/i/indicators/software_has_citation"
},
{
"name": "has_citation",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/software_has_citation"
},
{
"name": "software_has_citation",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/software_has_citation"
},
{
"name": "has_contribution_guidelines",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/has_contribution_guidelines"
},
{
"name": "software_has_tests",
"plugin": "RSFC",
"@id": "https://w3id.org/everse/i/indicators/software_has_tests"
},
{
"name": "has_package",
"plugin": "OEBFAIR",
"@id": "https://w3id.org/everse/i/indicators/has_published_package"
},
{
"name": "has_published_package",
"plugin": "OpenSSFScorecard",
"@id": "https://w3id.org/everse/i/indicators/has_published_package"
},
{
"name": "project_is_active",
"plugin": "OpenSSFScorecard",
"@id": "https://w3id.org/everse/i/indicators/project_is_active"
}
]
}
Loading