Source to access data from Zenodo based on record ID or DOI

[chpolste]

Description

• The Zenodo source translates record IDs and DOIs from the Zenodo namespace into URLs for individual files, then delegates to a MultiUrl source for accessing the files.
• The source offers file name matching based on Python's fnmatch, so users can select, e.g., all netcdf files of a record conveniently. Files can also be selected explicitly by providing a list of files.
• Adds requests as an explicit dependency. As far as I can tell, it was already pulled in by the multiurl dependency previously.

docs preview

Usage

# Access with record id, select all files by default
ekd.from_source("zenodo", 10458186)

# Access with DOI, select only files with grib extension
ekd.from_source("zenodo", "10.5281/zenodo.10458186", only="*.grib")

# Concept DOI is resolved to latest version, DOI accepted as URL, select specific files
ekd.from_source("zenodo", "https://doi.org/10.5281/zenodo.10458185", only=["FWI_T0.0_P0.8_EU_1981_2010.grib", ...])

Tests

The current test suite only checks that the Zenodo source produces correct URLs. It mocks the requests module and produces responses as expected from the Zenodo API to avoid spamming API requests to the actual Zenodo API.

Questions

• Is only a good name for the file selection argument?
• Should the test suite have a test that verifies against the actual Zenodo API? If so, I could create a dedicated Zenodo test data record with a few example files.

Contributor Declaration

By opening this pull request, I affirm the following:

• All authors agree to the Contributor License Agreement.
• The code follows the project's coding standards.
• I have performed self-review and added comments where needed.
• I have added or updated tests to verify that my changes are effective and functional.
• I have run all existing tests and confirmed they pass.

[codecov-commenter]

Codecov Report

❌ Patch coverage is 96.07843% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.35%. Comparing base (cfc1e5f) to head (035c8c6).

Files with missing lines	Patch %	Lines
tests/sources/test_zenodo.py	96.07%	3 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #933      +/-   ##
===========================================
+ Coverage    82.27%   82.35%   +0.08%     
===========================================
  Files          239      240       +1     
  Lines        17018    17120     +102     
  Branches       823      829       +6     
===========================================
+ Hits         14001    14099      +98     
- Misses        2770     2773       +3     
- Partials       247      248       +1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[Copilot]

Pull request overview

Adds a Zenodo data source that resolves record IDs and DOIs into downloadable file URLs.

Changes:

• Adds Zenodo identifier parsing, API access, and file filtering.
• Adds offline tests for supported identifiers and API failures.
• Documents the source and declares requests as a dependency.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.

File	Description
src/earthkit/data/sources/zenodo.py	Implements the Zenodo source.
tests/sources/test_zenodo.py	Tests parsing, selection, and failures.
docs/source/concepts/inputs/from_source.rst	Documents Zenodo usage.
pyproject.toml	Adds the requests dependency.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[sandorkertesz]

@chpolste, thanks for adding this new source!
As for your question:
Should the test suite have a test that verifies against the actual Zenodo API? If so, I could create a dedicated Zenodo test data record with a few example files.

Yes, it is important to have a real test like that. It should be protected by

@pytest.mark.long_test
@pytest.mark.download

Tests marked like that are skipped in the CI and are only active when pytest is run with the -E long option. See: https://earthkit-data.readthedocs.io/en/latest/development/tests.html

[sandorkertesz]

Maybe filter is a better name thanonly for this kwargs. The "file" source already has a filter kwarg with a similar meaning.