Skip to content

FPM-1338 shared processing configs - #353

Merged
richjam merged 4 commits into
stagingfrom
bug/FPM-1338-shared-processing-configs
Aug 24, 2026
Merged

FPM-1338 shared processing configs#353
richjam merged 4 commits into
stagingfrom
bug/FPM-1338-shared-processing-configs

Conversation

@richjam

@richjam richjam commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

https://jira.ceh.ac.uk/browse/FPM-1338

Overview

A data processing configuration can apply to more than one dataset, e.g. fdri-se-fofad-01-pr-battv_1min-load_raw is a step in both the battv_15min and battv_30min plans, so the metadata API returns it with two appliesToDataset values.

Previously, we assumed a 1-to-1 relationship so did applies_to_dataset[0], so the config was set against whichever dataset the API happened to list first and dropped from the others. One of the datasets would fail with KeyError: 'http://fdri.ceh.ac.uk/id/data-processing-configuration/fdri-se-fofad-01-pr-battv_1min-load_raw'

Main changes

  • map_processing_config_item now takes the specific AppliesToDataset entry it is mapping for, and reads ts_id
    and site_id from that entry rather than from applies_to_dataset[0].
  • _build_processing_configs now takes the dataset IDs the configs were requested for. Each config is mapped once per
    requested dataset it applies to, so which container a config lands on no longer depends on the order the API lists
    those datasets in

@richjam richjam changed the title Bug/fpm 1338 shared processing configs FPM-1338 shared processing configs Aug 24, 2026
@richjam
richjam merged commit 3512a7c into staging Aug 24, 2026
8 checks passed
@richjam
richjam deleted the bug/FPM-1338-shared-processing-configs branch August 24, 2026 13:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants