Skip to content

Upgrade miniwdl to 1.15.0 and refresh WDL unit test skip lists (#5377) - #5567

Merged
adamnovak merged 8 commits into
masterfrom
issues/5377-wdl12-allow-nested-inputs
Oct 8, 2026
Merged

adamnovak merged 8 commits into
masterfrom
issues/5377-wdl12-allow-nested-inputs

Conversation

@annagiroti

@annagiroti annagiroti commented Aug 18, 2026 •

Copy link
Copy Markdown
Collaborator

miniwdl 1.15.0 adds parsing support for WDL 1.2 hints sections, unblocking several previously-skipped conformance unit tests and letting most nested-input tests run. Removes 30 entries across the WDL 1.1 and 1.2 unsupported-test lists that now pass, and updates the remaining entries' comments to reflect their current (rather than pre-upgrade) failure reasons.

miniwdl 1.15.0 also changed CallCache.put()'s signature, adding two new required keyword arguments, inputs and add_paths. Threads the actual input bindings through to all three call sites (task cache, workflow cache, and the write_* file cache) so cache entries record real inputs; add_paths is passed empty since Toil doesn't track that concept yet.

Addresses #5377

Changelog Entry

To be copied to the draft changelog by merger:

  • PR submitter writes their recommendation for a changelog entry here

Reviewer Checklist

  • Make sure it is coming from issues/XXXX-fix-the-thing in the Toil repo, or from an external repo.
    • If it is coming from an external repo, make sure to pull it in for CI with:
      contrib/admin/test-pr otheruser theirbranchname issues/XXXX-fix-the-thing
      
    • If there is no associated issue, create one.
  • Read through the code changes. Make sure that it doesn't have:
    • Addition of trailing whitespace.
    • New variable or member names in camelCase that want to be in snake_case.
    • New functions without type hints.
    • New functions or classes without informative docstrings.
    • Changes to semantics not reflected in the relevant docstrings.
    • New or changed command line options for Toil workflows that are not reflected in docs/running/{cliOptions,cwl,wdl}.rst
    • New features without tests.
  • Comment on the lines of code where problems exist with a review comment. You can shift-click the line numbers in the diff to select multiple lines.
  • Finish the review with an overall description of your opinion.

Merger Checklist

  • Make sure the PR passed tests, including the Gitlab tests, for the most recent commit in its branch.
  • Make sure the PR has been reviewed. If not, review it. If it has been reviewed and any requested changes seem to have been addressed, proceed.
  • Merge with the Github "Squash and merge" feature.
    • If there are multiple authors' commits, add Co-authored-by to give credit to all contributing authors.
  • Copy its recommended changelog entry to the Draft Changelog.
  • Append the issue number in parentheses to the changelog entry.

miniwdl 1.15.0 adds parsing support for WDL 1.2 `hints` sections,
unblocking several previously-skipped conformance unit tests and
letting most nested-input tests run. Removes 30 entries across the
WDL 1.1 and 1.2 unsupported-test lists that now pass, and updates the
remaining entries' comments to reflect their current (rather than
pre-upgrade) failure reasons.
@annagiroti
annagiroti requested a review from adamnovak August 18, 2026 07:10

@adamnovak adamnovak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks OK, but I think for the return codes ones we might be removing information that's still maybe accurate.

I think the CI failed due to the code not type checking against the new MiniWDL, so you might need to fix that.

Comment thread src/toil/test/wdl/wdltoil_test.py Outdated
Comment on lines +65 to +66
"single_return_code_task", # Workflow did not return the correct return code! Got: 1. Expected: 0.
"all_return_codes_task", # Workflow did not return the correct return code! Got: 42. Expected: 0.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Has MiniWDL 1.15 learned about return_codes? I think the Got: codes are probably right for these and it's the expected codes that are wrong.

@adamnovak adamnovak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks OK, but we probably want to redesign our own caching stuff a bit if the MiniWDL caching now needs the inputs and not just the key.

Comment thread src/toil/wdl/wdltoil.py
Comment on lines 857 to +861
cache_key: str,
output_bindings: WDLBindings,
file_store: AbstractFileStore,
wdl_options: WDLContext,
input_bindings: WDLBindings,

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If we now need to pass the whole input bindings to the cache, we probably should move responsibility for computing the key inside this function.

@adamnovak

Copy link
Copy Markdown
Member

I looked a little at the failing tests.

I think basic_directory is failing because now the WDL code is seeing absolute paths, and its regex in out wdl-conformance-tests repo doesn't have a .* before the first item:

https://github.com/DataBiosphere/wdl-conformance-tests/blob/d3e05db85696f5c5c5a5f5e1785c64fb2229b097/conformance.yaml#L1233

@annagiroti I invited you to that repo so you should be able to fix it there. You can run that test in that repo with:

 python3 run.py --versions 1.2 --id basic_directory --runner toil-wdl-runner

For the spec tests, the test_map test has started failing with a bunc of complaints about:

	WDL.Error.EvalError: source-relative File/Directory path resolution requires runtime context

I'm not sure why, but you can run it with:

python3 setup_unit_tests.py --version 1.2 --repo https://github.com/adamnovak/wdl.git --branch patch-3 --force-pull
python3 run_unit.py --versions 1.2 --id test_map

(This grabs a patched version of the spec tests that has openwdl/wdl#799 so it actually contains only parse-able JSON. You could also set up the pinned version of the spec tests that Toil is using.)

@adamnovak

Copy link
Copy Markdown
Member

I'm worried about test_map not working anymore. Do we know why it worked before? Was it the absolute paths in the example that have now changed to relative paths?

Is there a way to get it to work again? Where are the stdlib instances that really want to communicate?

@adamnovak adamnovak linked an issue Sep 1, 2026 that may be closed by this pull request
@adamnovak adamnovak linked an issue Sep 1, 2026 that may be closed by this pull request
Comment thread src/toil/test/wdl/wdltoil_test.py Outdated
Comment on lines +38 to +40
WDL_CONFORMANCE_TEST_COMMIT = "12d6d8a54a11803fb529aeca18ee01cba01f1d3e"
# TODO: Revert to a pinned commit once DataBiosphere/wdl-conformance-tests#PR
# (basic_directory regex fix) merges to master.
WDL_CONFORMANCE_TEST_COMMIT = "fix-basic-directory-regex"

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It didn't work to pin 7e171c30feb8380d6f2d2808ee594227856a28f0?

@adamnovak adamnovak left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is going to be updated to point at the merge commit for our WDL conformance test setup.

But I still don't want it to be adding test_map to the set of tests that don't pass. We should spend some time trying to debug it so it goes back to passing, since I think it passed before. If we can't do that, we need a good excuse written up about how we're going to tolerate a regression here for a while and we're creating a new issue about fixing it again.

@adamnovak
adamnovak merged commit 1fdd203 into master Oct 8, 2026
3 checks passed
@adamnovak
adamnovak deleted the issues/5377-wdl12-allow-nested-inputs branch October 8, 2026 15:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support WDL 1.2 allow_nested_inputs hint Expose task object to WDL tasks

2 participants