Skip to content

also bwrap module installpath - #5214

Open
smoors wants to merge 16 commits into
easybuilders:developfrom
smoors:bwrap-modules
Open

also bwrap module installpath#5214
smoors wants to merge 16 commits into
easybuilders:developfrom
smoors:bwrap-modules

Conversation

@smoors

@smoors smoors commented Jun 30, 2026

Copy link
Copy Markdown
Contributor

this PR also bwraps the module installpath (and removes --installpath-modules from bwrap_eb_options as it's no longer needed).

this change is required for making bwrap work with HMNS: for a compiler module the HMNS has to know the non-bwrapped location, instead of what is passed to --installpath-modules

the main issue blocking this change was that bwrapping a module installpath causes the already installed modules to become unavailable. this is solved by copying the full module installpath to the bwrap module installpath (with overlayfs this is not necessary). the overhead of this should be fairly limited - on our systems the size is < 25MB and <2000 files.

other changes:

  • update the bwrap github action to print success/failure for each individual check.
  • avoid creating the modulepath directory when in --bwrap mode (before actually running with bwrap) in case the user does not have write permissions.
  • avoid adding duplicate bwrap binds (with e.g. easystacks).

@smoors smoors added the bug fix label Jun 30, 2026
@smoors smoors added this to the next release (5.3.2?) milestone Jun 30, 2026
@smoors
smoors marked this pull request as draft June 30, 2026 08:53
@smoors
smoors marked this pull request as ready for review June 30, 2026 18:31
@smoors
smoors requested a review from bartoldeman June 30, 2026 19:01
@smoors
smoors marked this pull request as draft July 1, 2026 21:32
@smoors smoors changed the title also bwrap module directories also bwrap module installpath Jul 3, 2026
@smoors
smoors marked this pull request as ready for review July 3, 2026 18:41
@smoors
smoors marked this pull request as draft July 3, 2026 19:12
@smoors
smoors marked this pull request as ready for review July 5, 2026 07:13
Samuel Moors added 2 commits July 27, 2026 17:42
@boegel

boegel commented Aug 6, 2026

Copy link
Copy Markdown
Member

I would really like to see a review from @bartoldeman before merging this...

Comment thread easybuild/tools/bwrap.py
Comment on lines 145 to 146
# bind mount all software directories
for mod in sorted(get_bwrap_info('modules_to_install')):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part is what breaks bwrap for HMNS:
It queries the modules and uses them as software directories. Hence we get a /tmp/ebinstall/software/MPI/GCC/12.2.0/OpenMPI/4.1.4/h5py
The modules_to_install field originates in dry_run that yields "full_mod_name" for each missing module.

Not fully sure how much this is related: it installs the module file into that software folder while the software is at the correct location:
/tmp/ebinstall/software/h5py/3.8.0-foss-2022b/lib/python3.10/site-packages/h5py

There is also a module file at /tmp/testbwrap/modules/all/MPI/GCC/12.2.0/OpenMPI/4.1.4/h5py, so correct there.

So somehow the software does end up in the right location but the module file does not.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants