-
Notifications
You must be signed in to change notification settings - Fork 729
[bazel] Sandbox bash and git on Windows with tools/bazel.bat #9341
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
AustinSchuh
wants to merge
16
commits into
wpilibsuite:main
Choose a base branch
from
AustinSchuh:bazel-windows-bash-sandbox
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+208
−0
Open
Changes from 7 commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
0bafb24
[bazel] Sandbox bash and git on Windows with tools/bazel.bat
AustinSchuh fe8186e
[bazel] Address review feedback on the Windows wrapper
AustinSchuh de5a088
[bazel] Harden the Windows wrapper's git bootstrap
AustinSchuh 3419056
[bazel] Stop using delayed expansion in the Windows wrapper
AustinSchuh 4ee4c11
[bazel] Only shorten the profile path when it has a space
AustinSchuh 67cb172
[bazel] Let Bazel isolate repository rules instead of the wrapper
AustinSchuh 1241b54
[bazel] Scope the git config isolation to repository rules
AustinSchuh 4dff70d
[bazel] Pass the classifier filter through strict repo env
AustinSchuh 2a42b43
[bazel] Drop BAZEL_GIT and GIT_BIN_PATH, trim the comments
AustinSchuh 15adcc6
Apply suggestion from @auscompgeek
AustinSchuh d7ec673
Apply suggestion from @auscompgeek
AustinSchuh 43674b5
[bazel] Drop the git http.ssl repo_env settings
AustinSchuh 3b97eb0
Merge remote-tracking branch 'github/bazel-windows-bash-sandbox' into…
AustinSchuh 81ed09d
[bazel] Update the pinned git, run Windows CI through Bazelisk
AustinSchuh 606f927
[bazel] Explain why Windows CI has to name Bazelisk
AustinSchuh 7639182
Merge remote-tracking branch 'origin/main' into bazel-windows-bash-sa…
AustinSchuh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,168 @@ | ||
| @echo off | ||
| :: Delayed expansion stays off for the whole script, explicitly rather than by | ||
| :: inheritance, so that a "!" in a path or in a forwarded Bazel argument is never | ||
| :: eaten while a line is reparsed. Nothing below may use "!var!". | ||
| setlocal disabledelayedexpansion | ||
|
|
||
| :: Bazelisk runs this wrapper instead of Bazel itself on Windows. It bootstraps | ||
| :: a private copy of PortableGit, which supplies both git and bash/sh, and puts | ||
| :: that in front of whatever is installed on the machine. Otherwise the shell | ||
| :: and git the build ends up using are whichever ones happen to be on PATH. | ||
| :: | ||
| :: Keeping the rest of the environment out of the build is Bazel's job rather | ||
| :: than this script's. Actions are covered by --incompatible_strict_action_env, | ||
| :: and repository rules by --experimental_strict_repo_env, which limits them to | ||
| :: PATH, PATHEXT, and the variables named by --repo_env. Anything exported here | ||
| :: for a repository rule has to be listed in .bazelrc as well, or it won't | ||
| :: arrive. | ||
|
|
||
| :: A space in the profile directory breaks paths downstream, so fall back to the | ||
| :: short (8.3) form for those users. Only for those users: 8.3 names have | ||
| :: nothing to do with spaces otherwise, and there is no reason to hand everyone | ||
| :: else a mangled path. | ||
| if not "%USERPROFILE%"=="%USERPROFILE: =%" ( | ||
| for %%I in ("%USERPROFILE%") do set "USERPROFILE=%%~sI" | ||
| ) | ||
| :: 8.3 name generation can be turned off, in which case there is nothing to fall | ||
| :: back to and Bazel is going to fail in a way that is hard to connect to this. | ||
| if not "%USERPROFILE%"=="%USERPROFILE: =%" ( | ||
| echo [Wrapper] Warning: %USERPROFILE% contains a space and has no 8.3 short name. >&2 | ||
| echo [Wrapper] Bazel may fail on it. Enable 8.3 names, or move the profile. >&2 | ||
| ) | ||
|
|
||
| set "WRAPPER_CACHE_DIR=%USERPROFILE%\.cache\bazel" | ||
|
|
||
| :: 1. Figure out which bazel we are supposed to hand off to. | ||
| if defined BAZEL_OVERRIDE ( | ||
| echo Actually calling "%BAZEL_OVERRIDE%" | ||
| set "BAZEL_TARGET=%BAZEL_OVERRIDE%" | ||
| ) else ( | ||
| rem Ensure Bazelisk integration. | ||
| if not defined BAZEL_REAL ( | ||
| echo Error: This script must be run via Bazelisk on Windows. >&2 | ||
| exit /b 1 | ||
|
AustinSchuh marked this conversation as resolved.
|
||
| ) | ||
| set "BAZEL_TARGET=%BAZEL_REAL%" | ||
| ) | ||
|
|
||
| :: 2. The pinned hermetic git. | ||
| set "GIT_RELEASE_TAG=v2.44.0.windows.1" | ||
|
AustinSchuh marked this conversation as resolved.
Outdated
|
||
| set "GIT_ARCHIVE_NAME=PortableGit-2.44.0-64-bit.7z.exe" | ||
| set "GIT_EXPECTED_SHA256=1fc64ca91b9b475ab0ada72c9f7b3addbe69a6c8f520be31425cf21841cca369" | ||
|
|
||
| :: Key the cache by the whole pin, tag and checksum both, so that any change to | ||
| :: the three lines above installs the new release instead of being short | ||
| :: circuited by a git.exe that an older revision of this script left behind. | ||
| set "GIT_CACHE_DIR=%WRAPPER_CACHE_DIR%\portable_git\%GIT_RELEASE_TAG%-%GIT_EXPECTED_SHA256:~0,12%" | ||
| set "GIT_EXE_PATH=%GIT_CACHE_DIR%\cmd\git.exe" | ||
|
|
||
| if exist "%GIT_EXE_PATH%" goto git_ready | ||
|
AustinSchuh marked this conversation as resolved.
|
||
| call :install_git | ||
| if errorlevel 1 exit /b 1 | ||
| :git_ready | ||
|
|
||
| :: 3. Put the pinned toolchain in front of anything installed on the machine. | ||
| :: PATH is one of the two variables --experimental_strict_repo_env still lets | ||
| :: through, so this is what a repository rule resolves bash, sh, and git to. | ||
| set "PATH=%GIT_CACHE_DIR%\cmd;%GIT_CACHE_DIR%\bin;%GIT_CACHE_DIR%\usr\bin;%PATH%" | ||
|
|
||
| :: Bind the tools explicitly as well, so nothing depends on a PATH lookup at all. | ||
| :: Each of these is passed through by a --repo_env line in .bazelrc. | ||
| set "BAZEL_SH=%GIT_CACHE_DIR%\bin\bash.exe" | ||
| set "BAZEL_GIT=%GIT_CACHE_DIR%\cmd\git.exe" | ||
| set "GIT_BIN_PATH=%GIT_CACHE_DIR%\cmd\git.exe" | ||
|
|
||
| :: Isolating the pinned git from the developer's ~/.gitconfig belongs in | ||
| :: .bazelrc, not here. Exported from this script it would reach every process | ||
| :: Bazel launches, and "bazel run //:copybara" needs the real configuration to | ||
| :: find a credential helper and an identity to push with. | ||
|
|
||
| :: 4. Execute Bazel. | ||
| "%BAZEL_TARGET%" %* | ||
| exit /b %ERRORLEVEL% | ||
|
|
||
| :: Strip the spaces certutil pads its digest with. A subroutine because reading | ||
| :: a variable assigned in the same block would need delayed expansion. | ||
| :set_computed_sha256 | ||
| set "SHA_LINE=%~1" | ||
| set "COMPUTED_SHA256=%SHA_LINE: =%" | ||
| exit /b 0 | ||
|
|
||
| :: Fetch, verify, and install the pinned PortableGit release. | ||
| :: | ||
| :: Everything happens in a staging directory private to this process and is | ||
| :: published with a single rename, so that two Bazelisk invocations racing on a | ||
| :: cold cache can't corrupt each other's download and a failed install never | ||
| :: leaves a half unpacked tree behind for the next invocation to trust. | ||
| :install_git | ||
| echo [Wrapper] Git %GIT_RELEASE_TAG% is not in the runtime cache. Fetching isolated PortableGit... >&2 | ||
|
|
||
| set "STAGE_DIR=%WRAPPER_CACHE_DIR%\portable_git\staging_%RANDOM%_%RANDOM%" | ||
| if exist "%STAGE_DIR%" rmdir /s /q "%STAGE_DIR%" | ||
| mkdir "%STAGE_DIR%" 2>nul | ||
| if not exist "%STAGE_DIR%" ( | ||
| echo Error: Unable to create the staging directory %STAGE_DIR% >&2 | ||
| exit /b 1 | ||
| ) | ||
|
|
||
| set "GIT_URL=https://github.com/git-for-windows/git/releases/download/%GIT_RELEASE_TAG%/%GIT_ARCHIVE_NAME%" | ||
| echo [Wrapper] Downloading from %GIT_URL% ... >&2 | ||
|
|
||
| curl -fL --silent --show-error --output "%STAGE_DIR%\git.7z.exe" "%GIT_URL%" | ||
| if errorlevel 1 ( | ||
| echo Error: Failed to download the hermetic Git toolchain >&2 | ||
| rmdir /s /q "%STAGE_DIR%" | ||
| exit /b 1 | ||
| ) | ||
|
|
||
| echo [Wrapper] Validating cryptographic payload checksum... >&2 | ||
| set "COMPUTED_SHA256=" | ||
| for /f "skip=1 delims=" %%A in ('certutil -hashfile "%STAGE_DIR%\git.7z.exe" SHA256 ^| findstr /v "CertUtil"') do call :set_computed_sha256 "%%A" | ||
|
AustinSchuh marked this conversation as resolved.
Outdated
|
||
|
|
||
| if /i not "%COMPUTED_SHA256%"=="%GIT_EXPECTED_SHA256%" ( | ||
| echo. >&2 | ||
| echo =============================================================== >&2 | ||
| echo SECURITY ERROR: Cryptographic checksum mismatch detected. >&2 | ||
| echo Expected: %GIT_EXPECTED_SHA256% >&2 | ||
| echo Received: %COMPUTED_SHA256% >&2 | ||
| echo =============================================================== >&2 | ||
| rmdir /s /q "%STAGE_DIR%" | ||
| exit /b 1 | ||
| ) | ||
| echo [Wrapper] Integrity verification successful. SHA256 matches. >&2 | ||
|
|
||
| echo [Wrapper] Extracting archive package... >&2 | ||
| "%STAGE_DIR%\git.7z.exe" -y -o"%STAGE_DIR%\portable_git" >nul | ||
| set "EXTRACT_STATUS=%errorlevel%" | ||
| del /q "%STAGE_DIR%\git.7z.exe" 2>nul | ||
|
|
||
| if not "%EXTRACT_STATUS%"=="0" ( | ||
| echo Error: Extracting the hermetic Git toolchain failed with status %EXTRACT_STATUS% >&2 | ||
| rmdir /s /q "%STAGE_DIR%" | ||
| exit /b 1 | ||
| ) | ||
| if not exist "%STAGE_DIR%\portable_git\cmd\git.exe" ( | ||
| echo Error: cmd\git.exe is missing from the extracted toolchain >&2 | ||
| rmdir /s /q "%STAGE_DIR%" | ||
| exit /b 1 | ||
| ) | ||
| if not exist "%STAGE_DIR%\portable_git\bin\bash.exe" ( | ||
| echo Error: bin\bash.exe is missing from the extracted toolchain >&2 | ||
| rmdir /s /q "%STAGE_DIR%" | ||
| exit /b 1 | ||
| ) | ||
|
|
||
| :: Publish with a rename. If another process won the race then GIT_CACHE_DIR | ||
| :: already exists, and move drops our copy inside it rather than replacing it, | ||
| :: so undo that and keep theirs -- it passed the same checks ours did. | ||
| move "%STAGE_DIR%\portable_git" "%GIT_CACHE_DIR%" >nul 2>&1 | ||
| if exist "%GIT_CACHE_DIR%\portable_git" rmdir /s /q "%GIT_CACHE_DIR%\portable_git" | ||
| rmdir /s /q "%STAGE_DIR%" 2>nul | ||
|
|
||
| if not exist "%GIT_EXE_PATH%" ( | ||
| echo Error: Failed to install the hermetic Git toolchain into %GIT_CACHE_DIR% >&2 | ||
| exit /b 1 | ||
| ) | ||
|
|
||
| echo [Wrapper] Isolated Git runtime setup completed successfully. >&2 | ||
| exit /b 0 | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.