Skip to content

Enable method-level parallelism for TemplateEngine CLI tests #55631

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Evangelink merged 2 commits into from
Aug 7, 2026
+17 −4
Merged

Enable method-level parallelism for TemplateEngine CLI tests #55631

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
f0ed962
Enable method-level parallelism for TemplateEngine CLI tests
Evangelink Aug 6, 2026
1d402ff
Document project-level test parallelization
Evangelink Aug 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 6 additions & 4 deletions test/AGENTS.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,12 +31,14 @@ Guidance for changes under `test/`.
`test/TestAssets/`. They are automatically deployed to Helix via `test/UnitTests.proj`.
- **Don't raise parallelism.** MSTest is repo-defaulted to `None` in
`test/Directory.Build.props` because of concurrency flakiness; a few projects opt
into `ClassLevel`. Cranking it up causes Helix over-subscription/timeouts.
into `ClassLevel` or `MethodLevel` after auditing their shared resources. Cranking it
up without that audit causes Helix over-subscription/timeouts and test interference.
- **In parallelized projects, prefer `[ResourceLock]` over `[DoNotParallelize]`.** In the
projects that do opt in (`Microsoft.NET.Build.Tests`, `dotnet-watch.Tests`,
`Microsoft.NET.Build.Containers.UnitTests`), MSTest's parallel-safety analyzers
(MSTEST0073–MSTEST0077) are active, and `MSTestAnalysisMode=Recommended` plus
`TreatWarningsAsErrors` makes them build errors. Fix them in this order:
`Microsoft.NET.Build.Containers.UnitTests`, `Microsoft.TemplateEngine.Cli.UnitTests`),
MSTest's parallel-safety analyzers (MSTEST0073–MSTEST0077) are active, and
`MSTestAnalysisMode=Recommended` plus `TreatWarningsAsErrors` makes them build errors.
Fix them in this order:
1. **Eliminate the shared state** — pass an environment variable to the child process
via `TestCommand.WithEnvironmentVariable(...)` instead of
`Environment.SetEnvironmentVariable`, and give each test its own scratch directory
Expand Down
1 change: 1 addition & 0 deletions test/Microsoft.TemplateEngine.Cli.UnitTests/Microsoft.TemplateEngine.Cli.UnitTests.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
<ImplicitUsings>enable</ImplicitUsings>
<StrongNameKeyId>MicrosoftAspNetCore</StrongNameKeyId>
<GenerateDocumentationFile>true</GenerateDocumentationFile>
<MSTestParallelizeScope>MethodLevel</MSTestParallelizeScope>
Comment thread
Evangelink marked this conversation as resolved.
</PropertyGroup>

<ItemGroup>
Expand Down
10 changes: 10 additions & 0 deletions .../Microsoft.TemplateEngine.Cli.UnitTests/PostActionTests/AddJsonPropertyPostActionTests.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,8 @@
namespace Microsoft.TemplateEngine.Cli.UnitTests.PostActionTests
{
[TestClass]
// These tests mutate the process-global Microsoft.DotNet.Cli.Utils.Reporter.
[ResourceLock(nameof(Reporter))]
public class AddJsonPropertyPostActionTests
{
// MSTest has no IClassFixture equivalent; a lazily-initialized static helper
Expand All @@ -22,15 +24,23 @@ public class AddJsonPropertyPostActionTests
new(() => new EnvironmentSettingsHelper());

private IEngineEnvironmentSettings _engineEnvironmentSettings = null!;
private IReporter _originalErrorReporter = null!;

[TestInitialize]
public void TestInitialize()
{
_originalErrorReporter = Reporter.Error;
_engineEnvironmentSettings = s_environmentSettingsHelper.Value.CreateEnvironment(
hostIdentifier: GetType().Name,
virtualize: true);
}

[TestCleanup]
public void TestCleanup()
{
Reporter.SetError(_originalErrorReporter);
}

[ClassCleanup]
public static void ClassCleanup()
{
Expand Down
Loading