Skip to content

[release/10.0.4xx] Add dotnet test --results-directory-layout - #55629

Open
Evangelink wants to merge 1 commit into
dotnet:release/10.0.4xxfrom
Evangelink:dev/amauryleve/backport-test-results-layout
Open

[release/10.0.4xx] Add dotnet test --results-directory-layout#55629
Evangelink wants to merge 1 commit into
dotnet:release/10.0.4xxfrom
Evangelink:dev/amauryleve/backport-test-results-layout

Conversation

@Evangelink

Copy link
Copy Markdown
Member

Backport of #55475 to the .NET 10.0.4xx servicing branch.

When dotnet test runs a solution through Microsoft.Testing.Platform, all test applications currently share the same TestResults directory. Reports configured with the same relative filename can overwrite one another and silently lose results.

This adds:

dotnet test --results-directory-layout per-module

The per-module layout isolates each test application under:

<results-directory>/<project>/<target-framework>_<runtime>

The default remains flat, preserving existing behavior unless users opt in.

The implementation was adapted to the older MTP command architecture on release/10.0.4xx; no additional prerequisite backports are required.

Validation

  • dotnet.Tests builds successfully.
  • 20 targeted parser, resolver, and end-to-end tests pass.
  • The full repository/redist build progressed through the changed CLI projects but remains blocked locally by unrelated native SDK layout prerequisites/file locking.

Fixes microsoft/testfx#10116.

When 'dotnet test' runs a solution, every test application resolves the same
shared TestResults directory, so reports configured with a relative file name
(--coverage-output, --report-trx-filename, ...) overwrite each other and
coverage data is silently lost (microsoft/codecoverage#226).

Add an SDK-side layout policy for the Microsoft.Testing.Platform path:

  dotnet test --results-directory-layout <flat|per-module>

'flat' is the default and keeps today's behavior. 'per-module' gives every
test application its own directory, modeled on the artifacts output layout:

  <results-directory>/<project>/<target-framework>_<runtime>

Microsoft.Testing.Platform gains no new option; the SDK simply passes an
explicit --results-directory per child process, so the whole module set is
resolved up front. That also lets the layout append an identity hash to the
project folder only when two distinct projects in the run share a name.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 0cb64c89-128b-4f8f-88cc-2741ffac3767
(cherry picked from commit d73fa64)
Copilot AI review requested due to automatic review settings August 6, 2026 08:28
@Evangelink Evangelink added Area-dotnet test backport PR that has been backported to a servicing branch labels Aug 6, 2026
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
2 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Backports dotnet test --results-directory-layout to the .NET 10.0.4xx servicing branch for the Microsoft.Testing.Platform (MTP) execution path, enabling an opt-in per-module results directory structure to prevent test report files from overwriting each other when multiple test applications run from a solution.

Changes:

  • Adds --results-directory-layout <flat|per-module> to the MTP dotnet test command definition, help text, and localized resources.
  • Introduces a TestResultsDirectoryResolver and threads it through the MTP execution pipeline so each test module can receive an isolated --results-directory.
  • Adds unit + end-to-end coverage plus new test assets to validate overwrite behavior, per-module isolation, and same-project-name disambiguation.
Show a summary per file
File Description
test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/TestProjectB/TestProjectB.csproj Adds MTP test-asset project B used to reproduce shared report filename overwrites.
test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/TestProjectB/Program.cs Writes a fixed relative report.txt into the MTP results directory to validate overwrite vs isolation.
test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/TestProjectA/TestProjectA.csproj Adds MTP test-asset project A used to reproduce shared report filename overwrites.
test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/TestProjectA/Program.cs Writes a fixed relative report.txt into the MTP results directory to validate overwrite vs isolation.
test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/MultiTestProjectSolutionWithSharedReportName.sln Adds solution containing two MTP test apps for overwrite regression coverage.
test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/global.json Forces MTP as the test runner for the new overwrite regression asset.
test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/src/Tests/Tests.csproj Adds one of two same-named projects to validate per-module disambiguation.
test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/src/Tests/Program.cs Writes a fixed relative report file to validate disambiguated per-module isolation.
test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/samples/Tests/Tests.csproj Adds the second same-named project to validate per-module disambiguation.
test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/samples/Tests/Program.cs Writes a fixed relative report file to validate disambiguated per-module isolation.
test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/MultiTestProjectSolutionWithDuplicateProjectNames.slnx Adds a solution model containing two same-named projects in different folders.
test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/global.json Forces MTP as the test runner for the duplicate-name disambiguation asset.
test/dotnet.Tests/CommandTests/Test/TestResultsDirectoryResolverTests.cs Adds focused unit tests for flat vs per-module, pivot naming, disambiguation, and stability.
test/dotnet.Tests/CommandTests/Test/TestCommandParserTests.cs Adds parser tests for --results-directory-layout defaulting/values and invalid-value rejection.
test/dotnet.Tests/CommandTests/Test/snapshots/MTPHelpSnapshotTests.VerifyMTPHelpOutput.verified.txt Updates MTP help snapshot to include --results-directory-layout documentation.
test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndRunsTests.cs Adds end-to-end coverage for per-module directory creation, overwrite regression, and disambiguation.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.zh-Hant.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.zh-Hans.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.tr.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ru.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.pt-BR.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.pl.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ko.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ja.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.it.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.fr.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.es.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.de.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.cs.xlf Adds localized string entries for the new option help/placeholder.
src/Cli/Microsoft.DotNet.Cli.Definitions/Commands/Test/TestCommandDefinition.MicrosoftTestingPlatform.cs Introduces --results-directory-layout option and wires it into the MTP command options list.
src/Cli/Microsoft.DotNet.Cli.Definitions/CommandDefinitionStrings.resx Adds resource strings for the new option help text and help-name placeholder.
src/Cli/dotnet/Commands/Test/MTP/TestResultsDirectoryResolver.cs Implements per-module results directory computation, pivot naming, and name disambiguation hashing.
src/Cli/dotnet/Commands/Test/MTP/TestModulesFilterHandler.cs Refactors to materialize/enumerate modules up front and enqueue after resolver construction.
src/Cli/dotnet/Commands/Test/MTP/TestApplicationActionQueue.cs Threads a shared resolver into worker tasks and per-module TestApplication creation.
src/Cli/dotnet/Commands/Test/MTP/TestApplication.cs Uses the resolver to supply the correct per-module --results-directory argument.
src/Cli/dotnet/Commands/Test/MTP/Options.cs Adds ResultsDirectoryLayout and extends PathOptions to carry the selected layout policy.
src/Cli/dotnet/Commands/Test/MTP/MSBuildUtility.cs Parses --results-directory-layout into PathOptions.ResultsDirectoryLayout.
src/Cli/dotnet/Commands/Test/MTP/MSBuildHandler.cs Exposes EnumerateTestModules() so the full module set can be used to build the resolver.
src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs Constructs the resolver from the complete module set and injects it into the action queue for both flows.

Copilot's findings

  • Files reviewed: 39/39 changed files
  • Comments generated: 0

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area-dotnet test backport PR that has been backported to a servicing branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants