[release/10.0.4xx] Add dotnet test --results-directory-layout - #55629
Open
Evangelink wants to merge 1 commit into
Open
[release/10.0.4xx] Add dotnet test --results-directory-layout#55629Evangelink wants to merge 1 commit into
Evangelink wants to merge 1 commit into
Conversation
When 'dotnet test' runs a solution, every test application resolves the same shared TestResults directory, so reports configured with a relative file name (--coverage-output, --report-trx-filename, ...) overwrite each other and coverage data is silently lost (microsoft/codecoverage#226). Add an SDK-side layout policy for the Microsoft.Testing.Platform path: dotnet test --results-directory-layout <flat|per-module> 'flat' is the default and keeps today's behavior. 'per-module' gives every test application its own directory, modeled on the artifacts output layout: <results-directory>/<project>/<target-framework>_<runtime> Microsoft.Testing.Platform gains no new option; the SDK simply passes an explicit --results-directory per child process, so the whole module set is resolved up front. That also lets the layout append an identity hash to the project folder only when two distinct projects in the run share a name. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0cb64c89-128b-4f8f-88cc-2741ffac3767 (cherry picked from commit d73fa64)
|
Azure Pipelines: Successfully started running 1 pipeline(s). 2 pipeline(s) were filtered out due to trigger conditions. There may be pipelines that require an authorized user to comment /azp run to run. |
Contributor
There was a problem hiding this comment.
Pull request overview
Backports dotnet test --results-directory-layout to the .NET 10.0.4xx servicing branch for the Microsoft.Testing.Platform (MTP) execution path, enabling an opt-in per-module results directory structure to prevent test report files from overwriting each other when multiple test applications run from a solution.
Changes:
- Adds
--results-directory-layout <flat|per-module>to the MTPdotnet testcommand definition, help text, and localized resources. - Introduces a
TestResultsDirectoryResolverand threads it through the MTP execution pipeline so each test module can receive an isolated--results-directory. - Adds unit + end-to-end coverage plus new test assets to validate overwrite behavior, per-module isolation, and same-project-name disambiguation.
Show a summary per file
| File | Description |
|---|---|
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/TestProjectB/TestProjectB.csproj | Adds MTP test-asset project B used to reproduce shared report filename overwrites. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/TestProjectB/Program.cs | Writes a fixed relative report.txt into the MTP results directory to validate overwrite vs isolation. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/TestProjectA/TestProjectA.csproj | Adds MTP test-asset project A used to reproduce shared report filename overwrites. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/TestProjectA/Program.cs | Writes a fixed relative report.txt into the MTP results directory to validate overwrite vs isolation. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/MultiTestProjectSolutionWithSharedReportName.sln | Adds solution containing two MTP test apps for overwrite regression coverage. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithSharedReportName/global.json | Forces MTP as the test runner for the new overwrite regression asset. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/src/Tests/Tests.csproj | Adds one of two same-named projects to validate per-module disambiguation. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/src/Tests/Program.cs | Writes a fixed relative report file to validate disambiguated per-module isolation. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/samples/Tests/Tests.csproj | Adds the second same-named project to validate per-module disambiguation. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/samples/Tests/Program.cs | Writes a fixed relative report file to validate disambiguated per-module isolation. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/MultiTestProjectSolutionWithDuplicateProjectNames.slnx | Adds a solution model containing two same-named projects in different folders. |
| test/TestAssets/TestProjects/MultiTestProjectSolutionWithDuplicateProjectNames/global.json | Forces MTP as the test runner for the duplicate-name disambiguation asset. |
| test/dotnet.Tests/CommandTests/Test/TestResultsDirectoryResolverTests.cs | Adds focused unit tests for flat vs per-module, pivot naming, disambiguation, and stability. |
| test/dotnet.Tests/CommandTests/Test/TestCommandParserTests.cs | Adds parser tests for --results-directory-layout defaulting/values and invalid-value rejection. |
| test/dotnet.Tests/CommandTests/Test/snapshots/MTPHelpSnapshotTests.VerifyMTPHelpOutput.verified.txt | Updates MTP help snapshot to include --results-directory-layout documentation. |
| test/dotnet.Tests/CommandTests/Test/GivenDotnetTestBuildsAndRunsTests.cs | Adds end-to-end coverage for per-module directory creation, overwrite regression, and disambiguation. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.zh-Hant.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.zh-Hans.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.tr.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ru.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.pt-BR.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.pl.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ko.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.ja.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.it.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.fr.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.es.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.de.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/xlf/CommandDefinitionStrings.cs.xlf | Adds localized string entries for the new option help/placeholder. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/Commands/Test/TestCommandDefinition.MicrosoftTestingPlatform.cs | Introduces --results-directory-layout option and wires it into the MTP command options list. |
| src/Cli/Microsoft.DotNet.Cli.Definitions/CommandDefinitionStrings.resx | Adds resource strings for the new option help text and help-name placeholder. |
| src/Cli/dotnet/Commands/Test/MTP/TestResultsDirectoryResolver.cs | Implements per-module results directory computation, pivot naming, and name disambiguation hashing. |
| src/Cli/dotnet/Commands/Test/MTP/TestModulesFilterHandler.cs | Refactors to materialize/enumerate modules up front and enqueue after resolver construction. |
| src/Cli/dotnet/Commands/Test/MTP/TestApplicationActionQueue.cs | Threads a shared resolver into worker tasks and per-module TestApplication creation. |
| src/Cli/dotnet/Commands/Test/MTP/TestApplication.cs | Uses the resolver to supply the correct per-module --results-directory argument. |
| src/Cli/dotnet/Commands/Test/MTP/Options.cs | Adds ResultsDirectoryLayout and extends PathOptions to carry the selected layout policy. |
| src/Cli/dotnet/Commands/Test/MTP/MSBuildUtility.cs | Parses --results-directory-layout into PathOptions.ResultsDirectoryLayout. |
| src/Cli/dotnet/Commands/Test/MTP/MSBuildHandler.cs | Exposes EnumerateTestModules() so the full module set can be used to build the resolver. |
| src/Cli/dotnet/Commands/Test/MTP/MicrosoftTestingPlatformTestCommand.cs | Constructs the resolver from the complete module set and injects it into the action queue for both flows. |
Copilot's findings
- Files reviewed: 39/39 changed files
- Comments generated: 0
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Backport of #55475 to the .NET 10.0.4xx servicing branch.
When
dotnet testruns a solution through Microsoft.Testing.Platform, all test applications currently share the sameTestResultsdirectory. Reports configured with the same relative filename can overwrite one another and silently lose results.This adds:
dotnet test --results-directory-layout per-moduleThe per-module layout isolates each test application under:
The default remains
flat, preserving existing behavior unless users opt in.The implementation was adapted to the older MTP command architecture on
release/10.0.4xx; no additional prerequisite backports are required.Validation
dotnet.Testsbuilds successfully.Fixes microsoft/testfx#10116.