[vs18.10] Expand built-in metadata references in the task host

[JanProvaznik]

Backport of #14770 to vs18.10. Fixes #14763.

Summary

Building the VS repository with CloudBuild in multithreaded mode (-mt) produces wrong file paths. This corrects it. Normal builds are not affected.

Who is affected today	Internal CloudBuild users building with -mt.
Symptom	A task receives the text %(Filename) instead of the file name. One build wrote a path ending in \%(Filename).cs.
Who is not affected	Every build that does not use a task host. The changed code only runs when a task runs in a separate process.
Risk	Low. The change corrects values that no build can use today: text where a path was meant, or a stale path.
Product code	72 lines.
Test code	602 lines.

Why this matters now

The defect is old, not a regression. It became visible because -mt sends almost every task to a separate process, where the defect lives. Before -mt, a build had to opt a single task into that process to see it, which is rare.

We need VS msbuild updated with the fix to continue Cloudbuild dogfooding

What was wrong

A project can set default metadata for a group of items:

<ItemDefinitionGroup>
  <PreprocessCompile>
    <OutputName>%(Filename)</OutputName>
  </PreprocessCompile>
</ItemDefinitionGroup>

A task in the MSBuild process reads OutputName as hello. The same task in a separate process reads %(Filename), the literal text. The task then uses that text as if it were a file name.

MSBuild resolves %(Filename) at the moment the metadata is read, not once in advance, because a task can rename an item and the value has to follow. When an item is sent to another process, the data that says "this value still needs to be resolved" is dropped. The receiving side returned the text as it was.

This change lets the receiving side recognise such a value and resolve it, so a task reads the same value in either process. Nothing about the data sent between processes changes.

Scope of the change

Area	Lines	What
Product	72	Resolve the value when it is read in a task host.
Tests	602	Five corrected behaviours, the parser, and an end-to-end run in a real task host.
Release infrastructure	2	VersionPrefix to 18.10.1, and insert into VS rel/stable.

The product change is 220 added lines in two files, of which 72 are code and the rest are documentation comments and braces. The code that runs during a normal build is untouched.

Five ways a task could tell that it was running in a task host are corrected. Each was measured, running the same task in both processes.

#	Difference	In process	In task host, before
1	The task reads the metadata.	hello	%(Filename)
2	The task renames the item, then reads again.	renamed	%(Filename)
3	The value uses %(RecursiveDir), e.g. out\%(RecursiveDir)%(Filename)%(Extension).	out\sub1\sub2\hello.txt	out\%(RecursiveDir)hello.txt
4	The task copies its input, with new TaskItem(input) or CopyMetadataTo.	hello	%(Filename)
5	The task reads a full path, renames the item, then reads again.	the new path	the old path

Row 5 is a separate old defect, not caused by this change. The released MSBuild returns the same stale path. It is corrected here because the fix for the other rows would otherwise have spread it further.

Why normal builds cannot be affected

The changed type is only created when items are sent between processes. There are four such places, all of them task host or resolver traffic. A task in the MSBuild process never reaches this code.

• No change to the data sent between processes, so no version negotiation and no packet version increase.
• A task host built from older sources runs older code and behaves exactly as before.
• The legacy task host used for .NET Framework 3.5 tasks has its own separate copy of this type and is untouched.
• No change wave. A project cannot choose whether a task runs in a task host under -mt, so no build chose the current behaviour and none can depend on it.

The full technical description, including two limits that are deliberately left as they are, is in #14770.

Backport notes

All five commits from #14770 apply to vs18.10 with no conflict. No changes were needed to make them apply.

Two release infrastructure changes are included:

• VersionPrefix moves from 18.10.0 to 18.10.1.
• The automatic VS insertion target moves from rel/insiders to rel/stable, since 18.10 has snapped to stable. This is step 4.6 of the release checklist, and matches vs18.9.

Validation evidence

The change was validated with three experimental Visual Studio insertions. The two forced variants differ by this fix, giving a direct control instead of relying only on unit-test results.

Insertion	Configuration	Result
773756	This fix, normal task placement	All required and optional product validations passed. Both performance systems reported no regression.
773755	Control: all eligible tasks forced into TaskHost, without this fix	Six C++ build scenarios failed where tasks consumed items after the process boundary. The failures covered multi-tool execution, assembler inputs, custom build, STL modules, and shader compilation.
773754	All eligible tasks forced into TaskHost, with this fix	The same six C++ scenarios passed.

This comparison establishes two separate facts:

1. The normal product configuration remains unaffected: the insertion containing the fix completed the full VS validation and performance runs without a regression.
2. Under the artificial configuration that exposes the defect, adding the fix removes all six task-item failures observed in the control.

The remaining failures do not distinguish the fix from the control:

• Both forced variants failed DownloadComponents in the same way because an isolated CLR4 TaskHost could not apply the parent process's assembly-version binding policy. The normal insertion passed. This is a separate TaskHost compatibility limitation, tracked by #14838.
• The bug forcing variants produced effectively identical performance costs from moving every task across a process boundary. The fix as it would be used insertion had no performance regression.

Conclusion: the controlled forced-task comparison reproduces the customer-impacting failure without the fix and removes it with the fix. Normal task execution shows no correctness or performance change, and no repeatable failure remains unique to the fix.

[Copilot]

Pull request overview

Backports the fix for #14763 to vs18.10 so that item-definition metadata containing built-in metadata references (for example %(Filename)) is expanded consistently when tasks execute out-of-proc in a TaskHost, matching in-proc behavior and preventing literal %(...) from leaking into task-observed paths/values.

Changes:

• Teach TaskParameterTaskItem (the marshalled item crossing the TaskHost boundary) to expand item-definition built-in metadata references on read while preserving task-written metadata as literal.
• Add a minimal BuiltInMetadataExpander in Microsoft.Build.Framework to expand only built-in metadata references in escaped strings, including correct handling of RecursiveDir.
• Add regression/unit tests covering cross-boundary parity, ItemSpec reassignment cache invalidation, and end-to-end TaskHost execution; bump servicing version to 18.10.1.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
src/Shared/TaskParameter.cs	Expands unexpanded built-in metadata on read for marshalled items; tracks task-written metadata; clears cached modifiers when ItemSpec changes; expands during metadata copy operations.
src/Shared/UnitTests/TaskParameter_Tests.cs	Adds extensive regression coverage for TaskParameter round-tripping and task-host boundary semantics.
src/Framework/BuiltInMetadataExpander.cs	Introduces a lightweight expander for built-in metadata references needed in task-host scenarios.
src/Framework.UnitTests/BuiltInMetadataExpander_Tests.cs	Adds unit tests validating expansion behavior, marker scanning, and allocation-sensitive behavior.
src/Build.UnitTests/BackEnd/MetadataObservationTask.cs	Adds a test task used to observe metadata values and confirm in-proc vs out-of-proc execution.
src/Build.UnitTests/BackEnd/ItemDefinitionMetadataInTaskHost_Tests.cs	Adds end-to-end TaskHost regression tests asserting task-observed metadata matches in-proc behavior.
eng/Versions.props	Updates VersionPrefix from 18.10.0 to 18.10.1 for the servicing backport.

---

💡 Add a code-review agent skill for context-aware, tailored reviews. Learn more in the docs.

[ViktorHofer]

Consider updating this value as well now that 18.10 is stable: https://github.com/JanProvaznik/msbuild/blob/3c551b98b940fb989ffb026282b499a953b0b97c/azure-pipelines/vs-insertion.yml#L64

Then you don't have to manually pick the insertion branch and we are prepared for the next months in case there's any other servicing event :)

[JanProvaznik]

Note

This comment was generated by GitHub Copilot CLI.

Good catch, done in 2aa0df0.

This is step 4.6 of the release checklist ("retarget AutoInsertTargetBranch for vs18.10 -> rel/stable" after the stable snap), and it now matches vs18.9. So the three branches line up: main -> main, vs18.10 -> rel/stable, and the previous stable branch already there.