-
Notifications
You must be signed in to change notification settings - Fork 8.5k
Advanced Paste additional customizations and PhiSilica provider #46727
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+2,371
−259
Merged
Changes from 101 commits
Commits
Show all changes
109 commits
Select commit
Hold shift + click to select a range
a48e741
initial
khmyznikov eed0e20
fix shortcut
khmyznikov ecc93ea
more customization
khmyznikov 1c78950
more improvements
khmyznikov 6072ec7
fix settings buttons
khmyznikov 74eafb5
Custom models picker, default models picker
khmyznikov f6149bb
default provider
khmyznikov a025d4d
add settings collapse
khmyznikov f8b1586
Phi Silica not tested
khmyznikov a3620f1
add settings ui manifest
khmyznikov a25ce4e
WinAppSdk 2.0 try
khmyznikov b5fc9fe
Revert "WinAppSdk 2.0 try"
khmyznikov 51c5d32
upgrade to 260317003
khmyznikov dc14845
Add LAF helper
khmyznikov 8224062
add workaround
khmyznikov c2789d6
launch hacks
khmyznikov a60eab4
add more params from sample
khmyznikov 32ffd3a
change workaround
khmyznikov 9421efe
update readme
khmyznikov fd96b46
Add laf as ADO secret
khmyznikov 0a2bc81
fix laf attestation string
khmyznikov f53e64a
fix xaml
khmyznikov 9f6d6b9
Fix build
khmyznikov 0343840
fix installer
khmyznikov 0bc4504
Merge branch 'gleb/advanced-paste' of https://github.com/microsoft/Po…
khmyznikov 401ddcd
move AP to packaged identity
khmyznikov d28ae33
fix ap package
khmyznikov 4fac403
debug build
khmyznikov 36366e1
fix msix
khmyznikov c46e614
fix build
khmyznikov 26bbfde
another try
khmyznikov 3dfeece
Merge branch 'main' into gleb/advanced-paste
khmyznikov 7436b46
another try 2
khmyznikov 9befd28
another try 3
khmyznikov ab9a8c9
fix ap msix search
khmyznikov ce8508e
fix ap vnext
khmyznikov c43261f
keep fixing msix
khmyznikov faae1ae
to prev
khmyznikov 00bc7ec
to prev 2
khmyznikov 749378d
to prev 3
khmyznikov 3281c0e
fix missing VCLibs
khmyznikov d628d73
AP DisableTransitiveFrameworkReferences
khmyznikov 011fc5a
exclude AP from audit
khmyznikov dbae67d
Fixing AP crash
khmyznikov d776b64
another thing
khmyznikov 68eb695
maybe final fix?
khmyznikov 822303a
fix LAF tokens
khmyznikov 9e02f48
fix debug tokens
khmyznikov 96c5d16
Fix AP uninstall
khmyznikov 3269de6
Merge branch 'main' into gleb/advanced-paste
khmyznikov 7416209
address PR review
khmyznikov ca7bc16
to prev
khmyznikov 8370908
build fix on VS2026
khmyznikov 79f850b
Merge remote-tracking branch 'origin/main' into gleb/advanced-paste
khmyznikov 6657da8
move to wasdk 2.0
khmyznikov c2caa46
to prev
khmyznikov 832edcd
fix vnext
khmyznikov d3c8b48
fix spelling
khmyznikov 17a0533
fix ap tests
khmyznikov 3619593
get back to sparse
khmyznikov 0fcd729
revert the CI changes
khmyznikov 125fffc
fix vnext
khmyznikov 279aaf6
fix the pipeline
khmyznikov 6d1c414
fix vnext
khmyznikov 7da7a0b
AdvancedPaste: flatten install layout to WinUI3Apps root
khmyznikov abcb77d
AdvancedPaste: rename ProjectPriFileName to PowerToys.AdvancedPaste.pri
khmyznikov 5931b3d
ESRPSigning_core: drop AdvancedPaste subfolder path
khmyznikov b2251ca
wixproj: pass CmdPalPackagePath/CmdPalPackageDir defines to WiX
khmyznikov f4ddb79
Revert flatten: restore AP subfolder layout with heat harvest + ICE03…
khmyznikov 7a3539f
Revert heat-based installer: AP is flat in WinUI3Apps like ImageResizer
khmyznikov ec475f0
to prev
khmyznikov 0f7c3f7
AdvancedPaste: pass PRI filename to ResourceLoader explicitly
khmyznikov c6176fb
AdvancedPaste: revert PRI to resources.pri for XAML LoadComponent
khmyznikov 020dd67
Revert PRI to PowerToys.AdvancedPaste.pri: resources.pri breaks Settings
khmyznikov 9ea621b
Bump WindowsAppSDK.Foundation to 2.0.22 (sparse PRI fix)
khmyznikov 450568a
Revert generated installer wxs files and stale build artifacts
khmyznikov 6f98152
revert change
khmyznikov cc55315
Merge branch 'main' into gleb/advanced-paste
khmyznikov e7e3db8
Merge branch 'main' into gleb/advanced-paste
khmyznikov e9221d2
Merge branch 'gleb/advanced-paste' of https://github.com/microsoft/Po…
khmyznikov ccbd288
Merge branch 'gleb/advanced-paste' of https://github.com/microsoft/Po…
khmyznikov 5e0f92f
Merge branch 'gleb/advanced-paste' of https://github.com/microsoft/Po…
khmyznikov 56af1e4
Merge branch 'gleb/advanced-paste' of https://github.com/microsoft/Po…
khmyznikov 046796a
Merge branch 'gleb/advanced-paste' of https://github.com/microsoft/Po…
khmyznikov 96fb1d0
docs: fix stale PRI and path references in advancedpaste.md
khmyznikov 2572b12
bump to 2.2.2-experimental9
khmyznikov aa09683
revert nuget config
khmyznikov faf0a7c
return Foundation version
khmyznikov a2ee84f
fix xaml warnings
khmyznikov c648980
temp fix for resources
khmyznikov a782972
Revert temp continueOnError on Touchdown step; service recovered
khmyznikov b63c319
Merge remote-tracking branch 'origin/main' into gleb/advanced-paste
khmyznikov ca38547
ci: run cmdpal versioning setup last so CmdPalVersion matches stamped…
khmyznikov 4d1171f
fix AP location call
khmyznikov 64071b7
add model prepare
khmyznikov 2019d0b
fix AI package version
khmyznikov 1729e26
add model download trigger + winappsdk version fix
khmyznikov 3768a95
Merge branch 'main' into gleb/advanced-paste
khmyznikov 714da1a
add unlock status
khmyznikov 54943c9
fix missing metadata
khmyznikov f6c8550
some knowlegde updates
khmyznikov 2710f0c
fix winmd collision
khmyznikov 4ce6529
Address PR review
khmyznikov b123a4f
fix wording
khmyznikov 05e4076
remove unused word
khmyznikov 6e6e637
Merge branch 'main' into gleb/advanced-paste
khmyznikov 780abe1
add more details for API state
khmyznikov c52d147
Restore SCL to spell-check expect.txt
niels9001 5ec6dd4
Address comments
khmyznikov File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -309,6 +309,12 @@ pwa | |
| AOT | ||
| Aot | ||
| ify | ||
| LAF | ||
| Laf | ||
| languagemodel | ||
| philm | ||
| phisilica | ||
| SCL | ||
| TFM | ||
|
|
||
| # YML | ||
|
|
||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
100 changes: 100 additions & 0 deletions
100
doc/devdocs/modules/advancedpaste-phisilica-local-testing.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,100 @@ | ||
| # Advanced Paste – Phi Silica local testing | ||
|
|
||
| How to build, register, and test **Phi Silica** in **Advanced Paste (AP)** on a dev machine, | ||
| plus the few things that actually break it. | ||
|
|
||
| ## How it fits together | ||
|
|
||
| AP ships as an **unpackaged, self-contained WinUI 3 exe** (`PowerToys.AdvancedPaste.exe`). | ||
| The Windows AI `LanguageModel` (Phi Silica) API is a **Limited Access Feature (LAF)**. For it | ||
| to work, all of these must line up: | ||
|
|
||
| 1. **Package identity** — AP runs with identity granted by the sparse MSIX | ||
| `Microsoft.PowerToys.SparseApp`. | ||
| 2. **Matching LAF creds** — the token/attestation baked into the exe match the registered | ||
| sparse package's publisher. | ||
| 3. **AI metadata deployed** — the `Microsoft.Windows.AI*.winmd` files ship next to the exe; | ||
| the AI runtime resolves them **at runtime**. | ||
| 4. **Model ready** — supported hardware and the on-device model downloaded | ||
| (`GetReadyState() == Ready`). | ||
|
|
||
| Two identities — the baked token must match the registered package's publisher: | ||
|
|
||
| | Build | Publisher Id | LAF creds | | ||
| |-------|--------------|-----------| | ||
| | **Dev** | `djwsxzxb4ksa8` | dev default in [`src/PhiSilicaLaf.props`](../../../src/PhiSilicaLaf.props) | | ||
| | **Prod** | `8wekyb3d8bbwe` | secret, injected only by `.pipelines/v2/release.yml` | | ||
|
|
||
| Non-secret pairing check: the exe's baked **Attestation** must equal the registered package's | ||
| **PublisherId**. | ||
|
|
||
| ## Build + register (dev loop) | ||
|
|
||
| ```powershell | ||
| $repo = "X:\GitHub\PowerToys"; $Plat = "ARM64"; $Cfg = "Debug" # or x64 / Release | ||
|
|
||
| # Build AP only (C#; reuses existing C++ outputs): | ||
| dotnet restore "$repo\src\modules\AdvancedPaste\AdvancedPaste\AdvancedPaste.csproj" /p:Platform=$Plat | ||
| & "$repo\tools\build\build.cmd" -Path "$repo\src\modules\AdvancedPaste\AdvancedPaste" ` | ||
| -Platform $Plat -Configuration $Cfg /p:BuildProjectReferences=false | ||
|
|
||
| # Register the dev sparse package (creates + trusts a dev cert, grants identity): | ||
| pwsh -ExecutionPolicy Bypass -File "$repo\src\PackageIdentity\BuildSparsePackage.ps1" ` | ||
| -Platform $Plat -Configuration $Cfg -DevRegister | ||
| # Expect: PublisherId djwsxzxb4ksa8, IsDevelopmentMode True | ||
| ``` | ||
|
|
||
| ## Check the API | ||
|
|
||
| `PowerToys.AdvancedPaste.exe` is a **GUI-subsystem** app — run directly in a console it prints | ||
| nothing and returns no exit code. **Redirect** stdout/stderr and wait: | ||
|
|
||
| ```powershell | ||
| $exe = "$repo\$Plat\$Cfg\WinUI3Apps\PowerToys.AdvancedPaste.exe" | ||
| $o = "$env:TEMP\ap.out"; $e = "$env:TEMP\ap.err" | ||
| $p = Start-Process $exe '--check-phi-silica' -Wait -PassThru -WindowStyle Hidden ` | ||
| -RedirectStandardOutput $o -RedirectStandardError $e | ||
| "exit=$($p.ExitCode) stdout=$((Get-Content $o -Raw).Trim())" | ||
| Get-Content $e -Raw # stderr: [phi-silica] LAF unlock status: <…>; ReadyState: <…> | ||
| ``` | ||
|
|
||
| | `--check-phi-silica` | `--prepare-phi-silica` (downloads the model) | | ||
| |----------------------|----------------------------------------------| | ||
| | `0` Available · `1` NotReady · `2` NotSupported / unlock failed | `0` Ready · `1` Failed · `2` NotSupported | | ||
|
|
||
| `--check` only reads state; use `--prepare` to trigger the model download (`EnsureReadyAsync`). | ||
| On failure it prints the `HRESULT` to stderr. | ||
|
|
||
| Confirm the running AP has identity: | ||
|
|
||
| ```powershell | ||
| $apPid = (Get-Process PowerToys.AdvancedPaste -EA SilentlyContinue | Select-Object -First 1).Id | ||
| if ($apPid) { & "$repo\src\PackageIdentity\Check-ProcessIdentity.ps1" -ProcessId $apPid } | ||
| # Expect a PFN ending in the publisher id that matches the baked attestation | ||
| ``` | ||
|
|
||
| ## What actually breaks it | ||
|
|
||
| - **Missing `.winmd` (most important).** The Windows AI runtime resolves | ||
| `Microsoft.Windows.AI*.winmd` from the app folder at runtime. If they aren't deployed, | ||
| `GetReadyState()` returns `NotReady` and `EnsureReadyAsync()` fails with | ||
| `RO_E_METADATA_NAME_NOT_FOUND` (`0x8000000F`) — even though identity, token, and the AI DLLs | ||
| are all correct. The build emits these winmd into `WinUI3Apps\`; the **installer must harvest | ||
| them** (`*.winmd` is in the inclusion list of | ||
| [`generateAllFileComponents.ps1`](../../../installer/PowerToysSetupVNext/generateAllFileComponents.ps1)). | ||
| Classic symptom: "works from the build output but not from the installer" → check that the | ||
| installed `WinUI3Apps\` contains `Microsoft.Windows.AI*.winmd`. | ||
| - **Dev/prod mismatch.** A dev-cred exe running against a prod sparse package (or vice versa) | ||
| makes the LAF unlock silently return `Unavailable`. Keep the exe and the registered package | ||
| the same flavor, and verify with the attestation == publisherId check above. | ||
| - **Forgot to redirect.** `--check-phi-silica` in a console prints nothing — that's the | ||
| GUI-subsystem quirk, not a result. | ||
|
|
||
| ## Cleanup | ||
|
|
||
| ```powershell | ||
| pwsh -ExecutionPolicy Bypass -File "$repo\src\PackageIdentity\BuildSparsePackage.ps1" -Unregister | ||
| ``` | ||
|
|
||
| ⚠️ This removes any `Microsoft.PowerToys.SparseApp` registration, **including a prod one** from | ||
| an installer — reinstall/repair PowerToys to restore it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.