Require structured GenPage agent messages - #275
Require structured GenPage agent messages#275Tony Nguyen (nguyentony95) wants to merge 4 commits into
Conversation
Ensure create, edit, and navigation-fix uploads provide user-facing Agent Thoughts and Summary sections using PAC-safe escaped line breaks. Enforce the contract across every recorded upload in Layer 1 evals. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
|
Tony Nguyen (@nguyentony95) please read the following Contributor License Agreement(CLA). If you agree with the CLA, please reply with the following information.
Contributor License AgreementContribution License AgreementThis Contribution License Agreement (“Agreement”) is agreed to by the party signing below (“You”),
|
Require a concise step-by-step implementation summary covering applicable requirements, assumptions, major functions, UX decisions, and final verification without exposing private chain-of-thought. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Bold numbered Agent Thoughts headings and require bulleted Summary items, with eval coverage for both formatting rules. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Require escaped blank lines before bold numbered headings so Adaptive Cards do not merge headings into preceding text. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
Updates the Model Apps GenPage skill and its eval harness to require a structured, PAC-safe --agent-message payload (escaped \n line breaks) so uploads consistently include user-facing “Agent Thoughts”, “Summary”, and “Final Code” sections and avoid leaking private reasoning.
Changes:
- Tighten GenPage skill guidance to require escaped, structured
--agent-messagecontent for create/edit/navigation-fix uploads. - Add a new Layer 1 workflow assertion (with unit tests) that validates the structured
--agent-messagecontract across all uploads in the workflow log. - Refresh Layer 1 fixtures and documentation/changelog to reflect the new contract.
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
| plugins/model-apps/skills/genpage/SKILL.md | Documents the required structured escaped --agent-message format and updates upload examples. |
| plugins/model-apps/skills/genpage/edit-flow.md | Aligns edit-flow upload instructions with the escaped structured --agent-message contract. |
| plugins/model-apps/CHANGELOG.md | Notes the new structured agent-message requirement and Layer 1 enforcement. |
| evals/model-apps/genpage/tests/run-layer-1.test.js | Updates mock artifacts to include a structured escaped --agent-message in recorded uploads. |
| evals/model-apps/genpage/tests/assertions-layer-1.test.js | Adds unit tests covering pass/fail cases for the new agent-message contract assertion. |
| evals/model-apps/genpage/lib/assertions-layer-1.js | Implements the new workflow assertion that validates structured escaped --agent-message on every upload. |
| evals/model-apps/genpage/fixtures/7-job-candidates-new-entities/workflow-log.md | Updates fixture upload log to include structured escaped --agent-message. |
| evals/model-apps/genpage/fixtures/5-kanban-task-board/workflow-log.md | Updates fixture upload log to include structured escaped --agent-message. |
| evals/model-apps/genpage/fixtures/4-case-wizard/workflow-log.md | Updates fixture upload log to include structured escaped --agent-message. |
| evals/model-apps/genpage/fixtures/2-mock-dashboard/workflow-log.md | Updates fixture upload log to include structured escaped --agent-message. |
| evals/model-apps/genpage/fixtures/2-mock-dashboard-real/workflow-log.md | Updates fixture upload log to include structured escaped --agent-message. |
| evals/model-apps/genpage/fixtures/18-sharepoint-connectors-on/workflow-log.md | Updates fixture upload log to include structured escaped --agent-message. |
| evals/model-apps/genpage/fixtures/17-weather-connectors-off/workflow-log.md | Updates fixture upload log to include structured escaped --agent-message. |
| evals/model-apps/genpage/fixtures/15-support-tickets-real/workflow-log.md | Updates fixture upload log to include structured escaped --agent-message. |
| evals/model-apps/genpage/fixtures/13-contact-localization/workflow-log.md | Updates fixture upload log to include structured escaped --agent-message. |
| evals/model-apps/genpage/fixtures/11-recruitment-pages-real/workflow-log.md | Updates multi-upload fixture logs (including re-upload) to include structured escaped --agent-message. |
| evals/model-apps/genpage/fixtures/11-recruitment-multi-page/workflow-log.md | Updates multi-page + PAGEREF re-upload fixture logs to include structured escaped --agent-message. |
| evals/model-apps/genpage/fixtures/1-account-card-gallery/workflow-log.md | Updates fixture upload log to include structured escaped --agent-message. |
| evals/model-apps/genpage/evals.json | Adds the new structured agent-message workflow assertion to the common checks list. |
| evals/model-apps/genpage/EVAL_GUIDE.md | Updates eval guide documentation/counts to include the new agent-message contract requirement. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| const summaryLines = sections[2].split('\\n').filter((line) => line.trim()); | ||
| if (summaryLines.some((line) => !line.trim().startsWith('- '))) { | ||
| return fail(`upload ${index + 1} Summary items must be Markdown bullets beginning with "- "`); | ||
| } |
| ```markdown | ||
| ## Phase 6 — Deploy | ||
| - Command: `pac model genpage upload --app-id <id> --code-file <path> --data-sources '<entities>' --prompt "<full prompt>" --model <model-id> --name "<page name>" --agent-message "<description>" --add-to-sitemap` | ||
| - Command: `pac model genpage upload --app-id <id> --code-file <path> --data-sources '<entities>' --prompt "<full prompt>" --model <model-id> --name "<page name>" --agent-message "# Agent Thoughts\nStep-by-Step Processing\n<bold numbered headings and bulleted implementation details>\n# Summary\n- <summary item>\n# Final Code\n" --add-to-sitemap` |
| --prompt "<Full page description from plan's ## User Requirements>" ` | ||
| --model "<current-model-id>" ` | ||
| --agent-message "Description of what was built and any relevant details" ` | ||
| --agent-message "# Agent Thoughts\nStep-by-Step Processing\n<Bold numbered headings with bulleted requirements, assumptions, major functions, UX decisions, and verification details>\n# Summary\n- <What was built>\n- <Additional relevant outcome>\n# Final Code\n" ` |
| --prompt "<Only the changes in this upload, e.g. 'Add a search box and sort by company name'>" ` | ||
| --model "<current-model-id>" ` | ||
| --agent-message "Description of what was changed in this upload" | ||
| --agent-message "# Agent Thoughts\nStep-by-Step Processing\n<Bold numbered headings with bulleted edit requirements, assumptions, implementation decisions, preserved behavior, and verification details>\n# Summary\n- <What changed in this upload>\n- <Additional relevant outcome>\n# Final Code\n" |
Akshay Maloo (akshaymaloo)
left a comment
There was a problem hiding this comment.
I don't think we need this anymore?
Summary
Agent Thoughts,Summary, andFinal Codesections\nsequences so PAC on Windows receives--agent-messageas a single argumentRuntime dependency
Rendering escaped structured sections requires the GenUX escaped-section support deployed with
MscrmControls.UxAgentControlversion1.0.285. Merge/release this instruction change only when that runtime support is available in the target environments.Validation