Optimize mobile Dataverse metadata execution

.github/workflows/mobile-apps-script-tests.yml

@@ -0,0 +1,40 @@
+# Functional unit tests for the mobile-apps plugin across supported operating
+# systems and Node versions. These scripts spawn Azure CLI shims and manipulate
+# filesystem paths, so cross-platform coverage protects real behavior.
+name: mobile-apps-script-tests
+
+on:
+    pull_request:
+        branches:
+            - main
+        paths:
+            - "plugins/mobile-apps/**"
+            - ".github/workflows/mobile-apps-script-tests.yml"
+
+jobs:
+    test-mobile-apps-scripts:
+        name: test-mobile-apps-scripts (${{ matrix.os }}, node ${{ matrix.node }})
+        runs-on: ${{ matrix.os }}
+        strategy:
+            fail-fast: false
+            matrix:
+                os:
+                    - ubuntu-latest
+                    - windows-latest
+                    - macos-latest
+                node:
+                    - 20
+                    - 22
+        steps:
+            - name: checkout
+              uses: actions/checkout@v4
+
+            - name: setup-node
+              uses: actions/setup-node@v4
+              with:
+                  node-version: ${{ matrix.node }}
+
+            - name: run-mobile-apps-script-tests
+              shell: bash
+              working-directory: plugins/mobile-apps
+              run: node --test scripts/tests/*.test.js

plugins/mobile-apps/agents/native-app-planner.md

@@ -498,13 +498,16 @@ Reject loop = re-spawn `screen-planner` with the user's feedback (layout, screen
 ### Step 5c — Cross-entity Read Audit (Round 2 data-model pass)
 
 **Print before spawning:**
-> "→ Auditing the locked screen plan for cross-entity reads (calc-column candidates from related_entity_fields blocks)…"
+> "→ Auditing the locked screen plan for supported cross-entity read paths…"
 
-**Run condition:** execute this step ONLY after Gate 4b has been approved AND the screen-planner's per-screen specs include at least one `related_entity_fields` block. Skip silently otherwise (no cross-entity reads = no calc-column proposals needed).
+**Run condition:** execute this step ONLY after Gate 4b has been approved AND the screen-planner's per-screen specs include at least one `related_entity_fields` block. Skip silently otherwise.
 
 **Detection (cheap):** before spawning, `Grep` the locked plan for `related_entity_fields:` in `<working_dir>/native-app-plan.md`. Zero matches → skip Step 5c entirely, mark `[x]` and proceed to Step 6. One or more matches → spawn the audit pass below.
 
-This step exists because of the runtime constraint documented at [`shared/references/data-performance.md` § Cross-entity Reads](${PLUGIN_ROOT}/shared/references/data-performance.md#cross-entity-reads) — the SDK has no `$expand`, so cross-entity fields on hot paths (lists, dashboards) MUST be denormalized via calculated columns at the data-model layer. The screen-planner emits `related_entity_fields` per screen; this step turns those into calc-column proposals.
+This step exists because the SDK has no `$expand`. It verifies that every
+cross-entity field uses a formatted lookup or bounded chained fetch, and flags
+hot-path fields that require an externally supplied projection. It never
+synthesizes calculated/formula metadata.
 
 #### 5c.1 — Spawn `data-model-architect` in `cross-entity-audit` mode
 
@@ -530,22 +533,23 @@ Wait for return; apply the Step 3.0 status switch:
 - `DONE_WITH_CONCERNS: <list>` → embed addendum, propagate concerns into your own final `DONE_WITH_CONCERNS:`.
 - `NEEDS_CONTEXT:` / `BLOCKED:` — propagate up per the standard switch.
 
-#### 5c.2 — Gate 1 addendum (calc-column approval)
+#### 5c.2 — Gate 1 addendum (cross-entity read paths)
 
 If 5c.1 wrote a `### Cross-entity Reads` addendum, present it to the user as a Gate 1 addendum (NOT a fresh Gate 1 — the original schema is already approved and unchanged):
 
 ```
 ## Gate 1 — Addendum: Cross-entity Reads
 
-The screen plan you approved at Gate 4b reads N fields from related entities (gate names on inspections, customer phones on orders, etc.). Because the Power Apps SDK has no $expand, those fields need calculated columns on the parent tables to display efficiently — otherwise list screens would either render "—" or trigger N+1 fetches per row.
+The screen plan reads N fields from related entities. The generated SDK has no
+$expand, so each field must use a formatted lookup, a bounded chained fetch, or
+an external server-owned projection.
 
-Proposed calculated columns (auto-derived from your screen plan, no schema reshape):
+Proposed read paths:
 
 [paste the ### Cross-entity Reads table from _dm_section.md]
 
-[paste the Chained-fetch fields (informational) table if present — these need NO schema change, the screen-builder handles them at scaffold time]
-
-Approve to add these calc columns to the data model? (Reject → revise the audit. Approve → /setup-datamodel will create them in Phase 6.1b.)
+Approve these read paths? Any `external-projection-required` row remains a
+blocker until the user supplies that projection outside this workflow.
 ```
 
 Reject loop = re-spawn data-model-architect in `mode: cross-entity-audit` with the user's feedback (e.g. "drop cr3e9_tailnumber_calc, the list doesn't actually show it"). Approve = mark `[x]` Gate 1 addendum approved, proceed to Step 6.

plugins/mobile-apps/agents/screen-builder.md

@@ -116,26 +116,26 @@ You will be invoked by `/create-mobile-app` Step 11 or `/edit-app` screen-rebuil
   ```
 
   For lookup labels, select the real `_<lookup>_value` field in your `$select` and read with `lookupName(record, '<lookupLogicalName>')`. For choice / status / boolean / datetime / money labels, read with `formattedValue(record, '<columnLogicalName>')` or fall back to the generated option const. On bounded lists that use `useSearchFilter(...)`, fields MUST be real string properties from generated types; never add an inferred display-name field just to make search prettier. Cursor lists do not use `useSearchFilter`; they push search into the service `filter` option.
-- **HARD RULE — Cross-entity Field Resolution.** Before writing the screen's `select: [...]` or load step, walk every UI field your spec displays. For each field that sources data from an entity OTHER than the screen's primary fetch target, follow this algorithm exactly. The full reference (with cost-profile rationale, calc-column naming, and pattern examples) is at [`shared/references/data-performance.md` § Cross-entity Reads](${PLUGIN_ROOT}/shared/references/data-performance.md#cross-entity-reads). The screen-builder MUST apply the rule mechanically — do NOT invent your own resolution.
+- **HARD RULE — Cross-entity Field Resolution.** Before writing the screen's `select: [...]` or load step, walk every UI field your spec displays. For each field that sources data from an entity OTHER than the screen's primary fetch target, follow this algorithm exactly. The full supported-path reference is at [`shared/references/data-performance.md` § Cross-entity Reads](${PLUGIN_ROOT}/shared/references/data-performance.md#cross-entity-reads). The screen-builder MUST apply the rule mechanically — do NOT invent your own resolution.
 
-  1. **Calc column check first.** Open `src/generated/models/<PrimaryEntity>Model.ts` and search for a column matching `<prefix>_<field>_calc` (or any `_calc`-suffixed column resolving the field you need). If present, add it to your `select: [...]` and render directly. Done — no chained fetch needed.
-  2. **No calc column? Branch on screen archetype × cardinality** (archetype is in your spec under `**Archetype:**`):
+  1. **Follow the planned recommendation** from `related_entity_fields`.
 
-     | Archetype | Cardinality | Action |
+     | Recommendation | Action |
      |---|---|---|
-     | List (`top ≥ 5`), Tab-root, Dashboard | 1:1 (N:1 lookup chain) | **STOP — do NOT chain a fetch in the list `map()` / `renderItem`.** Emit a `// TODO(cross-entity-read):` header comment naming the field, the related entity, and the recommended `<prefix>_<field>_calc` column to add. Render the cell as `'—'` for now. Then return `BLOCKED [<screen_name>]: list field <field> requires cross-entity read; needs calc column <prefix>_<field>_calc on <primary_entity> — re-run /setup-datamodel (or /add-dataverse for existing apps) to add the calc column.` Do NOT scaffold an N+1 fetch storm. |
-     | Detail (single record) | 1:1 (N:1 lookup chain) | Scaffold a chained `<RelatedService>.get(record._<lookup>_value, { select: [...] })` in the screen's load step. One record on screen = one extra round trip is fine. Display via `lookupName(...)` or direct field read. |
-     | Any | 1:many or M:N | Scaffold a chained `<ChildService>.getAll({ filter: \`_<parentid>_value eq '${id}'\`, select: [...] })`. Calc columns CANNOT traverse 1:many or M:N — chained fetch is the ONLY pattern. |
+     | `formatted-lookup` | Select the real `_<lookup>_value` field and render `lookupName(record, '<lookupLogicalName>')`. |
+     | `chained-fetch` | Perform one bounded related `get` / `getAll` in the screen load step, never inside `map()` or `renderItem`. |
+     | `external-projection-required` | Render no fake fallback data. Return `BLOCKED` and name the field/source. The user must supply a supported server-owned projection outside this workflow. |
 
-  3. **Verify before exit.** Every UI field in your spec must have either (a) a `select` entry on the primary fetch (covered by direct column or calc column) OR (b) a chained fetch path. If a field has neither, return `BLOCKED [<screen_name>]: field <field> on <screen> has no fetch path — add to spec or add calc column`.
+  2. **Verify before exit.** Every UI field in your spec must have either a
+     primary select, formatted lookup annotation, or bounded chained fetch.
+     Otherwise return `BLOCKED [<screen_name>]: field <field> requires an external projection`.
 
-  4. **TODO comment shape** (when emitting at step 2 / list branch):
+  3. **TODO comment shape** for an external projection:
 
      ```ts
      // TODO(cross-entity-read): screen displays <field> from related <entity>.
-     // Re-run /setup-datamodel (or /add-dataverse for existing apps) and add calc
-     // column <prefix>_<field>_calc to <primary_entity> for one-round-trip reads.
-     // List screens MUST NOT chain fetches in renderItem — N+1 storm.
+     // Supply a maker-created formula column or another server-owned projection,
+     // then rerun Dataverse reconciliation. Never chain reads in renderItem.
      ```
 
 - **HARD RULE — server-managed columns are NEVER in a create or update payload.** The Dataverse server owns these fields; including them in a `*Service.create({...})` or `*Service.update({...})` returns HTTP 400 on every save. Generated `create()` types may include server-managed fields (`ownerid`, `statecode`, primary IDs, etc.) because they mirror the full model; do **not** satisfy those types by emitting junk values. For any screen with create/update behavior, use a narrow write helper/type whose input contains only editable fields. If the skeleton imports an app-level helper, call it; otherwise define the helper inside your assigned screen file. Do **not** create or modify shared `src/utils/`, `src/hooks/`, or service files from a screen-builder. Forbidden keys in any create/update payload:
@@ -153,12 +153,17 @@ You will be invoked by `/create-mobile-app` Step 11 or `/edit-app` screen-rebuil
   };
 
   export async function createTask(input: CreateTaskInput): Promise<void> {
-    const payload: Record<string, unknown> = {
+    type CreateFields = Pick<
+      Parameters<typeof Cr3e9_tasksService.create>[0],
+      'cr3e9_name' | 'cr3e9_status' | 'cr3e9_projectid@odata.bind'
+    >;
+    const payload: Omit<CreateFields, 'cr3e9_projectid@odata.bind'>
+      & Partial<Pick<CreateFields, 'cr3e9_projectid@odata.bind'>> = {
       cr3e9_name: input.title,
       cr3e9_status: input.status,
     };
     if (input.projectId) {
-      payload['cr3e9_Project@odata.bind'] = `/cr3e9_projects(${input.projectId})`;
+      payload['cr3e9_projectid@odata.bind'] = `/cr3e9_projects(${input.projectId})`;
     }
 
     const result = await Cr3e9_tasksService.create(payload as Parameters<typeof Cr3e9_tasksService.create>[0]);
@@ -174,24 +179,27 @@ You will be invoked by `/create-mobile-app` Step 11 or `/edit-app` screen-rebuil
   if (!validId) return <MissingRecordIdState />;
   ```
 - **Lookup writes use `@odata.bind`, NEVER raw GUIDs.** When a form creates or updates a record with a parent reference (Task → Project, Comment → Task, Inspection → Site, etc.), the foreign key field is set with the entity-bind syntax. Setting it any other way either silently saves `null` (data loss — form looks like it succeeded) or 400s with a cryptic Dataverse error.
-  - **Required pattern** — use the lookup's **schema name** (PascalCase navigation property), suffix with `@odata.bind`, value is `/<entitySetName>(<guid>)`:
+  - **Required pattern** — open the generated target model and copy the exact quoted property ending in `@odata.bind`; value is `/<entitySetName>(<guid>)`:
     ```ts
     await Cr3e9_tasksService.create({
       cr3e9_name: title,
-      'cr3e9_Project@odata.bind': `/cr3e9_projects(${projectId})`,
+      'cr3e9_projectid@odata.bind': `/cr3e9_projects(${projectId})`,
       cr3e9_status: TaskStatus.Open, // choice = number
     });
     ```
   - **Forbidden patterns:**
     ```ts
     { _cr3e9_project_value: projectId }  // _value props are READ-ONLY; silently dropped on create
     { cr3e9_project: projectId }          // raw GUID on nav property; 400
-    { 'cr3e9_project@odata.bind': projectId }  // missing /entitySet(guid) wrapper; 400
+    { 'cr3e9_projectid@odata.bind': projectId }  // missing /entitySet(guid) wrapper; 400
     ```
   - **Finding the right names:**
-    - **Schema name** (left of `@odata.bind`): the lookup column's PascalCase logical name, usually exposed in the generated model file (`src/generated/models/<Entity>Model.ts`). Often differs from the `_value` read property by case + dropped underscore (read `_cr3e9_project_value`, write `cr3e9_Project@odata.bind`).
+    - **Write property** (left of `@odata.bind`): use the exact key declared in `src/generated/models/<Entity>Model.ts`. It is case-sensitive and may be lowercase even when raw Web API metadata exposes a PascalCase navigation property. Never guess or transform the read `_value` property.
     - **Entity set name** (inside `/(...)`): always the **plural** logical collection name — `cr3e9_projects`, not `cr3e9_project`. Use `pluralName` from the model file or check the generated service filename (`Cr3e9_projectsService.ts` ⇒ entity set is `cr3e9_projects`).
-    - When in doubt, grep `@odata.bind` in `src/generated/services/` for an existing example, or ask the `microsoft-learn` MCP server. Full reference: [`skills/add-dataverse/references/dataverse-reference.md` § Setting Lookups](${PLUGIN_ROOT}/skills/add-dataverse/references/dataverse-reference.md#setting-lookups-creatingupdating-records).
+    - Grep `@odata.bind` in the generated **model**, not the generated service. If the key is absent, return `BLOCKED` rather than inventing it.
+  - **Typed payload rule:** do not start write payloads as `Record<string, unknown>`. Define a `Pick<Parameters<typeof Service.create|update>[N], ...>` containing the exact writable keys, then use a single boundary cast only when generated base types incorrectly require server-managed fields. This makes misspelled or wrongly-cased lookup keys fail TypeScript.
+  - **Lookup filter rule:** generated mobile services do not reliably support raw Web API relationship traversal such as `lookupNav/relatedColumn eq ...`. Query the related table service first, then filter the source table using its read lookup GUID property (`_lookuplogicalname_value eq <guid>`).
+- **Authenticated profile rule:** when the plan links a profile table to `systemuser`, import the generated `SystemusersService`. Resolve token `oid → systemuserid` by filtering `azureactivedirectoryobjectid`, reject disabled/missing/duplicate users, then filter the profile with `_systemuserlookup_value eq <systemuserid>`. If `SystemusersService` is absent from the Generated Services snapshot, return `BLOCKED`; do not replace it with email matching or relationship traversal.
 
   - **Form picker UI** — when the form's spec calls for a parent picker (e.g., "select Project"), the picker stores the selected record's `id` (GUID string), and the submit handler converts it to the bind string at write time. Never store the bind string in component state — only in the API payload.
 - **Pagination rule:** If your spec says `pagination: cursor`, do NOT use `useListData` or `useSearchFilter`. Use the skeleton's `useCursorListData` call, React Query's `useInfiniteQuery`, or an app-specific `use<Entity>CursorList` hook with FlatList `onEndReached` per the pattern in [`data-performance.md`](${PLUGIN_ROOT}/shared/references/data-performance.md). Never fetch all records at once, and never treat `top: 50` as pagination. Real generated Dataverse services use SDK `maxPageSize` for page size and return `IOperationResult.skipToken` for the next page; pass that value back as `skipToken`. Always include deterministic `orderBy` with a unique key and `select` in the service call. Push search/filter into Dataverse with `filter`. If the generated service in the app does not expose `maxPageSize`/`skipToken` for an unbounded table, return `BLOCKED [<screen_name>]: generated service does not expose cursor paging for <Service>; do not downgrade to useListData`.