diff --git a/.github/workflows/mobile-apps-script-tests.yml b/.github/workflows/mobile-apps-script-tests.yml new file mode 100644 index 000000000..0d4ca1b03 --- /dev/null +++ b/.github/workflows/mobile-apps-script-tests.yml @@ -0,0 +1,40 @@ +# Functional unit tests for the mobile-apps plugin across supported operating +# systems and Node versions. These scripts spawn Azure CLI shims and manipulate +# filesystem paths, so cross-platform coverage protects real behavior. +name: mobile-apps-script-tests + +on: + pull_request: + branches: + - main + paths: + - "plugins/mobile-apps/**" + - ".github/workflows/mobile-apps-script-tests.yml" + +jobs: + test-mobile-apps-scripts: + name: test-mobile-apps-scripts (${{ matrix.os }}, node ${{ matrix.node }}) + runs-on: ${{ matrix.os }} + strategy: + fail-fast: false + matrix: + os: + - ubuntu-latest + - windows-latest + - macos-latest + node: + - 20 + - 22 + steps: + - name: checkout + uses: actions/checkout@v4 + + - name: setup-node + uses: actions/setup-node@v4 + with: + node-version: ${{ matrix.node }} + + - name: run-mobile-apps-script-tests + shell: bash + working-directory: plugins/mobile-apps + run: node --test scripts/tests/*.test.js diff --git a/plugins/mobile-apps/agents/data-model-architect.md b/plugins/mobile-apps/agents/data-model-architect.md index 2fad8b3c3..d7fe8146f 100644 --- a/plugins/mobile-apps/agents/data-model-architect.md +++ b/plugins/mobile-apps/agents/data-model-architect.md @@ -29,7 +29,9 @@ You will be invoked by `native-app-planner` or `/edit-app` with a prompt that in - **Read-only.** You MUST NOT run `npx power-apps add-data-source --api-id dataverse --org-url --resource-name `, table-creation HTTP calls, or any mutating PowerShell. Mutation happens later in `/add-dataverse` after user approval. - **Power Apps CLI failure refresh.** Follow [shared-instructions.md](../shared/shared-instructions.md) command-failure handling for any failed `npx power-apps *` command; retry the original command once after auth is corrected. -- **Reuse-first.** Always query existing tables and prefer reuse > extension > new. Don't propose a `cr123_customer` table if a standard `contact` table fits. +- **Reuse-first and target-grounded.** Query the exact target metadata for every proposed table, including standard tables, and prefer reuse > extension > new. Don't propose a `cr123_customer` table if a verified target `contact` table fits. +- **Never invent existing schema.** Never propose recreating or imitating a missing standard, managed, or solution-owned table/column. If discovery cannot run, you may still draft a plan from requirements, but mark it `Discovery skipped` so the user and `/add-dataverse` treat every decision as unverified — `/add-dataverse` re-reconciles against live metadata and adapts or defers anything that conflicts. +- **No automatic replacement.** This agent classifies schema as `Reuse`, `Extend`, `Create`, `Adapt` (create beside a conflicting object under a new name), `Defer` (leave out of this run), or `Unverified` (target metadata could not be read). Replacing an existing table/column requires a separately approved migration with dependency analysis and data movement; it is outside this workflow. A data-modelling conflict is never a blocker — it is an `Adapt` or a `Defer` with a recorded reason. - **Return a section, not a separate doc.** Output is a markdown `## Data Model` section the planner embeds verbatim. - **No JSON request bodies in the output.** Your `_dm_section.md` describes *what* to create (tables, columns, relationships) using the Mermaid ER + reuse/extend/create table + tier list. **Do NOT include POST body JSON** for `EntityDefinitions` or `RelationshipDefinitions` — `/add-dataverse` constructs those from its own canonical templates in [skills/add-dataverse/SKILL.md](../skills/add-dataverse/SKILL.md) Step 5b. JSON in your output is read as authoritative and will leak invented/wrong fields (e.g. `ReferencingAttribute` on a lookup) into the actual POST. - **No questions.** Do not ask the user anything — infer from the requirements provided. The planner runs the approval gate, not you. @@ -41,7 +43,7 @@ You will be invoked by `native-app-planner` or `/edit-app` with a prompt that in 2. Verify Dataverse access 3. Discover existing tables 4. Infer required entities from requirements -5. Score reuse / extend / create +5. Reconcile target metadata and score reuse / extend / create / block 6. Build dependency tiers 6a. Cross-entity Read Audit (when `_screens_section.md` exists OR `mode: cross-entity-audit`) 7. Produce the `## Data Model` section @@ -69,7 +71,7 @@ node "${PLUGIN_ROOT}/scripts/resolve-environment.js" Capture the **Environment URL** (e.g., `https://orgXXXXX.crm.dynamics.com`), **Environment ID**, and **Tenant ID** from the output. Use the URL as `` for subsequent script calls. -If resolution fails (not authenticated or environment not visible to the logged-in account), include a clear note in your output and stop further discovery — propose the data model from requirements only with a "Discovery skipped — environment not reachable" warning prepended to your section. +If resolution fails (not authenticated or environment not visible to the logged-in account), do not stop the run. Skip further discovery, prepend a `Discovery skipped — environment not reachable` warning to your section, and finish with `DONE_WITH_CONCERNS`. The plan is a draft for the user's Gate 1 review; `/add-dataverse` re-queries live metadata and blocks any mutation it cannot verify. ## Step 2 — Verify Dataverse Access @@ -79,18 +81,18 @@ If resolution fails (not authenticated or environment not visible to the logged- node "${PLUGIN_ROOT}/scripts/verify-dataverse-access.js" ``` -If it fails, prepend a "Dataverse access failed — `az login` required" note and skip Steps 3. +If it fails, skip Step 3 and Step 5's live queries, prepend a `Dataverse access failed — az login required` warning to your section, and finish with `DONE_WITH_CONCERNS`. Do not convert unverified guesses into confident decisions. ## Step 3 — Discover Existing Tables **Print before starting:** > "→ Discovering existing custom tables in the environment (cap: top 10 by relevance)…" -Query custom tables only (standard tables are well-known): +Query custom tables to discover conceptual reuse candidates. This broad query is advisory only; Step 5 still queries every selected custom, standard, and managed table by exact logical name before classifying it: ```bash node "${PLUGIN_ROOT}/scripts/dataverse-request.js" GET \ - "EntityDefinitions?\$select=LogicalName,DisplayName,Description&\$filter=IsCustomEntity eq true" + "EntityDefinitions?\$select=MetadataId,LogicalName,DisplayName,Description,IsCustomEntity,IsManaged,IsCustomizable,CanCreateAttributes&\$filter=IsCustomEntity eq true" ``` For the relevant tables, fetch their user-defined columns in a single call (system columns like `createdon`, `modifiedby`, `statecode`, `ownerid`, `versionnumber` are filtered out automatically): @@ -124,49 +126,83 @@ Standard table mappings to bias toward: | An activity event | `appointment`, `task`, `phonecall`, `email` | | A user / system identity | `systemuser` (read-only — never propose extending) | -## Step 5 — Score Reuse / Extend / Create +For every `Reuse` decision, add `Service required: yes|no`. Use `yes` whenever any screen, hook, role check, lookup picker, related-field fetch, or authenticated-identity flow reads the table. `systemuser` identity resolution is always `Service required: yes`; read-only means no schema mutation, not no generated data source. + +## Step 5 — Reconcile Target and Score Reuse / Extend / Create / Adapt / Defer / Unverified **Print before starting:** -> "→ Scoring each required entity as Reuse / Extend / Create against discovered tables…" +> "→ Reconciling every required table and column against live target metadata…" + +Before assigning any decision, resolve every required entity — including `contact`, `account`, `incident`, other standard tables, and managed-solution dependencies — in a **single** filtered query that also expands their columns: + +```bash +node "${PLUGIN_ROOT}/scripts/dataverse-request.js" GET \ + "EntityDefinitions?\$select=MetadataId,LogicalName,SchemaName,IsCustomEntity,IsManaged,IsCustomizable,CanCreateAttributes,PrimaryIdAttribute,PrimaryNameAttribute&\$filter=LogicalName eq '' or LogicalName eq ''&\$expand=Attributes(\$select=LogicalName,AttributeType,AttributeTypeName,RequiredLevel,IsManaged,IsCustomizable,IsPrimaryId,IsPrimaryName)" +``` + +Build the `$filter` by OR-ing every selected logical name. This is the [documented way to query multiple table definitions at once](https://learn.microsoft.com/power-apps/developer/data-platform/query-schema-definitions#basic-retrievemetadatachanges-example) and replaces 2N requests with one. Keep the expanded `$select` to base `AttributeMetadata` properties — one query [cannot cast to a derived column type](https://learn.microsoft.com/power-apps/developer/data-platform/query-schema-definitions#evaluate-other-options-to-retrieve-schema-definitions). + +A planned name **present** in `value[]` exists; a name **absent** from `value[]` does not. Interpret `IsCustomizable` and `CanCreateAttributes` as managed properties (`.Value`). Absence is actionable only after considering the planned dependency kind: an absent new custom table may be created; an absent standard, managed, reused, or extended dependency is `Defer` — it must be installed/imported or removed from the design, so leave it out of this run and record why. If the batched query itself fails, mark the affected entities `Unverified` rather than `Create`, and carry the reason into the section — `/add-dataverse` re-checks it at its own reconciliation step. For each required entity, classify it as one of: - **Reuse** — existing table fits as-is (no schema changes needed) -- **Extend** — existing table is the right concept but missing some columns; add only the missing ones -- **Create** — no existing table serves this purpose at all (neither by name nor by concept) +- **Extend** — existing table is the right concept, all same-name columns are compatible, missing columns are custom additions, and both `IsCustomizable.Value` and `CanCreateAttributes.Value` permit extension +- **Create** — the planned item is explicitly a new custom table, the exact logical name returns 404, the publisher prefix is verified, and all required-existing dependencies are present +- **Adapt** — a same-name custom table/column is incompatible or the intended custom table cannot be safely extended, so create an app-owned alternative under a new verified logical name and record the alias +- **Defer** — a standard, managed, solution-owned, or required-existing dependency is missing/incompatible, or no safe app-owned alternative can preserve the intended semantics; leave it out of this run and record the prerequisite +- **Unverified** — discovery could not run for this entity (Step 1/2 failure or a non-200/404 response). Record the intended decision plus the reason; `/add-dataverse` re-checks it before any write + +Classify every planned column before finalizing its table decision: + +- **Reuse** — same logical name exists with a compatible `AttributeType` / `AttributeTypeName.Value`. +- **Create** — column is absent, is explicitly a custom addition, and the target table permits attributes. The containing table becomes `Extend`, or the column is included inline when its table is `Create`. +- **Adapt** — a same-name custom column is incompatible and the table permits a new custom column under a verified logical name; record the old-to-new alias. +- **Defer** — a required standard/managed column is absent or incompatible, the target cannot be customized, or a renamed custom column would change the intended semantics. +- **Unverified** — the live column metadata could not be read reliably; preserve the intended decision for `/add-dataverse` to re-check. + +`Extend` is a table decision, not a column operation. Do not classify any item as `Replace`; Dataverse cannot change column types in place, and replacement needs an explicit migration outside this workflow. **Decision priority (HARD — apply in order, stop at first match):** -1. **Standard table match** → always prefer a standard table (`contact`, `account`, `incident`, etc.) over creating a custom table for the same concept. Reuse if it fits; Extend if it needs columns. +1. **Standard table match** → always prefer a standard table (`contact`, `account`, `incident`, etc.) over creating a custom table for the same concept, but verify it by exact target GET. Reuse if it fits; Extend only when live managed properties permit the planned custom columns. If it is missing or incompatible, Defer — never create a custom imitation. 2. **Existing custom table by name** → if the proposed logical name already exists in the Step 3 results, it MUST be Reuse or Extend. See collision check below. 3. **Existing custom table by concept** → if a different-named existing table serves the same business purpose (e.g., an existing `cr8142a_site` table for a new "Inspection Site" entity), prefer Extend over Create. -4. **Create** → only when no existing table — standard or custom — serves the entity's purpose. The business use case genuinely requires a fresh schema. +4. **Extension-cost guardrail** → reuse remains preferred when the existing record is authoritative/shared, but do not extend a merely similar custom table by default when the app would add more than 8 columns or more than 50% of the final required schema. Prefer a new app-owned table unless shared identity, integrations, security, or reporting make the existing table the true system of record. Record the metadata-write tradeoff and rationale in `Why`. +5. **Create** → only when no existing table — standard or custom — serves the entity's purpose, the proposed custom logical name is absent in the target, and every required-existing dependency is verified. +6. **Adapt / Defer / Unverified** → incompatible custom schema becomes Adapt when a safe app-owned alias preserves semantics; missing/incompatible standard, managed, or required-existing schema becomes Defer; unavailable target facts become Unverified. Never reinterpret any of these states as Create under the conflicting logical name. > **⚠️ Plan-time collision check (HARD).** Before classifying any entity as `Create`, look up its **proposed logical name** (e.g. `cr8142a_inspection`) in the Step 3 IsCustomEntity result. If a row with that exact `LogicalName` already exists, the entity **CANNOT** be classified as `Create`. Apply the following decision tree in order: > > 1. **Downgrade to Reuse** — the existing table's columns from Step 3 already cover what the plan needs (≥70% column overlap or all required columns present). No schema changes. -> 2. **Downgrade to Extend** — the existing table is the right concept but missing some columns (any overlap, or same entity type). Add only the missing columns; never remove or rename existing ones. -> 3. **Rename and Create** — use ONLY when the existing table is a completely different entity concept (e.g., `cr8142a_inspection` exists but contains payroll or product catalog data — fundamentally incompatible). Bump the proposed name to `_v2` and document the rename in the Notes column. +> 2. **Downgrade to Extend** — the existing table is the right concept but missing some custom columns (any overlap, or same entity type), and live `IsCustomizable.Value` plus `CanCreateAttributes.Value` permit extension. Add only the missing columns; never remove or rename existing ones. +> 3. **Adapt (rename and create)** — use ONLY when the existing custom table is a completely different entity concept (e.g., `cr8142a_inspection` exists but contains payroll or product catalog data — fundamentally incompatible). Bump the proposed name to `_v2`, record the alias, and document the evidence in the Notes column. > -> **Default is Reuse or Extend — not Rename.** Rename-and-Create is the exceptional path, not the fallback. If unsure whether a schema is compatible, prefer Extend and add the missing columns — it is always safer to extend than to duplicate tables. +> **Default is Reuse or Extend only when compatibility is proven.** Adapt is the exceptional path, not the fallback. If live metadata could not be read, mark Unverified. If metadata is available but compatibility or customizability cannot be established safely, Defer and record what evidence or prerequisite is missing; never extend merely to keep the workflow moving. > > Surfacing the collision at PLAN time (not at create time) prevents the user from approving Gate 1 with a name that will explode at Step 5a of `/add-dataverse`. Build a table: ```markdown -| Required entity | Decision | Existing match | Why | Missing columns | -|---|---|---|---|---| -| Customer profile | Reuse | `contact` | Standard table; name/email/phone fields match | — | -| Job site | Create | — | No matching custom or standard table for this concept | n/a | -| Inspection report | Extend | `cr123_inspection` (existing) | Same concept; has site reference, missing photos field | `cr123_photourl` (Image) | -| Equipment inspection | Extend | `cr3e9_inspection` (existing) | Name match; different FK schema but same inspection concept — add equipment-specific columns | `cr3e9_equipmentid` (Lookup), `cr3e9_equipmenttype` (Choice) | -| Payroll record | Create (renamed from cr3e9_inspection) | `cr3e9_inspection` exists | Existing table is inspection data — fundamentally different concept; using `cr3e9_payrollrecord` | n/a | +| Required entity | Decision | Existing match | Target evidence | Column decisions | Why | +|---|---|---|---|---|---| +| Customer profile | Reuse | `contact` | 200; exact standard table verified | Reuse: fullname, emailaddress1 | Standard table and required columns exist | +| Job site | Create | — | 404; verified custom prefix | Create inline: cr123_name, cr123_address | No matching custom or standard table for this concept | +| Inspection report | Extend | `cr123_inspection` | 200; customizable + can create attributes | Reuse: cr123_name; Create: cr123_photo | Same concept; one custom column is missing | +| Required managed asset | Defer | — | 404 for required-existing dependency | Defer: all dependent fields | Install/import the owning solution; never recreate it. Left out of this run, not a blocker | ``` ## Step 6 — Build Dependency Tiers -Order new tables so foreign keys can resolve. From [data-architecture-reference.md](${PLUGIN_ROOT}/skills/add-dataverse/references/data-architecture-reference.md): +Order new tables so foreign keys can resolve. See [dataverse-reference.md § Pre-flight ordering](${PLUGIN_ROOT}/skills/add-dataverse/references/dataverse-reference.md#pre-flight-ordering): + +Before assigning tiers, remove redundant relationships. Dataverse already +provides `ownerid`, `createdby`, `modifiedby`, `createdon`, and `modifiedon` on +user-owned tables. Do not create app-prefixed lookups that duplicate those +standard ownership/audit relationships unless the business meaning is distinct +(for example, `approvedby` or `assignedinspector`). Reuse the standard columns +and document their role instead. - **Tier 0** — reference tables (no lookups out) - **Tier 1** — primary entities (lookups to Tier 0) @@ -176,13 +212,16 @@ Order new tables so foreign keys can resolve. From [data-architecture-reference. ## Step 6a — Cross-entity Read Audit **Print before starting:** -> "→ Auditing planned screens for cross-entity reads (calc-column candidates)…" +> "→ Auditing planned screens for supported cross-entity read paths…" **Run condition:** execute this step when EITHER (a) `/_screens_section.md` exists at this point in the workflow OR (b) you were invoked with `mode: cross-entity-audit`. **Skip silently otherwise** (default-mode first-pass run, before screen-planner has produced its section) — the orchestrator will re-spawn you in `mode: cross-entity-audit` after Gate 4a/4b lands. When `mode: cross-entity-audit`, the orchestrator's prompt also includes the path to the existing `_dm_section.md` so you can append (do NOT regenerate it from scratch — Steps 1–6 are skipped in this mode). -This step exists because of the runtime constraint documented at [`shared/references/data-performance.md` § Cross-entity Reads](${PLUGIN_ROOT}/shared/references/data-performance.md#cross-entity-reads) — the SDK has no `$expand`, so cross-entity fields on hot paths (lists, dashboards) MUST be denormalized via calculated columns at the data-model layer. This step proposes those calc columns based on the screen plan; `/setup-datamodel` (or `/add-dataverse`) Phase 6.1b creates them. +This step exists because the generated SDK has no `$expand`. It classifies each +cross-entity field into a supported formatted lookup or bounded chained fetch. +Dataverse does not support defining calculated/formula expressions through +code, so this audit never proposes generated formula metadata. **Algorithm:** @@ -190,36 +229,28 @@ This step exists because of the runtime constraint documented at [`shared/refere 2. **Per entry, branch on `recommends`:** - - **`recommends: calc-column`** — confirm the `cardinality` is `1:1` (calc columns CANNOT traverse 1:many or M:N). If cardinality is wrong, downgrade silently to `chained-fetch` and add a `DONE_WITH_CONCERNS` note. Otherwise, propose a calculated column on the **primary entity of that screen** (the entity its primary `Data` service queries): - - **Logical name:** `__calc` (lowercased, e.g. `cr3e9_gatename_calc`) - - **Schema name:** PascalCase variant (e.g. `Cr3e9_GateName_calc`) - - **Display name:** human label from the planner's `field` value (e.g. "Gate name") - - **Type:** matches the resolved field's TypeScript type → Dataverse type (`string` → `Edm.String`, `datetime` → `Edm.DateTimeOffset`, `decimal` / `money` → `Edm.Decimal`, `integer` → `Edm.Int32`, `boolean` → `Edm.Boolean`) - - **Formula source:** the dotted path from the planner's `source` field, normalized — e.g. `cr3e9_flightid → cr3e9_gateid → cr3e9_gatename` becomes `cr3e9_flightid.cr3e9_gateid.cr3e9_gatename`. The formula is N:1 lookup chain only; no aggregations, no conditionals, no string concat in v0. + - **`recommends: formatted-lookup`** — verify the source is the primary + display name of a direct lookup. Create no column. + - **`recommends: chained-fetch`** — create no column. The screen-builder + performs one bounded related request outside row rendering. + - **`recommends: external-projection-required`** — record a blocker for a hot + list/dashboard field that cannot use the direct lookup annotation. Omit + the field until the user supplies a maker-created formula column or other + server-owned projection. - - **`recommends: chained-fetch`** — do NOT add any column. The screen-builder handles this at scaffold time per the decision table in `data-performance.md`. Just include the entry in the addendum's `Chained-fetch fields (informational)` row so the user sees what the screen-builder will scaffold. - -3. **De-duplicate.** A field driven by N screens (e.g. "Gate name" used on home, list, AND detail) collapses to ONE calc-column row in the addendum. Track all driving screens in the `Driven by` column. - -4. **Cap at 20 calc columns per parent entity.** If you exceed, truncate and add a `DONE_WITH_CONCERNS` note — large calc-column counts indicate a denormalization problem that should be solved at the data-model level (probably an extracted entity), not by piling on calc columns. +3. **De-duplicate.** Collapse identical source/resolution pairs and track all + consuming screens. 5. **Emit the addendum.** Write the `### Cross-entity Reads (auto-derived from screen plan)` subsection of `_dm_section.md`. Schema: ```markdown ### Cross-entity Reads (auto-derived from screen plan) - | Calc column | On table | Type | Resolves | Driven by | - |---|---|---|---|---| - | cr3e9_flightnumber_calc | cr3e9_inspection | string | cr3e9_flightid.cr3e9_flightnumber | inspections list | - | cr3e9_gatename_calc | cr3e9_inspection | string | cr3e9_flightid.cr3e9_gateid.cr3e9_gatename | home, inspections list | - | cr3e9_tailnumber_calc | cr3e9_inspection | string | cr3e9_flightid.cr3e9_aircraftid.cr3e9_tailnumber | inspections list | - - **Chained-fetch fields (informational — screen-builder will scaffold these, no schema changes):** - - | Field | On screen | Cardinality | Source | + | Field | Resolution | Source | Driven by | |---|---|---|---| - | Defect count | inspection detail | 1:many | cr3e9_inspectionzoneid → cr3e9_defect | - | Inspector email | inspection detail | 1:1 | _ownerid_value → systemuser.internalemailaddress | + | Flight | formatted-lookup | cr3e9_flightid primary display | inspections list | + | Inspector email | chained-fetch | _ownerid_value → systemuser.internalemailaddress | inspection detail | + | Gate code | external-projection-required | cr3e9_flightid → cr3e9_gateid → cr3e9_code | home | ``` In `mode: default` (Step 6a runs because `_screens_section.md` was found), append this subsection to the Step 7 output. In `mode: cross-entity-audit`, append it directly to the existing `_dm_section.md` (read it, append the subsection AFTER `### Notes` if present, otherwise at the end, then write back) and skip Step 7 entirely — return immediately. @@ -252,12 +283,15 @@ Write the section to a file in the working directory named `_dm_section.md` (the - Reuse: existing tables - Extend: tables (add columns only) - Create: new tables across tiers +- Adapt: app-owned aliases created beside incompatible custom schema +- Defer: missing/incompatible dependencies left out of this run +- Unverified: tables discovery could not confirm (re-checked by `/add-dataverse`) -### Reuse / Extend / Create +### Target Reconciliation -| Required entity | Decision | Match | Why | Missing columns | -|---|---|---|---|---| -| ... | ... | ... | ... | ... | +| Required entity | Decision | Existing match | Target evidence | Column decisions | Why | +|---|---|---|---|---|---| +| ... | ... | ... | ... | ... | ... | ### ER Diagram @@ -291,7 +325,7 @@ erDiagram - Signature images from pen input normalize `data:image/png;base64,...` before Image column writes. ``` -If discovery was skipped (Step 1 or 2 failure), prepend the appropriate warning to the section, omit the "Reuse" column from the table, and fill all decisions as "Create" with a note that the user should re-run with environment access for accurate reuse detection. +If any row is `Adapt` or `Defer`, write the evidence and reason into the section and finish with `DONE_WITH_CONCERNS` naming each one, so the user sees it at Gate 1 and can revise the design before `/add-dataverse` runs. Never return `BLOCKED` for a data-modelling conflict — that status is reserved for hard walls such as an unwritable working directory. If discovery was skipped (Step 1 or Step 2 failure), prepend the matching warning, mark every decision `Unverified`, and say the user should re-run with environment access for accurate reuse detection. ## Return Status @@ -300,7 +334,7 @@ You MUST return your final message with one of these four status codes as the ** | Code | When to use | Example first line | |---|---|---| | `DONE` | Section written cleanly, all entities resolved, no caveats | `DONE` | -| `DONE_WITH_CONCERNS: ` | Section written but you fell back from a planned reuse, guessed a column type, or skipped Dataverse discovery — the user should review before approving Gate 1 | `DONE_WITH_CONCERNS: contact reuse skipped (env access denied), all tables marked Create` | +| `DONE_WITH_CONCERNS: ` | Section written, but discovery was skipped, an entity is `Adapt`/`Defer`/`Unverified`, or a design caveat remains | `DONE_WITH_CONCERNS: contact reuse skipped (env access denied), all decisions unverified` | | `NEEDS_CONTEXT: ` | Cannot complete without more info from the orchestrator — e.g. requirements brief is too thin to infer entities, or no environment was selected | `NEEDS_CONTEXT: requirements brief lists no nouns; need explicit entity list from user` | | `BLOCKED: ` | Hit a hard wall — file system error writing `_dm_section.md`, plugin root unreadable, environment resolver crashed. The planner MUST escalate to the user, never silently retry | `BLOCKED: cannot write to /_dm_section.md (permission denied)` | @@ -313,6 +347,6 @@ You MUST return your final message with one of these four status codes as the ** After the status line and a blank line, write: -> Data Model section written to `/_dm_section.md`. Summary: . ER diagram includes . +> Data Model section written to `/_dm_section.md`. Summary: . ER diagram includes . The planner reads the file and embeds the contents verbatim into `native-app-plan.md`. diff --git a/plugins/mobile-apps/agents/native-app-planner.md b/plugins/mobile-apps/agents/native-app-planner.md index 94ca48083..009da1e8a 100644 --- a/plugins/mobile-apps/agents/native-app-planner.md +++ b/plugins/mobile-apps/agents/native-app-planner.md @@ -498,13 +498,16 @@ Reject loop = re-spawn `screen-planner` with the user's feedback (layout, screen ### Step 5c — Cross-entity Read Audit (Round 2 data-model pass) **Print before spawning:** -> "→ Auditing the locked screen plan for cross-entity reads (calc-column candidates from related_entity_fields blocks)…" +> "→ Auditing the locked screen plan for supported cross-entity read paths…" -**Run condition:** execute this step ONLY after Gate 4b has been approved AND the screen-planner's per-screen specs include at least one `related_entity_fields` block. Skip silently otherwise (no cross-entity reads = no calc-column proposals needed). +**Run condition:** execute this step ONLY after Gate 4b has been approved AND the screen-planner's per-screen specs include at least one `related_entity_fields` block. Skip silently otherwise. **Detection (cheap):** before spawning, `Grep` the locked plan for `related_entity_fields:` in `/native-app-plan.md`. Zero matches → skip Step 5c entirely, mark `[x]` and proceed to Step 6. One or more matches → spawn the audit pass below. -This step exists because of the runtime constraint documented at [`shared/references/data-performance.md` § Cross-entity Reads](${PLUGIN_ROOT}/shared/references/data-performance.md#cross-entity-reads) — the SDK has no `$expand`, so cross-entity fields on hot paths (lists, dashboards) MUST be denormalized via calculated columns at the data-model layer. The screen-planner emits `related_entity_fields` per screen; this step turns those into calc-column proposals. +This step exists because the SDK has no `$expand`. It verifies that every +cross-entity field uses a formatted lookup or bounded chained fetch, and flags +hot-path fields that require an externally supplied projection. It never +synthesizes calculated/formula metadata. #### 5c.1 — Spawn `data-model-architect` in `cross-entity-audit` mode @@ -530,22 +533,23 @@ Wait for return; apply the Step 3.0 status switch: - `DONE_WITH_CONCERNS: ` → embed addendum, propagate concerns into your own final `DONE_WITH_CONCERNS:`. - `NEEDS_CONTEXT:` / `BLOCKED:` — propagate up per the standard switch. -#### 5c.2 — Gate 1 addendum (calc-column approval) +#### 5c.2 — Gate 1 addendum (cross-entity read paths) If 5c.1 wrote a `### Cross-entity Reads` addendum, present it to the user as a Gate 1 addendum (NOT a fresh Gate 1 — the original schema is already approved and unchanged): ``` ## Gate 1 — Addendum: Cross-entity Reads -The screen plan you approved at Gate 4b reads N fields from related entities (gate names on inspections, customer phones on orders, etc.). Because the Power Apps SDK has no $expand, those fields need calculated columns on the parent tables to display efficiently — otherwise list screens would either render "—" or trigger N+1 fetches per row. +The screen plan reads N fields from related entities. The generated SDK has no +$expand, so each field must use a formatted lookup, a bounded chained fetch, or +an external server-owned projection. -Proposed calculated columns (auto-derived from your screen plan, no schema reshape): +Proposed read paths: [paste the ### Cross-entity Reads table from _dm_section.md] -[paste the Chained-fetch fields (informational) table if present — these need NO schema change, the screen-builder handles them at scaffold time] - -Approve to add these calc columns to the data model? (Reject → revise the audit. Approve → /setup-datamodel will create them in Phase 6.1b.) +Approve these read paths? Any `external-projection-required` row remains a +blocker until the user supplies that projection outside this workflow. ``` Reject loop = re-spawn data-model-architect in `mode: cross-entity-audit` with the user's feedback (e.g. "drop cr3e9_tailnumber_calc, the list doesn't actually show it"). Approve = mark `[x]` Gate 1 addendum approved, proceed to Step 6. diff --git a/plugins/mobile-apps/agents/screen-builder.md b/plugins/mobile-apps/agents/screen-builder.md index 476d60d52..6bc996f84 100644 --- a/plugins/mobile-apps/agents/screen-builder.md +++ b/plugins/mobile-apps/agents/screen-builder.md @@ -32,7 +32,7 @@ You will be invoked by `/create-mobile-app` Step 11 or `/edit-app` screen-rebuil - **Use shared code via path aliases — NEVER re-define inline.** The project has `@/components`, `@/hooks`, `@/utils`, `@/tokens` configured in tsconfig. Import from them: - **Components:** `import { LoadingState, ErrorState, EmptyState, ScreenHeader, ModalHeader, BottomActionBar, FloatingActionButton, FilterChipRow, FormField, RowPick, StatusPill, AvatarInitials, InfoRow, ActionRow, SectionHeader } from '@/components'` - **Hooks:** `import { useListData, useCursorListData, useSearchFilter } from '@/hooks'` — use `useListData` only for bounded list screens whose spec says `pagination: none`. For unbounded Dataverse screens whose spec says `pagination: cursor`, use `useCursorListData`, `useInfiniteQuery`, or an app-specific cursor hook generated by the orchestrator. Use `useSearchFilter` only for bounded client-side lists; cursor lists must push search into the service call with `filter`. - - **Utils:** `import { formatDate, formatDateTime, formatRelative, truncate, pluralize, choiceLabel, STATUS_TONES } from '@/utils'` + - **Utils:** `import { formatDate, formatDateTime, formatRelative, truncate, pluralize, choiceLabel, STATUS_TONES } from '@/utils'`. Dynamic Dataverse routes additionally import `normalizeDataverseGuid`. - **Generated:** `import { FooService } from '@/generated/services/FooService'` and `import type { Foo } from '@/generated/models/FooModel'` - **Native:** `import { captureFromCamera } from '@/native/camera'` Do NOT define `function LoadingState()`, `function formatDate()`, `function Field()`, `function Section()`, or status color maps inside your screen. Do NOT write the `useState(loading) + useFocusEffect(load) + onRefresh` pattern manually — use `useListData` instead. @@ -116,26 +116,26 @@ You will be invoked by `/create-mobile-app` Step 11 or `/edit-app` screen-rebuil ``` For lookup labels, select the real `__value` field in your `$select` and read with `lookupName(record, '')`. For choice / status / boolean / datetime / money labels, read with `formattedValue(record, '')` or fall back to the generated option const. On bounded lists that use `useSearchFilter(...)`, fields MUST be real string properties from generated types; never add an inferred display-name field just to make search prettier. Cursor lists do not use `useSearchFilter`; they push search into the service `filter` option. -- **HARD RULE — Cross-entity Field Resolution.** Before writing the screen's `select: [...]` or load step, walk every UI field your spec displays. For each field that sources data from an entity OTHER than the screen's primary fetch target, follow this algorithm exactly. The full reference (with cost-profile rationale, calc-column naming, and pattern examples) is at [`shared/references/data-performance.md` § Cross-entity Reads](${PLUGIN_ROOT}/shared/references/data-performance.md#cross-entity-reads). The screen-builder MUST apply the rule mechanically — do NOT invent your own resolution. +- **HARD RULE — Cross-entity Field Resolution.** Before writing the screen's `select: [...]` or load step, walk every UI field your spec displays. For each field that sources data from an entity OTHER than the screen's primary fetch target, follow this algorithm exactly. The full supported-path reference is at [`shared/references/data-performance.md` § Cross-entity Reads](${PLUGIN_ROOT}/shared/references/data-performance.md#cross-entity-reads). The screen-builder MUST apply the rule mechanically — do NOT invent your own resolution. - 1. **Calc column check first.** Open `src/generated/models/Model.ts` and search for a column matching `__calc` (or any `_calc`-suffixed column resolving the field you need). If present, add it to your `select: [...]` and render directly. Done — no chained fetch needed. - 2. **No calc column? Branch on screen archetype × cardinality** (archetype is in your spec under `**Archetype:**`): + 1. **Follow the planned recommendation** from `related_entity_fields`. - | Archetype | Cardinality | Action | + | Recommendation | Action | |---|---|---| - | List (`top ≥ 5`), Tab-root, Dashboard | 1:1 (N:1 lookup chain) | **STOP — do NOT chain a fetch in the list `map()` / `renderItem`.** Emit a `// TODO(cross-entity-read):` header comment naming the field, the related entity, and the recommended `__calc` column to add. Render the cell as `'—'` for now. Then return `BLOCKED []: list field requires cross-entity read; needs calc column __calc on — re-run /setup-datamodel (or /add-dataverse for existing apps) to add the calc column.` Do NOT scaffold an N+1 fetch storm. | - | Detail (single record) | 1:1 (N:1 lookup chain) | Scaffold a chained `.get(record.__value, { select: [...] })` in the screen's load step. One record on screen = one extra round trip is fine. Display via `lookupName(...)` or direct field read. | - | Any | 1:many or M:N | Scaffold a chained `.getAll({ filter: \`__value eq '${id}'\`, select: [...] })`. Calc columns CANNOT traverse 1:many or M:N — chained fetch is the ONLY pattern. | + | `formatted-lookup` | Select the real `__value` field and render `lookupName(record, '')`. | + | `chained-fetch` | Perform one bounded related `get` / `getAll` in the screen load step, never inside `map()` or `renderItem`. | + | `external-projection-required` | Render no fake fallback data. Return `BLOCKED` and name the field/source. The user must supply a supported server-owned projection outside this workflow. | - 3. **Verify before exit.** Every UI field in your spec must have either (a) a `select` entry on the primary fetch (covered by direct column or calc column) OR (b) a chained fetch path. If a field has neither, return `BLOCKED []: field on has no fetch path — add to spec or add calc column`. + 2. **Verify before exit.** Every UI field in your spec must have either a + primary select, formatted lookup annotation, or bounded chained fetch. + Otherwise return `BLOCKED []: field requires an external projection`. - 4. **TODO comment shape** (when emitting at step 2 / list branch): + 3. **TODO comment shape** for an external projection: ```ts // TODO(cross-entity-read): screen displays from related . - // Re-run /setup-datamodel (or /add-dataverse for existing apps) and add calc - // column __calc to for one-round-trip reads. - // List screens MUST NOT chain fetches in renderItem — N+1 storm. + // Supply a maker-created formula column or another server-owned projection, + // then rerun Dataverse reconciliation. Never chain reads in renderItem. ``` - **HARD RULE — server-managed columns are NEVER in a create or update payload.** The Dataverse server owns these fields; including them in a `*Service.create({...})` or `*Service.update({...})` returns HTTP 400 on every save. Generated `create()` types may include server-managed fields (`ownerid`, `statecode`, primary IDs, etc.) because they mirror the full model; do **not** satisfy those types by emitting junk values. For any screen with create/update behavior, use a narrow write helper/type whose input contains only editable fields. If the skeleton imports an app-level helper, call it; otherwise define the helper inside your assigned screen file. Do **not** create or modify shared `src/utils/`, `src/hooks/`, or service files from a screen-builder. Forbidden keys in any create/update payload: @@ -153,12 +153,17 @@ You will be invoked by `/create-mobile-app` Step 11 or `/edit-app` screen-rebuil }; export async function createTask(input: CreateTaskInput): Promise { - const payload: Record = { + type CreateFields = Pick< + Parameters[0], + 'cr3e9_name' | 'cr3e9_status' | 'cr3e9_projectid@odata.bind' + >; + const payload: Omit + & Partial> = { cr3e9_name: input.title, cr3e9_status: input.status, }; if (input.projectId) { - payload['cr3e9_Project@odata.bind'] = `/cr3e9_projects(${input.projectId})`; + payload['cr3e9_projectid@odata.bind'] = `/cr3e9_projects(${input.projectId})`; } const result = await Cr3e9_tasksService.create(payload as Parameters[0]); @@ -168,17 +173,22 @@ You will be invoked by `/create-mobile-app` Step 11 or `/edit-app` screen-rebuil - **If a create flow needs the new record immediately, pre-generate its Dataverse ID.** Do not navigate, upload, create child rows, or build lookup binds from `result.data?.`. Use the create-then-navigate rule below: call `newId()` from `@/utils`, include the primary ID field in the create payload, check `result.success`, then navigate/bind/upload using the ID you generated before the POST. This is the only safe pattern for flows like scan → detail, create parent → child form, or create evidence row → upload image. - **Dynamic route IDs are untrusted.** Detail/edit/upload screens that read `useLocalSearchParams()` MUST normalize and validate the route ID before any Dataverse service call. Treat missing values and literal strings like `'undefined'` / `'null'` as invalid. Never pass `String(params.id ?? '')` directly into `Service.get(id)`, `Service.update(id, ...)`, or `Service.upload(id, ...)`; `enabled: !!id` is not enough because `'undefined'` is truthy. Required pattern: ```ts - const id = normalizeRouteId(params.id); - const validId = isDataverseId(id); - const query = useQuery({ enabled: validId, queryFn: () => Service.get(id) }); - if (!validId) return ; + const params = useLocalSearchParams<{ id?: string | string[] }>(); + const rawId = Array.isArray(params.id) ? params.id[0] : params.id; + const id = normalizeDataverseGuid(rawId); + const query = useQuery({ + enabled: !!id, + queryFn: () => Service.get(id!), + }); + if (!id) return ; ``` + Import `normalizeDataverseGuid` from `@/utils`. Do not invent alternate ID helpers or an inline UUID regex. - **Lookup writes use `@odata.bind`, NEVER raw GUIDs.** When a form creates or updates a record with a parent reference (Task → Project, Comment → Task, Inspection → Site, etc.), the foreign key field is set with the entity-bind syntax. Setting it any other way either silently saves `null` (data loss — form looks like it succeeded) or 400s with a cryptic Dataverse error. - - **Required pattern** — use the lookup's **schema name** (PascalCase navigation property), suffix with `@odata.bind`, value is `/()`: + - **Required pattern** — open the generated target model and copy the exact quoted property ending in `@odata.bind`; value is `/()`: ```ts await Cr3e9_tasksService.create({ cr3e9_name: title, - 'cr3e9_Project@odata.bind': `/cr3e9_projects(${projectId})`, + 'cr3e9_projectid@odata.bind': `/cr3e9_projects(${projectId})`, cr3e9_status: TaskStatus.Open, // choice = number }); ``` @@ -186,12 +196,19 @@ You will be invoked by `/create-mobile-app` Step 11 or `/edit-app` screen-rebuil ```ts { _cr3e9_project_value: projectId } // _value props are READ-ONLY; silently dropped on create { cr3e9_project: projectId } // raw GUID on nav property; 400 - { 'cr3e9_project@odata.bind': projectId } // missing /entitySet(guid) wrapper; 400 + { 'cr3e9_projectid@odata.bind': projectId } // missing /entitySet(guid) wrapper; 400 ``` - **Finding the right names:** - - **Schema name** (left of `@odata.bind`): the lookup column's PascalCase logical name, usually exposed in the generated model file (`src/generated/models/Model.ts`). Often differs from the `_value` read property by case + dropped underscore (read `_cr3e9_project_value`, write `cr3e9_Project@odata.bind`). + - **Write property** (left of `@odata.bind`): use the exact key declared in `src/generated/models/Model.ts`. It is case-sensitive and may be lowercase even when raw Web API metadata exposes a PascalCase navigation property. Never guess or transform the read `_value` property. - **Entity set name** (inside `/(...)`): always the **plural** logical collection name — `cr3e9_projects`, not `cr3e9_project`. Use `pluralName` from the model file or check the generated service filename (`Cr3e9_projectsService.ts` ⇒ entity set is `cr3e9_projects`). - - When in doubt, grep `@odata.bind` in `src/generated/services/` for an existing example, or ask the `microsoft-learn` MCP server. Full reference: [`skills/add-dataverse/references/dataverse-reference.md` § Setting Lookups](${PLUGIN_ROOT}/skills/add-dataverse/references/dataverse-reference.md#setting-lookups-creatingupdating-records). + - Grep `@odata.bind` in the generated **model**, not the generated service. If the key is absent, return `BLOCKED` rather than inventing it. + - **Typed payload rule:** do not start write payloads as `Record`. Define a `Pick[N], ...>` containing the exact writable keys, then use a single boundary cast only when generated base types incorrectly require server-managed fields. This makes misspelled or wrongly-cased lookup keys fail TypeScript. + - **Lookup filter rule:** generated mobile services do not reliably support raw Web API relationship traversal such as `lookupNav/relatedColumn eq ...`. Query the related table service first, then filter the source table using its read lookup GUID property (`_lookuplogicalname_value eq `). +- **Authenticated profile rule:** when the plan links a profile table to `systemuser`, import the generated `SystemusersService`. Resolve token `oid → systemuserid` by filtering `azureactivedirectoryobjectid`, and reject disabled, missing, or duplicate users. + - Before writing the profile filter, open the generated profile model and locate the exact declared read lookup property for the planned `systemuser` lookup column. For a planned lookup logical name such as `new_systemuserid`, the expected shape is `_new_systemuserid_value`, but the generated model declaration is authoritative. + - Copy that exact model property into the filter: ` eq `. Never emit guessed placeholders such as `_lookup_value`, `_systemuserlookup_value`, or `_
_value`. + - If the planned lookup column is absent, multiple candidate `systemuser` lookups exist, or the exact read key is not declared in the generated model, return `BLOCKED []: exact systemuser lookup read key not verified`. Do not fall back to email matching or relationship traversal. + - If `SystemusersService` is absent from the Generated Services snapshot, return `BLOCKED`; do not invent a raw Web API client. - **Form picker UI** — when the form's spec calls for a parent picker (e.g., "select Project"), the picker stores the selected record's `id` (GUID string), and the submit handler converts it to the bind string at write time. Never store the bind string in component state — only in the API payload. - **Pagination rule:** If your spec says `pagination: cursor`, do NOT use `useListData` or `useSearchFilter`. Use the skeleton's `useCursorListData` call, React Query's `useInfiniteQuery`, or an app-specific `useCursorList` hook with FlatList `onEndReached` per the pattern in [`data-performance.md`](${PLUGIN_ROOT}/shared/references/data-performance.md). Never fetch all records at once, and never treat `top: 50` as pagination. Real generated Dataverse services use SDK `maxPageSize` for page size and return `IOperationResult.skipToken` for the next page; pass that value back as `skipToken`. Always include deterministic `orderBy` with a unique key and `select` in the service call. Push search/filter into Dataverse with `filter`. If the generated service in the app does not expose `maxPageSize`/`skipToken` for an unbounded table, return `BLOCKED []: generated service does not expose cursor paging for ; do not downgrade to useListData`. @@ -940,7 +957,13 @@ Before finishing the screen, mentally verify: 24. **Forms wrap content in ``** with `behavior="padding"` on iOS. 25. **Form submit calls `router.back()` on success by default.** Exception: create-then-navigate workflows that immediately continue into the created record MUST pre-generate a Dataverse GUID with `newId()` from `@/utils` (which wraps `Crypto.randomUUID()`), include it as the primary ID field in the create payload, then navigate using that known ID. Do not read the new ID from `result.data`, and do not refetch/search after create just to find it. Never use meaningful/sensitive IDs, and do not use this exception for normal saves, bulk inserts, or sample data unless an immediate follow-up operation requires the ID. 25. **Route intent matrix is enforced in code.** Singleton routes use `router.navigate(...)`; detail drill-down uses `router.push(...)`; auth/guard redirects use `router.replace(...)`. `router.push(...)` to known singleton routes is a generation error. -25. **Dynamic route IDs are validated before Dataverse calls.** Every detail/edit/upload route normalizes `useLocalSearchParams()` values and rejects missing, `'undefined'`, `'null'`, or non-GUID IDs before calling `Service.get`, `Service.update`, or `Service.upload`. `enabled: !!id` is not sufficient. Bug killed: HTTP 400 `table(undefined)` after create-then-navigate. +25. **Dynamic route IDs use the shared Dataverse GUID normalizer before every Dataverse call.** Every detail/edit/upload route MUST import `normalizeDataverseGuid` from `@/utils`; never write an inline UUID/GUID regex. Normalize array-valued Expo params first, then call the helper: + ```ts + const params = useLocalSearchParams<{ id?: string | string[] }>(); + const rawId = Array.isArray(params.id) ? params.id[0] : params.id; + const id = normalizeDataverseGuid(rawId); + ``` + Reject an undefined result before calling `Service.get`, `Service.update`, `Service.delete`, `Service.upload`, or `Service.download*`. `enabled: !!rawId`, a generic RFC UUID validator, and checks for only `'undefined'` / `'null'` are forbidden. Dataverse sequential GUIDs do not guarantee RFC version bits. Bug killed: valid Dataverse IDs rejected locally and HTTP 400 `table(undefined)` after create-then-navigate. 25. **Scanner Dataverse writes are locked and reset on focus.** Any scanner callback that creates/updates Dataverse rows, uploads evidence, or navigates to a created record uses a `useRef` in-flight lock plus `paused` state, passes `paused` and `resetKey` to `BarcodeScannerView`, resets lock/paused/resetKey inside `useFocusEffect`, and routes manual code entry through the same guarded mutation. Bug killed: rapid QR callbacks creating duplicate or broken scan rows, and scanner stuck when returning from detail. 25. **Camera evidence screens show a visible Take picture action.** Any evidence/photo capture flow has a first-class `Take picture` / `Take evidence photo` button wired to `takePhoto()` from `src/native/camera`. Gallery/upload/file picker actions may exist only as secondary siblings, never as the only visible capture path. Bug killed: evidence screen where camera capture exists technically but users cannot find it. 26. **Create / update payloads NEVER include server-managed columns.** Forbidden keys in any `*Service.create({...})` or `*Service.update({...})` object literal: `ownerid`, `owneridtype`, `statecode`, `statuscode`, `importsequencenumber`, `overriddencreatedon`, `timezoneruleversionnumber`, `utcconversiontimezonecode`, `versionnumber`, `createdon`, `modifiedon`, `createdby`, `modifiedby`. If the generated model type marks them required, put the unavoidable cast inside the narrow write helper — never emit junk like `ownerid: ''` or `statecode: 0` to satisfy the type. Bug killed: every HTTP 400 on save. diff --git a/plugins/mobile-apps/agents/screen-planner.md b/plugins/mobile-apps/agents/screen-planner.md index e1982db91..07062b3f1 100644 --- a/plugins/mobile-apps/agents/screen-planner.md +++ b/plugins/mobile-apps/agents/screen-planner.md @@ -210,12 +210,14 @@ Always include these baseline screens (already in template — keep them): Then design the user's screens. For a typical CRUD app: - **List screen** per primary entity (e.g., `accounts/index.tsx`) -- **Detail screen** per primary entity (e.g., `accounts/[id].tsx`) +- **Detail screen** per primary entity (e.g., `accounts/[id].tsx` when it has no children, or `accounts/[id]/index.tsx` when it owns child workflows) - **Create/edit form screen** per primary entity (e.g., `accounts/new.tsx`, `accounts/[id]/edit.tsx`) - Plus any workflow-specific screens (e.g., `capture-receipt.tsx`) **Folder rule (HARD — prevents phantom tabs):** any entity that has children (`[id]`, `new`, `edit`, sub-screens) becomes a **folder** with `/index.tsx` for the list/root view and the children inside. Never use a flat `accounts.tsx` AND a sibling `accounts/[id].tsx` — expo-router auto-registers every top-level `.tsx` under `app/(app)/` as a tab/drawer entry, so a flat `accounts.tsx` next to an `accounts/` folder produces both a phantom "accounts" tab AND the real "accounts" tab. Folders collapse the whole stack into one navigable entry. +**Dynamic-route collision rule (HARD):** never emit both `/[id].tsx` and `/[id]/.tsx`. Expo Router maps the file and folder to the same `[id]` navigator entry and crashes with `duplicate screen named '[id]'`. When a detail route has child workflows, its detail file is `/[id]/index.tsx`; child files and `_layout.tsx` live in that same `[id]/` folder. + Decision rule per top-level destination: | Destination has any sub-routes? | Layout | @@ -226,7 +228,10 @@ Decision rule per top-level destination: Examples: - `home.tsx` (no children) → flat file `app/(app)/home.tsx` - `profile.tsx` (no children) → flat file `app/(app)/profile.tsx` -- `inspections` (list + detail + form) → folder `app/(app)/inspections/` with `index.tsx`, `[id].tsx`, `new.tsx` +- `inspections` (list + detail + form, no detail children) → folder `app/(app)/inspections/` with `index.tsx`, `[id].tsx`, `new.tsx` +- `inspections` (detail owns photo/edit children) → `app/(app)/inspections/[id]/index.tsx`, `app/(app)/inspections/[id]/photo.tsx`, and `app/(app)/inspections/[id]/edit.tsx`; never also create `inspections/[id].tsx` + +**Authenticated Dataverse identity rule:** when app identity links through `systemuser`, require `SystemusersService` in the data-source/service plan and record the profile table's planned `systemuser` lookup logical column. Resolve the access-token `oid` with `SystemusersService.getAll({ filter: "azureactivedirectoryobjectid eq and isdisabled eq false", top: 2 })`. The screen-builder must open the generated profile model and use the exact declared read property corresponding to that lookup column (for example `_new_systemuserid_value` for `new_systemuserid`). Never put generic placeholders such as `_lookup_value` or `_systemuserlookup_value` in a concrete filter, and do not use relationship traversal (`lookupNavigation/azureactivedirectoryobjectid`). Keep total screen count tight — under 8 for v0 unless the requirements explicitly demand more. The user can iterate later. @@ -379,7 +384,7 @@ For each screen the user adds, provide this compact shape: - **Profile content** — REQUIRED on the Profile screen only. List 2-4 app-specific profile sections based on the app requirements and target users, for example `Role + team`, `Assigned site/territory`, `Default queue filters`, `App support/contact`, `Environment/app version`. Include any generated service needed for those sections; otherwise use local/static app context plus `useAuth()`. - **Sign-out affordance** — REQUIRED on the Profile screen and omitted from every other screen. Write `visible Button "Sign out" using useAuth().signOut with confirm`; sign-out returns to `/login` after completion. - **Data** — which generated services it calls, with method names (e.g., bounded lookup: `AccountsService.getAll({ top: 50, orderBy: ['name asc'], select: ['name'] })`; cursor list: `InspectionsService.getAll({ maxPageSize: 50, orderBy: ['scheduledDate asc', 'inspectionid asc'], select: [...] })` plus `skipToken` continuation support) -- **Related entity fields** (REQUIRED if any UI field on the screen displays data from an entity OTHER than the primary `Data` service's table; OMIT entirely otherwise) — one entry per cross-entity field. The `data-model-architect`'s Step 6a Cross-entity Read Audit reads this block to decide which calculated columns to propose. Mechanical schema: +- **Related entity fields** (REQUIRED if any UI field on the screen displays data from an entity OTHER than the primary `Data` service's table; OMIT entirely otherwise) — one entry per cross-entity field. The `data-model-architect` audits that each field has a supported read path. Mechanical schema: ```yaml related_entity_fields: @@ -387,16 +392,19 @@ For each screen the user adds, provide this compact shape: source: cardinality: "1:1" | "1:many" | "M:N" archetype_class: list | detail | tab-root | dashboard - recommends: calc-column | chained-fetch + recommends: formatted-lookup | chained-fetch | external-projection-required ``` **Mechanical derivation of `recommends`** (no judgement — pick from this table): | `archetype_class` | `cardinality` | `recommends` | |---|---|---| - | `list`, `tab-root`, `dashboard` | `1:1` | `calc-column` | + | `list`, `tab-root`, `dashboard` | direct lookup primary display | `formatted-lookup` | + | `list`, `tab-root`, `dashboard` | any other `1:1` related field | `external-projection-required` | | `detail` | `1:1` | `chained-fetch` | - | any | `1:many` or `M:N` | `chained-fetch` (calc columns can't traverse) | + | `list`, `tab-root`, `dashboard` | `1:many` or `M:N` per-row field/aggregate | `external-projection-required` | + | `detail` | `1:many` | `chained-fetch` | + | `detail` | `M:N` | `external-projection-required` unless a generated intersect-table service and exact bounded query contract are already named in the approved data model | **`archetype_class` mapping from `Archetype`:** `List` → `list`; `Tab-root` → `tab-root` (or `dashboard` if `Operational pattern: home-dashboard` / `assignment-dashboard`); `Detail` → `detail`; `Form` / `Modal-Sheet` / `Auth` / `Empty-onboarding` → `detail` (cold path, single-record context). @@ -410,22 +418,22 @@ For each screen the user adds, provide this compact shape: source: cr3e9_flightid → cr3e9_flightnumber cardinality: "1:1" archetype_class: list - recommends: calc-column + recommends: formatted-lookup - field: "Gate name" source: cr3e9_flightid → cr3e9_gateid → cr3e9_gatename cardinality: "1:1" archetype_class: list - recommends: calc-column + recommends: external-projection-required - field: "Defect count" source: cr3e9_inspectionzoneid → cr3e9_defect (1:many) cardinality: "1:many" archetype_class: list - recommends: chained-fetch + recommends: external-projection-required ``` **Hard rule:** if the screen displays a related-entity field but you do NOT emit a `related_entity_fields` block for it, the data-model-architect cannot propose the calc column, the screen-builder will hit `BLOCKED` at scaffold time, and the user will see a `—` cell in the built app. The block is the ONLY signal — there is no fallback inference. - **Audit** (omit for read-only / non-write screens) — one line per audit-bearing action: `: event (); payload: `. Example: `On submit: event 100000006 (Inspection Submitted); payload: inspectionId, submittedAt, defectCount, openCriticalCount.` The screen-builder wraps the payload field list in `JSON.stringify({...})` and writes the full `cr3e9_audit_log_entriesService.create(...)` call from the Generated Services table — do NOT spell out the wrapper or service name. -- **Lookup writes** — for form/edit screens that set a parent reference (Task → Project, Comment → Task, etc.), explicitly list each lookup field with its `@odata.bind` name + entity set, e.g. `'cr3e9_Project@odata.bind': '/cr3e9_projects()'`. Without this the screen-builder will guess and silently lose the relationship. Skip for read-only and no-lookup screens. +- **Lookup writes** — for form/edit screens that set a parent reference (Task → Project, Comment → Task, etc.), explicitly copy the exact quoted `@odata.bind` property from the generated target model and pair it with the entity set, e.g. `'cr3e9_projectid@odata.bind': '/cr3e9_projects()'` when that exact key exists in `src/generated/models/Model.ts`. Never derive casing from Dataverse schema-name conventions. Without the generated-model key, mark the spec `BLOCKED: lookup write key not verified`. Skip for read-only and no-lookup screens. - **Pagination** — `cursor` if the table has no natural record ceiling (visits, inspections, work orders, tickets, any user-created records over time); `none` if the table is a bounded lookup (status types, categories, job types). When `cursor`, include SDK `maxPageSize: 50`, deterministic `orderBy` with a unique key, `select`, `skipToken` continuation support, and server-side `filter` for search in the data spec. Do not imply that `top: 50` alone is pagination. - **Native capabilities** — which native modules/wrappers it uses, and which iOS/Android platforms or permission states need fallback handling. For PDF/pen screens, be precise: `document-picker` (`expo-document-picker`) for user-picked files; `pdf-report` (`expo-print`, plus `expo-sharing` only when present and sharing is required) for generated local PDFs; `native-pdf-viewer` (`@microsoft/power-apps-native-pdf-viewer` 0.2.9+) for HTTPS PDF URLs and local `file://` URIs; `pen-input` (`@microsoft/power-apps-native-pen-input`) for signature/ink capture. For location screens, distinguish `geolocation` (`@microsoft/power-apps-native-bglocation`) — continuous/background tracking with native Dataverse sync, needs start/stop/tracking-status UI plus a permission-denied state — from one-shot `location` (`expo-location`) for a single foreground coordinate read. - **Calendar library** — REQUIRED for screens with `Calendar pattern` unless the pattern is `timeline-day-list`. Write `react-native-calendars` and name the exact components expected, for example `CalendarProvider`, `ExpandableCalendar`, `AgendaList`, `Calendar`, `CalendarList`, or `Agenda`. The package must also appear in `### JavaScript Dependencies`; the screen-builder imports it directly after the orchestrator installs it. No `/add-native` wrapper or native rebuild is involved. @@ -472,7 +480,7 @@ This is the target shape for every spec. ~120 words, ~450 tokens. No inlined cat - **UX contract:** header title = current zone name; primary action = `Save & Continue` bottom CTA; disabled reason = "Capture required photo first" when evidence missing; FAB = `extended FAB` on defects, label "Add defect"; badge count = `defects.filter(d => d.zone === currentZone).length`. - **Data:** `Cr3e9_zoneprogressService.getAll({ filter: 'cr3e9_inspectionid eq ', orderBy: 'cr3e9_zone asc' })`, `Cr3e9_zoneprogressService.update(...)` on save. - **Audit:** On zone Save: event 100000001 (Zone Step Completed); payload: zoneIndex, zoneName, completedAt, evidenceCount, defectCount. -- **Lookup writes:** `'cr3e9_Inspection@odata.bind': '/cr3e9_inspections()'` on every zone-progress upsert. +- **Lookup writes:** exact generated-model key, for example `'cr3e9_inspectionid@odata.bind': '/cr3e9_inspections()'`, on every zone-progress upsert. - **Pagination:** `none` (6-row bounded set). - **Native capabilities:** `expo-camera`, `expo-image-picker` (capture tiles). - **Navigation:** from inspection detail; pushes to defect form; pops back to inspection summary on last zone Save. @@ -533,7 +541,7 @@ Section format (same in all phases): | OAuth callback | `/oauth-callback` | `app/oauth-callback.tsx` | default | Connector consent return | — | — | template (keep) | | Home | `/(app)/home` | `app/(app)/home.tsx` | default | Today dashboard: assignment, progress, stats, recent inspections | `cr123_inspectionService.getAll({ top: 5 })` | — | replace template | | Inspections list | `/(app)/inspections` | `app/(app)/inspections/index.tsx` | default | List + filter | `cr123_inspectionService.getAll` | — | new | -| Inspection detail | `/(app)/inspections/[id]` | `app/(app)/inspections/[id].tsx` | default | View + edit one | `getById`, `update` | — | new | +| Inspection detail | `/(app)/inspections/[id]` | `app/(app)/inspections/[id]/index.tsx` | default | View + edit one | `getById`, `update` | — | new | | New inspection | `/(app)/inspections/new` | `app/(app)/inspections/new.tsx` | modal | Create form, slides up from list | `create` | — | new | | Capture photo | `/(app)/inspections/[id]/photo` | `app/(app)/inspections/[id]/photo.tsx` | modal | Take or pick photo | `update` (photo column) | `expo-camera`, `expo-image-picker` | new | | Profile | `/(app)/profile` | `app/(app)/profile.tsx` | default | User info + sign out | `useAuth()` only | — | new | @@ -687,7 +695,7 @@ Navigation: |-----------------|-----------------------------|-----------------------------------------|--------------|-----------|-------------------|---------------| | Home | /(app)/home | app/(app)/home.tsx | default | Tab-root | - | - | | Inspections | /(app)/inspections | app/(app)/inspections/index.tsx | default | List | InspectionService | - | -| Inspection ID | /(app)/inspections/[id] | app/(app)/inspections/[id].tsx | default | Detail | InspectionService | - | +| Inspection ID | /(app)/inspections/[id] | app/(app)/inspections/[id]/index.tsx | default | Detail | InspectionService | - | | New Inspection | /(app)/inspections/new | app/(app)/inspections/new.tsx | modal | Form | InspectionService | camera | | Profile | /(app)/profile | app/(app)/profile.tsx | default | Tab-root | - | - | diff --git a/plugins/mobile-apps/scripts/check-routes.js b/plugins/mobile-apps/scripts/check-routes.js index fe817bbea..edb705228 100644 --- a/plugins/mobile-apps/scripts/check-routes.js +++ b/plugins/mobile-apps/scripts/check-routes.js @@ -67,7 +67,7 @@ function findTsxFiles(dir) { // app/(app)/inspections/index.tsx → /inspections function fileToRoute(filePath, appRoot) { - const rel = path.relative(appRoot, filePath); + const rel = path.relative(appRoot, filePath).replace(/\\/g, '/'); const noExt = rel.replace(/\.tsx$/, ''); if (/(^|\/)_layout$/.test(noExt)) return null; // layouts aren't screens if (/(^|\/)\+not-found$/.test(noExt)) return null; // not-found boundary @@ -243,9 +243,28 @@ function main() { } const files = findTsxFiles(appRoot); + const screenFiles = files.filter(file => path.basename(file) !== '_layout.tsx'); + const fileFolderCollisionFindings = []; + for (const file of screenFiles) { + const relative = path.relative(appRoot, file).replace(/\\/g, '/'); + const base = relative.replace(/\.tsx$/, ''); + const children = screenFiles.filter(other => { + const otherRelative = path.relative(appRoot, other).replace(/\\/g, '/'); + return otherRelative.startsWith(`${base}/`); + }); + if (children.length > 0) { + fileFolderCollisionFindings.push({ + route: fileToRoute(file, appRoot), + file, + childFiles: children, + kind: 'file-folder-route-collision', + }); + } + } // Build dest registry: { route: { file, declaredKeys, declaredRaw } } const dests = {}; + const duplicateRouteFindings = []; // Collect senders: array of { fromFile, route, params } const allSenders = []; @@ -255,11 +274,20 @@ function main() { if (route) { const declared = parseLocalSearchParams(content); - dests[route] = { - file, - declaredKeys: declared ? declared.keys : null, - declaredRaw: declared ? declared.raw : null, - }; + if (dests[route]) { + duplicateRouteFindings.push({ + route, + file, + otherFile: dests[route].file, + kind: 'duplicate-route', + }); + } else { + dests[route] = { + file, + declaredKeys: declared ? declared.keys : null, + declaredRaw: declared ? declared.raw : null, + }; + } } const senders = parseSenders(content); @@ -289,11 +317,12 @@ function main() { } // Diff: for each dest, what's received but not declared? - const findings = []; + const findings = [...fileFolderCollisionFindings, ...duplicateRouteFindings]; for (const r of destRouteList) { const d = dests[r]; const r2 = received[r]; if (r2.sources.length === 0) continue; // unreachable destination — different bug class + if (Object.keys(r2.params).length === 0) continue; // navigation without params needs no declaration if (!d.declaredKeys) { // Destination receives params but has NO useLocalSearchParams call. // That's a real issue if the screen uses any of those params. @@ -340,10 +369,24 @@ function main() { process.exit(0); } - console.error(`✗ check-routes: ${findings.length} destination(s) missing param declarations.\n`); + console.error(`✗ check-routes: ${findings.length} route contract issue(s).\n`); for (const f of findings) { console.error(` Route: ${f.route}`); console.error(` File: ${path.relative(cwd, f.file)}`); + if (f.kind === 'file-folder-route-collision') { + console.error(` Issue: Route file conflicts with a same-name child folder.`); + console.error(` Child: ${f.childFiles.map(child => path.relative(cwd, child)).join(', ')}`); + console.error(` Fix: Move ${path.basename(f.file)} to ${path.basename(f.file, '.tsx')}/index.tsx.`); + console.error(''); + continue; + } + if (f.kind === 'duplicate-route') { + console.error(` Issue: Duplicate Expo route.`); + console.error(` Other: ${path.relative(cwd, f.otherFile)}`); + console.error(` Fix: If the route owns child screens, use /index.tsx and remove the sibling .tsx file.`); + console.error(''); + continue; + } if (f.kind === 'no-declaration') { console.error(` Issue: No useLocalSearchParams<>() call, but ${Object.keys(f.receivedParams).length} param(s) are sent here.`); } else { diff --git a/plugins/mobile-apps/scripts/create-calculated-column.js b/plugins/mobile-apps/scripts/create-calculated-column.js index a1c9f0a6c..48c31e6ed 100755 --- a/plugins/mobile-apps/scripts/create-calculated-column.js +++ b/plugins/mobile-apps/scripts/create-calculated-column.js @@ -1,409 +1,59 @@ #!/usr/bin/env node - -// Creates a Dataverse calculated column on a parent table that resolves a value -// from a related entity via an N:1 dotted-path navigation chain. -// -// Companion to /add-dataverse Step 5c and /setup-datamodel Phase 5. -// See `shared/references/data-performance.md` § Cross-entity Reads for the -// runtime constraint that motivates this helper (the SDK has no $expand). -// -// Usage: -// node create-calculated-column.js \ -// --table \ -// --column __calc \ -// --type string|datetime|decimal|integer|boolean \ -// --formula ".<...>." \ -// [--display "Display Name"] \ -// [--solution ] \ -// [--formula-xml ''] -// -// Exit codes: -// 0 — created successfully -// 1 — bad args, auth failure, network failure, or HTTP 4xx/5xx from Dataverse -// -// Output (JSON to stdout, single line): -// Success: { "status": 204, "table": "...", "column": "...", "type": "..." } -// Failure: { "status": , "table": "...", "column": "...", "error": "..." } - -const { getAuthToken, makeRequest } = require('./lib/validation-helpers'); - -// ─── arg parsing ──────────────────────────────────────────────────────────── - -function parseArgs() { - const argv = process.argv.slice(2); - if (argv.length < 1 || argv[0].startsWith('--')) { - usage('envUrl is required as the first positional argument'); - } - - const out = { - envUrl: argv[0].replace(/\/+$/, ''), - table: null, - column: null, - type: null, - formula: null, - formulaXml: null, - display: null, - solution: null, - }; - - for (let i = 1; i < argv.length; i++) { - const flag = argv[i]; - const next = argv[i + 1]; - switch (flag) { - case '--table': out.table = next; i++; break; - case '--column': out.column = next; i++; break; - case '--type': out.type = (next || '').toLowerCase(); i++; break; - case '--formula': out.formula = next; i++; break; - case '--formula-xml': out.formulaXml = next; i++; break; - case '--display': out.display = next; i++; break; - case '--solution': out.solution = next; i++; break; - default: - usage(`Unknown flag: ${flag}`); - } - } - - if (!out.table) usage('--table is required'); - if (!out.column) usage('--column is required'); - if (!out.type) usage('--type is required (string|datetime|decimal|integer|boolean)'); - if (!out.formula && !out.formulaXml) { - usage('--formula (dotted path) or --formula-xml (full workflow XML) is required'); - } - - if (!/^[a-z0-9_]+$/.test(out.column)) { - usage(`--column must be lowercase alphanumeric + underscores: got "${out.column}"`); - } - if (!/^(string|datetime|decimal|integer|boolean)$/.test(out.type)) { - usage(`--type must be one of string|datetime|decimal|integer|boolean: got "${out.type}"`); - } - - // Default display name: strip _calc suffix, title-case the rest - if (!out.display) { - const base = out.column.replace(/^[a-z0-9]+_/, '').replace(/_calc$/i, ''); - out.display = base - .split('_') - .filter(Boolean) - .map((w) => w.charAt(0).toUpperCase() + w.slice(1)) - .join(' ') || out.column; - } - - return out; -} - -function usage(msg) { - process.stderr.write(`Error: ${msg}\n\n`); - process.stderr.write( - 'Usage:\n' + - ' node create-calculated-column.js \\\n' + - ' --table
--column --type string|datetime|decimal|integer|boolean \\\n' + - ' --formula "" [--display "..."] [--solution ] [--formula-xml ]\n' - ); - process.exit(1); -} - -// ─── payload construction ─────────────────────────────────────────────────── - -const TYPE_MAP = { - string: { odataType: 'Microsoft.Dynamics.CRM.StringAttributeMetadata', attrType: 'String', extra: { MaxLength: 200, FormatName: { Value: 'Text' } } }, - datetime: { odataType: 'Microsoft.Dynamics.CRM.DateTimeAttributeMetadata', attrType: 'DateTime', extra: { Format: 'DateAndTime', DateTimeBehavior: { Value: 'UserLocal' } } }, - decimal: { odataType: 'Microsoft.Dynamics.CRM.DecimalAttributeMetadata', attrType: 'Decimal', extra: { MinValue: -100000000000, MaxValue: 100000000000, Precision: 2 } }, - integer: { odataType: 'Microsoft.Dynamics.CRM.IntegerAttributeMetadata', attrType: 'Integer', extra: { MinValue: -2147483648, MaxValue: 2147483647, Format: 'None' } }, - boolean: { odataType: 'Microsoft.Dynamics.CRM.BooleanAttributeMetadata', attrType: 'Boolean', extra: { - DefaultValue: false, - OptionSet: { - TrueOption: { Value: 1, Label: { LocalizedLabels: [{ Label: 'Yes', LanguageCode: 1033 }] } }, - FalseOption: { Value: 0, Label: { LocalizedLabels: [{ Label: 'No', LanguageCode: 1033 }] } }, - }, - } }, -}; - -// Build the calculated-column payload. SourceType: 1 = Calculated. -function buildPayload({ column, type, display, formulaXml }) { - const t = TYPE_MAP[type]; - if (!t) throw new Error(`unsupported type: ${type}`); - - // Schema name = column with the publisher prefix kept lowercase but the - // remainder PascalCase (Dataverse convention for calc columns). - const schemaName = toSchemaName(column); - +'use strict'; + +/** + * Safety guard for legacy callers. + * + * Dataverse exposes FormulaDefinition in metadata, but Microsoft does not + * support defining calculated, rollup, or formula expressions through code. + * Legacy calculated columns require internal workflow XAML that has no public + * authoring contract. Formula columns use a different serialized definition, + * but that definition must also be created through the Power Apps editor. + * + * This helper therefore fails before authentication or mutation. Callers must + * use a supported runtime read path, or require the formula column to be + * created in Power Apps and then validate/reuse it as an existing dependency. + */ + +const DOCUMENTATION_URL = + 'https://learn.microsoft.com/power-apps/developer/data-platform/specialized-columns'; + +function parseArgs(argv) { + const args = { environmentUrl: argv[2] || null }; + for (let index = 3; index < argv.length; index += 1) { + const flag = argv[index]; + if (!flag.startsWith('--')) continue; + const value = argv[index + 1]; + args[flag.slice(2)] = value && !value.startsWith('--') ? value : true; + if (args[flag.slice(2)] !== true) index += 1; + } + return args; +} + +function blockedResult(args) { return { - '@odata.type': t.odataType, - AttributeType: t.attrType, - AttributeTypeName: { Value: `${t.attrType}Type` }, - SchemaName: schemaName, - LogicalName: column, - DisplayName: { LocalizedLabels: [{ Label: display, LanguageCode: 1033 }] }, - Description: { - LocalizedLabels: [ - { Label: 'Calculated read-only column auto-derived from screen plan (cross-entity read).', LanguageCode: 1033 }, - ], - }, - RequiredLevel: { Value: 'None' }, - SourceType: 1, - SourceTypeMask: 1, - FormulaDefinition: formulaXml, - IsValidForCreate: false, - IsValidForUpdate: false, - ...t.extra, + status: 'BLOCKED', + code: 'UNSUPPORTED_FORMULA_DEFINITION_API', + table: args.table || null, + column: args.column || null, + error: + 'Dataverse does not support defining calculated, rollup, or formula expressions through code. ' + + 'Create the computed column in the Power Apps editor, then rerun reconciliation to validate ' + + 'and reuse it, or use a supported formatted-lookup/chained-fetch runtime read path.', + documentation: DOCUMENTATION_URL, }; } -// Dataverse schema-name convention for calc columns: keep the publisher prefix -// lowercase, PascalCase the remainder, drop any trailing _calc → _calc kept as-is. -function toSchemaName(logical) { - const m = logical.match(/^([a-z0-9]+)_(.+)$/); - if (!m) return logical; - const [, prefix, rest] = m; - const parts = rest.split('_'); - const pascal = parts - .map((p, idx) => (idx === parts.length - 1 && p === 'calc' ? 'calc' : p.charAt(0).toUpperCase() + p.slice(1))) - .join(''); - return `${prefix}_${pascal}`; +function main() { + const result = blockedResult(parseArgs(process.argv)); + process.stdout.write(`${JSON.stringify(result)}\n`); + process.exitCode = 2; } -// ─── formula → workflow XML ───────────────────────────────────────────────── -// -// Dataverse calculated-column formulas are stored as workflow-style XML, NOT as -// a plain dotted-path string. v0 of this helper supports the most common shape: -// an N:1 navigation chain that ends in a primitive read. -// -// For complex formulas (conditionals, arithmetic, string concat), the caller -// MUST pass the full workflow XML via --formula-xml and skip the dotted-path -// translation. -// -// Reference: the canonical workflow XML produced by the maker portal for a -// simple lookup-field calc column. We emit a minimal-but-valid variant. - -function dottedPathToWorkflowXml({ table, column, formula, type }) { - const t = TYPE_MAP[type]; - if (!t) throw new Error(`unsupported type for formula synthesis: ${type}`); - const segments = formula.split('.').map((s) => s.trim()).filter(Boolean); - if (segments.length < 2) { - throw new Error( - `--formula must traverse at least one navigation hop. Got "${formula}". ` + - 'For a non-related field, use a regular (non-calc) column.' - ); - } - // Last segment = target attribute; everything before = navigation chain. - const targetAttr = segments[segments.length - 1]; - const navChain = segments.slice(0, -1); - - // Build the GetEntityProperty chain in reverse: innermost reads from the - // first hop's related entity, outermost wraps the final target attribute. - // Dataverse's calculated-column engine accepts a flat single-Read activity - // for an N-hop chain by referencing the dotted attribute name directly on - // the first navigation property — this is the "shortcut" form supported on - // modern envs. Older orgs may require a nested Read tree; we emit the flat - // form first and surface a 400 with a clear message if it's rejected. - - const dottedTail = navChain.length === 1 - ? targetAttr - : `${navChain.slice(1).join('.')}.${targetAttr}`; - - // XML-escape attribute values - const xe = (s) => String(s).replace(/[<>&"']/g, (c) => ({ - '<': '<', '>': '>', '&': '&', '"': '"', "'": ''', - }[c])); +if (require.main === module) main(); - return [ - '', - '', - ' ', - ` ${xe(`${table}.${navChain[0]}`)}`, - ' ', - ` `, - ' ', - ` ${xe(table)}`, - ' ', - ` `, - '', - ].join('\n'); -} - -// ─── HTTP ─────────────────────────────────────────────────────────────────── - -async function postAttribute({ envUrl, table, payload, token, solution }) { - const url = `${envUrl}/api/data/v9.2/EntityDefinitions(LogicalName='${encodeURIComponent(table)}')/Attributes`; - const headers = { - Authorization: `Bearer ${token}`, - Accept: 'application/json', - 'Content-Type': 'application/json', - }; - if (solution) headers['MSCRM.SolutionUniqueName'] = solution; - - const res = await makeRequest({ - url, - method: 'POST', - headers, - body: JSON.stringify(payload), - includeHeaders: true, - timeout: 30000, - }); - return res; -} - -// ─── main ─────────────────────────────────────────────────────────────────── - -async function main() { - const args = parseArgs(); - let token = await getAuthToken(args.envUrl); - if (!token) { - process.stderr.write('Failed to get Azure CLI token. Run `az login` first.\n'); - process.exit(1); - } - - let formulaXml; - try { - formulaXml = args.formulaXml - ? args.formulaXml - : dottedPathToWorkflowXml({ - table: args.table, - column: args.column, - formula: args.formula, - type: args.type, - }); - } catch (e) { - console.log(JSON.stringify({ - status: 0, - table: args.table, - column: args.column, - error: `formula synthesis failed: ${e.message}`, - })); - process.exit(1); - } - - const payload = buildPayload({ - column: args.column, - type: args.type, - display: args.display, - formulaXml, - }); - - // Up to 3 attempts on 401 (token refresh) or 429 (rate-limit backoff). - let lastRes = null; - for (let attempt = 0; attempt < 3; attempt++) { - const res = await postAttribute({ - envUrl: args.envUrl, - table: args.table, - payload, - token, - solution: args.solution, - }); - lastRes = res; - - if (res.error) { - // Network-level failure — retry once - if (attempt < 2) continue; - break; - } - - // Treat 2xx as success (POST to /Attributes returns 204 No Content normally) - if (res.statusCode >= 200 && res.statusCode < 300) { - console.log(JSON.stringify({ - status: res.statusCode, - table: args.table, - column: args.column, - type: args.type, - display: args.display, - solution: args.solution || null, - })); - process.exit(0); - } - - // Token refresh on 401 - if (res.statusCode === 401 && attempt < 2) { - const fresh = await getAuthToken(args.envUrl); - if (!fresh) break; - // mutate token via closure for next iteration - // (simpler than rewriting the loop body) - // eslint-disable-next-line no-func-assign, no-unused-vars - // re-assign by shadowing - // Note: token is const above; re-bind via a let variable here. - // Restart loop with fresh token. - // Using a small inline trampoline to keep the change minimal. - return retryWithFreshToken(args, fresh, formulaXml); - } - - // 429 backoff (15s) - if (res.statusCode === 429 && attempt < 2) { - await new Promise((r) => setTimeout(r, 15000)); - continue; - } - - // Other 4xx/5xx — don't retry, surface error - break; - } - - // Failure path - let errMsg = lastRes?.error || `HTTP ${lastRes?.statusCode}`; - if (lastRes?.body) { - try { - const parsed = JSON.parse(lastRes.body); - const dvErr = parsed?.error?.message; - if (dvErr) errMsg = `${errMsg}: ${dvErr}`; - } catch { - errMsg = `${errMsg}: ${lastRes.body.slice(0, 500)}`; - } - } - console.log(JSON.stringify({ - status: lastRes?.statusCode || 0, - table: args.table, - column: args.column, - error: errMsg, - })); - process.exit(1); -} - -// Token-refresh trampoline — replays the POST once with a fresh token. -async function retryWithFreshToken(args, freshToken, formulaXml) { - const payload = buildPayload({ - column: args.column, - type: args.type, - display: args.display, - formulaXml, - }); - const res = await postAttribute({ - envUrl: args.envUrl, - table: args.table, - payload, - token: freshToken, - solution: args.solution, - }); - - if (res.statusCode >= 200 && res.statusCode < 300) { - console.log(JSON.stringify({ - status: res.statusCode, - table: args.table, - column: args.column, - type: args.type, - display: args.display, - solution: args.solution || null, - })); - process.exit(0); - } - - let errMsg = res.error || `HTTP ${res.statusCode}`; - if (res.body) { - try { - const parsed = JSON.parse(res.body); - const dvErr = parsed?.error?.message; - if (dvErr) errMsg = `${errMsg}: ${dvErr}`; - } catch { - errMsg = `${errMsg}: ${res.body.slice(0, 500)}`; - } - } - console.log(JSON.stringify({ - status: res.statusCode || 0, - table: args.table, - column: args.column, - error: errMsg, - })); - process.exit(1); -} - -main().catch((e) => { - process.stderr.write(`Unhandled error: ${e.stack || e.message}\n`); - process.exit(1); -}); +module.exports = { + DOCUMENTATION_URL, + blockedResult, + parseArgs, +}; diff --git a/plugins/mobile-apps/scripts/dataverse-request.js b/plugins/mobile-apps/scripts/dataverse-request.js index 55aea7603..49763b25b 100644 --- a/plugins/mobile-apps/scripts/dataverse-request.js +++ b/plugins/mobile-apps/scripts/dataverse-request.js @@ -5,7 +5,7 @@ // // Arguments: // envUrl - Dataverse environment URL (e.g., https://org123.crm.dynamics.com) -// method - HTTP method: GET, POST, PATCH, DELETE — OR — BATCH-RECORDS (see below) +// method - HTTP method: GET, POST, PUT, PATCH, DELETE, BATCH-RECORDS, or BATCH-METADATA // apiPath - API path after /api/data/v9.2/ (e.g., "EntityDefinitions?$filter=...") // For BATCH-RECORDS mode, this is treated as a label (e.g. "Tier 0") for logging only. // @@ -13,9 +13,10 @@ // --body Request body as JSON string // --include-headers Include response headers in output (for OData-EntityId etc.) // --solution Sets MSCRM.SolutionUniqueName header (target metadata at a solution) +// --tenant-id Uses the resolved environment tenant without shell-global state // // BATCH-RECORDS mode (record-level inserts only — NEVER use for metadata writes): -// node dataverse-request.js BATCH-RECORDS
')/Attributes?\$select=LogicalName,AttributeType,RequiredLevel" +Read the results as follows: + +- **A planned name present in `value[]`** — the table exists. Cache its expanded `Attributes` as that table's **attribute snapshot** for Steps 5a and 5b. +- **A planned name absent from `value[]`** — the table does not exist. This is the equivalent of a 404 in the matrix below. +- Interpret `IsCustomizable` and `CanCreateAttributes` as managed properties and read their `.Value` fields. + +If the batched query itself fails (non-2xx), retry it once; if it fails again, split it into per-table queries so one unreadable name cannot hide the rest. Any name still unreadable after that is `unverified`: STOP before writes for that reconciliation scope. Authentication, permission, timeout, and malformed-response failures are not evidence that a name is free. If the URL would exceed a practical length with very many tables, split it into a few filtered queries — still far fewer than one request per table. + +**Only if the plan contains alternate keys or M:N relationships**, add the matching expands so Steps 5b and 5d never need their own per-item probes. `EntityDefinitions` also supports expanding [`Keys`, `ManyToManyRelationships`, `ManyToOneRelationships`, and `OneToManyRelationships`](https://learn.microsoft.com/power-apps/developer/data-platform/query-schema-definitions#evaluate-other-options-to-retrieve-schema-definitions): + +```text +&$expand=Attributes($select=...),Keys($select=SchemaName,KeyAttributes,EntityKeyIndexStatus),ManyToManyRelationships($select=SchemaName) ``` -Schema divergence handling is in Step 5b's per-column pre-flight (not a Step 4 prompt). The pre-flight auto-skips columns that already exist with the same type, and STOPs only on incompatible type drift — no separate confirmation needed here. +Do not add these expands when the plan has no keys or M:N relationships — they enlarge the response for no benefit, and standard tables carry many of both. + +#### Step 4a — Targeted derived-metadata barrier + +The base attribute snapshot is sufficient for ordinary columns, but it cannot +prove that a same-named lookup, choice, Boolean, or computed column has the same +semantics. Before classifying any such existing column as compatible: + +1. Write the planned derived-column contract to + `/.tmp/derived-metadata-expected.json`. Each row contains: + `table`, `logicalName`, `kind`, `type`, `sourceType`, plus: + - `lookupTarget` for lookups; + - exact integer/label `options` for Choice, MultiSelect Choice, and Boolean; + - exact `sourceTypeMask` and serialized `formulaDefinition` for an explicitly + approved, maker-created computed dependency. +2. Build one `BATCH-METADATA` GET operation list for the affected existing + tables only. Reuse one process/token and query: + - `ManyToOneRelationships` once per child table containing planned lookups; + - the applicable derived attribute collections + (`PicklistAttributeMetadata`, `MultiSelectPicklistAttributeMetadata`, + `BooleanAttributeMetadata`) once per table/type, expanding `OptionSet`; + - the applicable typed attribute collection once per table/type for any + explicitly reused computed column, selecting + `LogicalName,SourceType,SourceTypeMask,FormulaDefinition`. + + Do not issue one process per column and do not scan every customizable table. + The exact planned names from Step 4 are the scope. + Write the operation array to + `/.tmp/derived-metadata-operations.json`, then run: + + ```bash + node "${CLAUDE_SKILL_DIR}/../../scripts/dataverse-request.js" \ + BATCH-METADATA derived-reconciliation \ + --operations "$(cat /.tmp/derived-metadata-operations.json)" \ + --tenant-id '' + ``` + + Do not pass `--continue-on-error`; the first unreadable required metadata + collection must stop the barrier. +3. Any non-2xx response, missing result slot, malformed option metadata, absent + lookup target, or unavailable `FormulaDefinition` makes that scope + `unverified`. **STOP before writes.** Authentication, throttling, permission, + and parse failures are never compatibility evidence. +4. Normalize the live results into + `/.tmp/derived-metadata-live.json`. Each row uses: + `table`, `logicalName`, `type`, `sourceType`, `sourceTypeMask`, + `lookupTargets`, `options: [{ value, label }]`, and `formulaDefinition`. + Lookup target arrays must contain exactly the approved target. Choice + mappings must be non-empty with unique integer values and non-empty labels; + Boolean mappings must contain exactly values 0 and 1. Then run: + + ```bash + node "${PLUGIN_ROOT}/scripts/validate-derived-metadata.js" \ + --expected "/.tmp/derived-metadata-expected.json" \ + --actual "/.tmp/derived-metadata-live.json" + ``` + +5. A lookup is compatible only when its complete target set matches. Planned choice + values must exist with the same labels; extra live values are allowed. + Ordinary planned columns require `SourceType` 0. A maker-created computed + dependency is reusable only when its source type, source-type mask, and exact + `FormulaDefinition` match the approved artifact; the `Invalid` mask bit + always blocks reuse. + +This phase is read-only and uses the V2 long-lived executor. It must not add +per-column child-process/token overhead back into the fast path. + +Build and print a reconciliation matrix before Step 5: + +| Target result | Table decision | Column decisions | Action | +|---|---|---|---| +| Present; all planned base and derived metadata compatible | `reuse` | existing columns `reuse` | No schema write. | +| Present; custom columns missing; table customizable and can create attributes | `extend` | compatible `reuse`; absent custom `create` | Queue missing ordinary columns for sequential creation; relationships remain Pass 2. | +| Absent; plan says Create; logical name uses the verified publisher prefix | `create` | ordinary columns `create` inline; lookups deferred | Create once after the complete-payload self-check. | +| Absent; plan says Reuse/Extend or dependency is standard/managed/required-existing | `defer` | dependent columns `defer` | Never recreate a standard or managed table. Drop the dependent lookups/columns from this run, continue with everything else, and list them under Deferred in Step 9. | +| Present; same-name column has incompatible `AttributeType` / `AttributeTypeName.Value` | `extend` | incompatible column `adapt` | Auto-rename the planned column via the probe sequence below, record it in the alias map, and create it alongside the existing one. Never modify or delete the existing column. | +| Present; columns missing but `IsCustomizable.Value=false` or `CanCreateAttributes.Value=false` | `reuse` | missing columns `defer` | The target cannot be extended by this workflow. Reuse the columns that do exist, drop the rest from this run, and list them under Deferred in Step 9. | +| Batched query failed (non-2xx) after retry and per-table split | `unverified` | unknown | STOP before writes for the affected reconciliation scope and surface the concrete environment/auth/permission error. | + +`replace` is not an automatic state in this workflow. Replacing a table or column requires an explicitly approved migration with dependency analysis and data movement, so a conflict resolves to `adapt` (rename beside it) or `defer` (leave it out) instead — both of which leave existing data untouched. + +**Decide-before-write barrier (HARD):** finish reconciliation for every table and column before the first metadata write. Every item must come out of Step 4 as `reuse`, `extend`, `create`, `adapt`, or `defer` — never as an unresolved conflict. Deciding renames up front is what keeps relationships, screens, and sample data pointing at the same names. + +**No dead ends (HARD):** a data-modelling conflict must never stop the run. Adapt it (rename beside the existing object) or defer it (drop it from this run), then keep going and report it in Step 9. Only environment faults stop this skill — failed auth, an environment mismatch, or a target the user has no privilege to write to. Those are not data-modelling problems and the user cannot resolve them by editing the plan. + +**Idempotency criterion (HARD):** re-running this skill against an already-applied plan MUST perform **zero** metadata writes. Every table, column, relationship, key, and calc column resolves to `reuse` or an "already exists, skipped" outcome from the Step 4 snapshot. If a re-run issues any POST, the reconciliation missed something — report it rather than writing. Use this as the acceptance check after any change to Steps 4, 5, or 5a–5d. ### Step 5 — Create / extend tables **Print before starting:** > "→ Creating/extending tables in tier order (sequential — Dataverse serializes metadata writes). For each: pre-flight check, then 'Creating
…' before the POST and '✓
' on 2xx response." -> **⚠️ Concurrency rule — do not violate.** All Dataverse metadata operations in Steps 5, 6, and 6b are **strictly sequential**: issue one HTTP request, wait for a 2xx response, then issue the next. Do NOT batch, parallelize, or fire concurrent requests. Dataverse serializes metadata writes via an exclusive lock; parallel calls return `429 TooManyRequests`, `MetadataLockHeldException`, or `404 EntityNotFound` for lookups whose parent hasn't committed yet. +> **⚠️ Concurrency rule — do not violate.** All Dataverse metadata operations in Steps 5, 6, and 6b are **strictly sequential**: issue one HTTP request, wait for a 2xx response, then issue the next. Do NOT parallelize or use OData `$batch`. Dataverse serializes metadata writes via an exclusive lock; parallel calls return `429 TooManyRequests`, `MetadataLockHeldException`, or `404 EntityNotFound` for lookups whose parent hasn't committed yet. > > Specifically: > - **Within a tier:** create tables one at a time. @@ -196,35 +312,62 @@ Schema divergence handling is in Step 5b's per-column pre-flight (not a Step 4 p > - **Lookups:** POST to `/RelationshipDefinitions` only **after** both endpoint tables exist and have returned 2xx. > - **Extensions:** column POSTs to an existing table are also serial — same lock applies. -#### Step 5a — Pre-flight collision check (per table, before each POST) +For multiple already-reconciled operations, prefer the local `BATCH-METADATA` +executor. It is **not** OData `$batch`: one Node process reuses one token and +issues requests strictly one at a time in array order, stopping on the first +non-2xx response by default. + +```bash +node "${CLAUDE_SKILL_DIR}/../../scripts/dataverse-request.js" \ + BATCH-METADATA schema-writes \ + --operations '' \ + --solution '' \ + --tenant-id '' +``` + +Each operation is `{ "index", "method", "apiPath", "body" }`; an operation may +override the command-level `solution`. Build the array only after the full +metadata snapshot and desired/live diff. Preserve dependency order: new tables +with ordinary columns inline, extension columns, relationships, projections, +then alternate keys. Never pass `--continue-on-error` for schema creation. The +result includes per-operation `status` and `durationMs`; after a failure, +reconcile that component and resume with only the remaining operations. + +#### Step 5a — Pre-flight collision check (from the Step 4 snapshot) + +Before each create, confirm the target name is actually free: name-prefix collisions from stale solutions, reserved system names, and soft-deleted tombstones all fail the POST, and Dataverse takes ~1 minute to return the conflict error. A failure here can leave Tier 0 partially created and make a Tier 1 lookup fail on a phantom parent. Step 4 already collected this evidence for every planned name, so this step reads it rather than re-querying. -Step 4 listed *known* custom tables you intend to reuse. Step 5a probes for *unknown* problems on a per-create basis: name-prefix collisions from stale solutions, soft-deleted tombstones, and reserved system names. Skipping this check costs ~1 minute per failed POST (Dataverse takes its time returning the conflict error) and can leave Tier 0 partially created when a Tier 1 lookup fails on a phantom parent. +**For every `Create` entry, resolve its target state from the Step 4 batch — do not re-query per table.** Step 4 already fetched every planned logical name, so reuse that result: -**For every `Create` entry, before its POST, probe the target logical name:** +| Step 4 result for this name | Meaning | Action | +|---|---|---| +| **Absent from `value[]`** | Name is free | Proceed with POST. | +| **Present** + `IsCustomEntity: true` + `MetadataId` matches memory-bank | We created this earlier — idempotent re-run | Skip the POST, mark as created, continue. | +| **Present** + `IsCustomEntity: true` + `MetadataId` *not* in memory-bank | Foreign collision | Reconcile live columns and customization properties below; never auto-extend an uncustomizable target. | +| **Present** + `IsCustomEntity: false` | Reserved system table name | Auto-recover via rename (see below). | + +Only re-probe a single name when Step 4's batch did not cover it (for example a rename candidate generated later in this step): ```bash node "${CLAUDE_SKILL_DIR}/../../scripts/dataverse-request.js" GET \ - "EntityDefinitions(LogicalName='_
')?\$select=MetadataId,LogicalName,IsCustomEntity" + "EntityDefinitions(LogicalName='_
')?\$select=MetadataId,LogicalName,IsCustomEntity,IsManaged,IsCustomizable,CanCreateAttributes" \ + --tenant-id '' ``` -Branch on the response: +Tombstones and hidden collisions are **not** reliably visible to either form — Dataverse can report a name as free and still reject the POST minutes later. Those are caught by the POST-time collision rescue below, which is the real safety net: -| Status / body | Meaning | Action | +| POST response | Meaning | Action | |---|---|---| -| **404 NotFound** | Name is free | Proceed with POST. | -| **200 OK** + `IsCustomEntity: true` + `MetadataId` matches memory-bank | We created this earlier — idempotent re-run | Skip the POST, mark as created, continue. | -| **200 OK** + `IsCustomEntity: true` + `MetadataId` *not* in memory-bank | Foreign collision | Auto-recover (see below) — do NOT prompt. | -| **200 OK** + `IsCustomEntity: false` | Reserved system table name | Auto-recover via rename (see below). | | **5xx** with `0x80060890` or message `"object with same name exists in solution"` | Tombstone (soft-deleted, ~30 min purge TTL) | Auto-recover via rename (see below). | -| **400** with `0x80044363`, `"schema name ... is not unique"`, or `"same name already exists"` | Hidden Dataverse collision / recent-delete tombstone not visible to `EntityDefinitions` GET | Auto-recover via rename (see below), then retry the POST once. | +| **400** with `0x80044363`, `"schema name ... is not unique"`, or `"same name already exists"` | Hidden Dataverse collision / recent-delete tombstone | Auto-recover via rename (see below), then retry the POST once. | -**Important:** Step 5a is a best-effort preflight, not the final authority. Dataverse can return 404 for a recently deleted table and still reject the create POST minutes later because the schema name remains reserved internally. Treat that POST-time 400 as a recoverable name collision, not a data-model failure. +**Important:** treat a POST-time collision as a recoverable name conflict, not a data-model failure — the schema name can stay reserved internally after a delete even when metadata reports it as free. #### Auto-recovery — reuse/extend first, rename as last resort **Priority order when Step 5a hits a name collision:** -1. **Adopt as Extend (preferred)** — if the existing table's `Attributes` overlap with the planned columns by ≥50%, or the existing table is the same conceptual entity: add only the missing columns via per-column POST (Step 5b Extend path). No prompt needed — extend automatically and log `→ Extending existing with missing columns.` +1. **Adopt as Extend (preferred)** — only if the existing table is the same concept, every same-name column is type-compatible, planned missing columns are custom additions, and live `IsCustomizable.Value` plus `CanCreateAttributes.Value` both permit extension. Add only the missing columns via Step 5b and log `→ Extending existing with missing columns.` 2. **Adopt as Reuse** — if the existing table's schema already covers all planned columns: skip Step 5b for this entry, keep it in Step 6 for service generation. No prompt. Log `→ Reusing existing (all required columns present).` 3. **Rename and Create (last resort)** — only when the existing table is a fundamentally different entity (e.g., planned table is an inspection log but existing `` is a payroll record — incompatible concept, incompatible columns). Prompt the user before proceeding. @@ -232,8 +375,9 @@ Branch on the response: | Situation | Action | |---|---| -| Foreign collision + schema overlap ≥50% | Auto-Extend (no prompt) | +| Foreign collision + compatible concept/schema + extension allowed | Auto-Extend (no prompt) | | Foreign collision + all planned columns present | Auto-Reuse (no prompt) | +| Foreign collision + incompatible column or extension forbidden | Auto-rename the conflicting column beside it, or defer it (no prompt) | | Foreign collision + incompatible concept | Prompt (see below) | | Reserved system name | Auto-rename (no prompt) | | Tombstone (0x80060890 / same-name-exists) | Auto-rename (no prompt) | @@ -243,11 +387,11 @@ Branch on the response: ``` | Option | What it means | |---|---| -| Extend existing (default) | Add required columns to . Safer — avoids duplicate tables. | -| Rename and Create | Auto-renamed to . Existing table left untouched. | +| Rename and Create (default) | Use a free custom logical name for the genuinely different entity. Existing table stays untouched. | +| Reuse existing as-is | Point the generated services at the existing table and skip the planned columns it lacks. | ``` -Default to "Extend existing" so an empty answer auto-proceeds. Rename-and-Create is the opt-in exception, not the default. +Never offer Extend for an incompatible concept or column shape. This prompt is a preference, not a gate: an empty, skipped, or unanswered response defaults to **Rename and Create** so the run always proceeds. Maintain a run-level logical-name alias map for every auto-rename. Example: @@ -267,7 +411,7 @@ For each candidate in order, GET `EntityDefinitions(LogicalName='')?$ - 404 → free, **take it**, stop probing. - 200 or 5xx (collision) → next candidate. -If all 4 probes collide, surface a `BLOCKED: cannot find a free alternative for ` and stop. +If all 4 collide, keep probing `3`, `4`, … through `20`. This sequence is designed never to dead-end: if even those collide, use `<4-char run token>`, which is unique to this run. Never abandon a table for want of a free name. **On a successful auto-rename, do these in order BEFORE the POST:** @@ -289,21 +433,28 @@ If the Step 5b table POST fails after a 404 preflight with any Dataverse name-co - message contains `same name already exists` - message contains `object with same name exists in solution` -First attempt auto-Extend: re-GET the existing table's attributes and compare with the plan. If ≥50% overlap, switch to Extend path (add missing columns). Otherwise, run the auto-rename probe sequence, update `native-app-plan.md`, `## Screens`, `memory-bank.md`, and the run-level alias map, then retry the table POST exactly once with the resolved name. Print: +First attempt auto-Extend: compare the plan against that table's attribute snapshot from Step 4 (re-fetch only if it is absent). If ≥50% overlap, switch to Extend path (add missing columns). Otherwise, run the auto-rename probe sequence, update `native-app-plan.md`, `## Screens`, `memory-bank.md`, and the run-level alias map, then retry the table POST exactly once with the resolved name. Print: > `→ Dataverse still has reserved from a recent delete/hidden collision. Using and continuing.` -If the retry also returns a collision signature, continue probing the remaining candidates. If all candidates collide, return `BLOCKED: cannot find a free alternative for `. +If the retry also returns a collision signature, continue probing the remaining candidates, then the numeric tail, then the run-token form described above. **On successful POST**, immediately re-GET to capture the server-assigned `MetadataId` and write it to memory-bank (Step 6d updates `.datamodel-manifest.json`; you also append to `memory-bank.md` under "Created tables" with the GUID and solution name). This lets future `/add-dataverse` runs distinguish "we own this" from "name collision." #### Step 5b — Create / extend +Run this step in two explicit passes: + +1. **Pass 1 — tables + ordinary columns:** create every new table with all planned non-lookup columns inline, then extend existing tables with any missing non-lookup columns. Computed formula definitions are never synthesized by this workflow. +2. **Pass 2 — lookups + relationships:** only after Pass 1 has succeeded for every tier, create the planned `RelationshipDefinitions`. A lookup is created by its relationship and must not appear in a table's initial `Attributes` array. + +Both passes remain strictly sequential. Do not use `$batch` for metadata writes. + For each `Create` decision, in **tier order** (Tier 0 → Tier 1 → Tier 2 → …), POST a new EntityDefinition. Skip if Step 5a returned a known-self match (idempotent). > **⚠️ Inline ALL planned columns into the Create POST body — do NOT POST columns individually.** > -> Dataverse processes the `Attributes: [...]` array atomically with the table create. Inline form: 1 round trip, ~3-8s. Per-column form: N+1 round trips, each ~3-8s. For a 5-column table that's 24s saved per table on the lock-serialized metadata path. +> Dataverse accepts all non-lookup columns in the initial table's `Attributes: [...]` array. Inline form: 1 round trip, ~3-8s. Per-column form: N+1 round trips, each ~3-8s. For a 5-column table that's roughly 24s saved on the lock-serialized metadata path. Do not describe this metadata create as transactionally atomic: if the request fails after Dataverse starts processing it, the table or some attributes may remain, which is why the recovery path below exists. > > **Wrong** (N round trips): > ```json @@ -334,9 +485,20 @@ For each `Create` decision, in **tier order** (Tier 0 → Tier 1 → Tier 2 → ```bash node "${CLAUDE_SKILL_DIR}/../../scripts/dataverse-request.js" POST EntityDefinitions \ --body '' \ - --solution '' + --solution '' \ + --tenant-id '' ``` +**Complete-payload self-check (HARD — do this before the POST, no extra tooling):** re-read the body you just built against the plan and confirm all five statements. If any fails, fix the body and re-check; never POST a partial table and repair it with per-column POSTs. + +1. **Every planned ordinary column is present** — String, Memo, Integer, BigInt, Decimal, Money, DateTime, Boolean, Choice, MultiSelect Choice, Image, and File. Count `Attributes[]` and compare with the plan's column count for this table. +2. **No lookup metadata is inline** — no `LookupAttributeMetadata`, `CustomerAttributeMetadata`, or `OwnerAttributeMetadata`. Those are created by their relationship in Pass 2. +3. **No deferred or server-owned column is inline** — no primary-id column and no calculated/rollup column (Step 5c owns those). +4. **Exactly one `IsPrimaryName: true` attribute exists**, and `PrimaryNameAttribute` matches its `SchemaName` (lowercased logical form). +5. **No duplicate `SchemaName`** in `Attributes[]` (compare case-insensitively). + +Microsoft documents both halves of this contract: [ordinary columns may be included when the table is created](https://learn.microsoft.com/power-apps/developer/data-platform/webapi/create-update-column-definitions-using-web-api#create-columns), while a [lookup is created with its one-to-many relationship](https://learn.microsoft.com/power-apps/developer/data-platform/webapi/create-update-entity-relationships-using-web-api#create-a-one-to-many-relationship). `dataverse-request.js` stays the only script in this path — it already owns auth, retry, `--solution` routing, and 401/429 handling. + Body skeleton — **all planned columns inline in `Attributes: [...]`** (this example shows primary + 3 additional; expand the array to fit every column from the plan): > **⚠️ `IsAvailableOffline` + `ChangeTrackingEnabled` MUST be set to `true` at create time** for any table the app intends to make available offline. Without these two flags the table cannot be added to a `mobileofflineprofile`, and `/setup-offline-profile` will have to fix them via a separate metadata PUT (the `/enable-tables-offline` skill handles that, but it doubles the metadata-lock-serialized round trips). Empirically verified 2026-05-18 in the chanel-rm demo: 7 custom tables created without these flags caused 7 prereq-revert drift entries; fixed by post-hoc enablement. Default these to `true` for all UserOwned tables created by `/add-dataverse` unless the user has explicitly opted out of offline support. The flags are no-ops at runtime for apps that don't use offline profiles. @@ -398,36 +560,38 @@ Body skeleton — **all planned columns inline in `Attributes: [...]`** (this ex For each `Extend` decision, POST a new column to the existing table. -> **⚠️ Per-column pre-flight (HARD — required for idempotent re-runs).** Before each column POST, probe whether the column already exists. This catches: -> - Partial failures from a prior `EntityDefinitions` POST that created the table + some columns but not all (the body is non-atomic — server commits each Attribute one at a time). -> - User re-runs after fixing a typo in one column's metadata. -> - Re-applying a plan after a network drop mid-Step-5b. -> -> Without this check, the second POST returns `400: attribute already exists` (`0x80044153`) and the run aborts mid-tier. +> **⚠️ Table-level pre-flight (HARD — required for idempotent re-runs).** Reuse the complete attribute snapshot fetched for this table in Step 4. If the table was discovered only during collision recovery, or no current snapshot exists, fetch all attributes exactly once: > > ```bash > node "${CLAUDE_SKILL_DIR}/../../scripts/dataverse-request.js" GET \ -> "EntityDefinitions(LogicalName='
')/Attributes(LogicalName='')?\$select=LogicalName,AttributeType" +> "EntityDefinitions(LogicalName='
')/Attributes?\$select=MetadataId,LogicalName,SchemaName,AttributeType,AttributeTypeName,RequiredLevel,IsManaged,IsCustomizable,IsPrimaryId,IsPrimaryName" \ +> --tenant-id '' > ``` > -> | Status | Meaning | Action | -> |---|---|---| -> | **404** | Column doesn't exist | Proceed with POST. | -> | **200** + `AttributeType` matches the spec | Already created (idempotent re-run) | Skip the POST, log `↻ (already exists, skipped)`, continue. | -> | **200** + `AttributeType` differs from the spec | Schema drift — column type was changed manually OR plan changed since last run | **STOP** and surface to user: "Column `` exists but is ``, plan expected ``. Dataverse does NOT allow column-type changes via API — you must delete the column manually and re-run." Do NOT silently overwrite. | +> Build a local `{ lowerCaseLogicalName → { AttributeType, AttributeTypeName, IsManaged, IsCustomizable } }` map and classify **every** planned non-lookup column before issuing any POST. Prefer `AttributeTypeName.Value` when `AttributeType` is `Virtual`, so File and Image columns are not incorrectly treated as compatible generic virtual attributes: +> +> | Snapshot result | Action | +> |---|---| +> | Name exists and `AttributeType` matches | Skip it and log `↻ (already exists, skipped)`. | +> | Name exists and `AttributeType` differs | Dataverse does not allow column-type changes via API, so **auto-rename the planned column** using the same probe sequence (`v2` → `v3` → `2` → …), add it to the alias map, and create it beside the existing one. Log `→ exists as ; created as instead.` Never modify or delete the existing column. | +> | Name is absent | Add it to the ordered missing-column queue. | +> +> This single snapshot catches partial creates, corrected re-runs, and network-drop recovery without paying one GET round trip per column. -After pre-flight returns 404, POST the column (always pass `--solution`): +After the complete comparison passes, POST the missing-column queue **one column at a time, sequentially** (no `$batch`; always pass `--solution`): ```bash node "${CLAUDE_SKILL_DIR}/../../scripts/dataverse-request.js" POST \ "EntityDefinitions(LogicalName='
')/Attributes" \ --body '' \ - --solution '' + --solution '' \ + --tenant-id '' ``` -**The same pre-flight applies inside Create POSTs that include initial Attributes.** If a Create POST partially failed earlier (table + some columns committed), the retry path is to do **per-column** pre-flight + POST instead of re-POSTing the whole `EntityDefinitions` body — re-POSTing returns `0x80060888 entity already exists`. After Step 5a says "table exists with our MetadataId" (idempotent re-run match), iterate the planned `Attributes` and pre-flight each one against `/Attributes(LogicalName='')`, then POST only the missing ones. +**Recovery after a partial Create:** do not re-POST the whole `EntityDefinitions` body because the table now exists and Dataverse returns `0x80060888`. Fetch that table's complete attribute snapshot once, run the same local comparison, and sequentially POST only the missing non-lookup columns. This is the only time a table originally classified as Create should use the per-column path. Column shapes that have non-obvious gotchas (handle carefully): +- **Pass-2 barrier (HARD)** — do not create any lookup or relationship while Pass 1 is still creating or extending tables. Accumulate relationship definitions, wait until every table and ordinary column has returned 2xx, then process the relationships sequentially in dependency order. - **Lookup** — POST to `/RelationshipDefinitions`, not `/Attributes`. > **⚠️ Do NOT improvise the body. Copy the skeleton below verbatim and replace only the placeholders in `<>` brackets.** @@ -470,9 +634,20 @@ Column shapes that have non-obvious gotchas (handle carefully): node "${CLAUDE_SKILL_DIR}/../../scripts/dataverse-request.js" POST \ RelationshipDefinitions \ --body '' \ - --solution '' + --solution '' \ + --tenant-id '' ``` + **Pre-flight the lookup (HARD — required for idempotent re-runs).** A lookup can only pre-exist if the **referencing (child) table** already existed at Step 4, so when the child was created in this run, skip the probe and POST. Otherwise look for the lookup's foreign-key column — the lowercased `Lookup.SchemaName`, e.g. `_id` — in that child table's Step 4 attribute snapshot: + + | Snapshot result | Action | + |---|---| + | Present with `AttributeType: Lookup` | Skip the POST and log `↻ (relationship already exists, skipped)`. | + | Present with any other `AttributeType` | A non-lookup column already owns that name. Auto-rename the lookup's `Lookup.SchemaName` via the probe sequence, record it in the alias map, and POST the relationship with the new name. Never overwrite the existing column. | + | Absent | POST the relationship. | + + This costs no extra round trip: the referencing attribute is an ordinary attribute on the child table, so it is already in the snapshot Step 4 fetched. Without this check a re-run POSTs a duplicate relationship and fails the run mid-Pass-2. + - **Many-to-Many (M:N)** — also POST to `/RelationshipDefinitions`, but with `ManyToManyRelationshipMetadata`. Dataverse creates an auto-named intersect table. > **⚠️ Do NOT improvise the body.** Required fields: `SchemaName`, `Entity1LogicalName`, `Entity2LogicalName`, `IntersectEntityName`, and the two `AssociatedMenuConfiguration` blocks. Do not include lookup or cascade fields — those are 1:N concepts. @@ -499,9 +674,9 @@ Column shapes that have non-obvious gotchas (handle carefully): } ``` - **Pre-flight M:N:** GET `RelationshipDefinitions(SchemaName='__')?$select=SchemaName` — 404 → proceed; 200 → skip (already exists). + **Pre-flight M:N:** a relationship can only pre-exist on a table that already existed at Step 4. If **either** endpoint was just created in this run, skip the probe — nothing can be there. Otherwise read `ManyToManyRelationships` from that table's Step 4 snapshot: present → skip (already exists); absent → proceed. Query `RelationshipDefinitions(SchemaName='__')?$select=SchemaName` only when the snapshot did not cover it. - **In the generated service:** M:N relationships are queried via the intersect entity name (e.g., `cr123_tag_inspection`) — the SDK does not expose a direct M:N navigation helper; the screen-builder must query the intersect table directly or via a calculated column approach. Flag this in the Step 7 summary if any M:N relationships are created. + **In the generated service:** M:N relationships are queried via the intersect entity name (e.g., `cr123_tag_inspection`) — the SDK does not expose a direct M:N navigation helper; the screen-builder must query the intersect table directly. Flag this in the Step 7 summary if any M:N relationships are created. - **Column `@odata.type` and required fields — reference table (verified against Dataverse OData API):** @@ -547,38 +722,24 @@ If the column type is not a simple string/int/boolean, surface a one-line confir After all mutations, re-run the existing-tables query (Step 4) to confirm everything landed. -### Step 5c — Create calculated columns from `### Cross-entity Reads` - -**Print before starting:** -> "→ Creating calculated columns from the plan's ### Cross-entity Reads subsection (one HTTP call per row). Skip if the subsection is absent." - -**Run condition:** the planner / data-model-architect emits a `### Cross-entity Reads (auto-derived from screen plan)` subsection inside `## Data Model` of `native-app-plan.md` when the screen plan reads any field from a related entity. Parse that subsection. If absent or empty, **skip Step 5c entirely** — proceed to Step 6. +### Step 5c — Enforce the supported computed-column boundary -This step exists because of the runtime constraint documented at [`shared/references/data-performance.md` § Cross-entity Reads](${PLUGIN_ROOT}/shared/references/data-performance.md#cross-entity-reads): the SDK has no `$expand`, so cross-entity fields on hot paths (lists, dashboards, tab roots) MUST be denormalized via calculated columns at the data-model layer. The `### Chained-fetch fields (informational)` subsection (if present) is documentation only — the screen-builder handles those at scaffold time, no schema change. +Dataverse metadata exposes `FormulaDefinition`, but Microsoft explicitly does +not support defining calculated, rollup, or formula expressions through code. +Legacy workflow XAML and generated formula serialization must not be synthesized. -**Algorithm:** +- Do not invoke `create-calculated-column.js`; it is a fail-closed guard for + legacy callers. +- Cross-entity fields must use a supported formatted lookup annotation or + bounded chained fetch as documented in `data-performance.md`. +- For a hot list field that cannot use either path, mark it + `external-projection-required` and omit it from this mutation run. The user + may create a formula column in Power Apps or supply another server-owned + projection outside this PR, then rerun Step 4a to validate and reuse it. +- Never create an ordinary copied field without an explicit refresh owner. -1. Parse the `### Cross-entity Reads` table. Each row has columns: `Calc column | On table | Type | Resolves | Driven by`. -2. **Run AFTER all regular columns + relationships from Step 5b have been created** (the formula chain references real columns + lookups; creating the calc column before its dependencies returns HTTP 400 from Dataverse). -3. **Per row**, invoke the helper: - - ```bash - node "${PLUGIN_ROOT}/scripts/create-calculated-column.js" \ - --table \ - --column \ - --type \ - --formula "" \ - --display "" \ - --solution '' - ``` - -4. **One at a time, sequentially.** Calc-column creation is metadata mutation — same concurrency rule as table creation. Print `✓ ` after each success. -5. **On failure** — the helper script prints the OData error inline. Common cases: - - `400 — formula references unknown attribute` → the relationship or column the formula needs has not been created yet. Verify Step 5b finished cleanly before retrying. - - `400 — calculated formula not allowed on this navigation` → the dotted path tries to traverse 1:many or M:N. The architect should have caught this at Step 6a; flag in summary, skip the row, continue. - - Surface non-recoverable errors to the user with the offending row, then proceed to the next row. Do NOT abort the whole step on one bad row. - -6. After all rows are processed, the publish step (Step 6b) below picks up calc columns automatically — no extra publish call needed. +Reference: +https://learn.microsoft.com/power-apps/developer/data-platform/specialized-columns ### Step 5d — Create alternate keys for unique business identifiers @@ -595,7 +756,8 @@ This step exists because of the runtime constraint documented at [`shared/refere node "${CLAUDE_SKILL_DIR}/../../scripts/dataverse-request.js" POST \ "EntityDefinitions(LogicalName='
')/Keys" \ --body '' \ - --solution '' + --solution '' \ + --tenant-id '' ``` Body skeleton: @@ -609,11 +771,12 @@ Body skeleton: } ``` -**Pre-flight each key before POST** so re-runs are idempotent: +**Pre-flight each key before POST** so re-runs are idempotent. A key can only pre-exist on a table that already existed at Step 4, so for a table created in this run, skip straight to the POST. Otherwise read `Keys` from that table's Step 4 snapshot. Query it directly only when the snapshot did not cover that table: ```bash node "${CLAUDE_SKILL_DIR}/../../scripts/dataverse-request.js" GET \ - "EntityDefinitions(LogicalName='
')?\$select=LogicalName&\$expand=Keys(\$select=SchemaName,KeyAttributes,EntityKeyIndexStatus)" + "EntityDefinitions(LogicalName='
')?\$select=LogicalName&\$expand=Keys(\$select=SchemaName,KeyAttributes,EntityKeyIndexStatus)" \ + --tenant-id '' ``` | Existing key state | Action | @@ -637,7 +800,7 @@ Add alternate keys to `.datamodel-manifest.json` for the table: **Print before starting:** > "→ Generating TypeScript services for tables via `npx power-apps add-data-source` (sequential). Print '✓
Service.ts' after each." -For each table the app will use (regardless of reuse/extend/create), generate the TS layer from the app root. The CLI reads the environment ID from `power.config.json`; pass the environment URL resolved earlier in the skill: +For each table in `SERVICE_REQUIRED_TABLES` (regardless of reuse/extend/create), generate the TS layer from the app root. Do not derive this list from Creation Order alone because reused tables are intentionally absent from creation tiers. The CLI reads the environment ID from `power.config.json`; pass the environment URL resolved earlier in the skill: ```bash npx power-apps add-data-source --api-id dataverse --org-url --resource-name @@ -645,6 +808,12 @@ npx power-apps add-data-source --api-id dataverse --org-url --resource- Run **one at a time — sequentially**, not in parallel. The Power Apps CLI writes `src/generated/connectorSchemas.ts` and other generated files non-atomically; concurrent invocations corrupt them. +After generation, verify each required table appears in `power.config.json` `databaseReferences.default.cds.dataSources` and that a matching file exists in `src/generated/services/`. If any reused or custom table is missing, STOP before screen generation: + +```text +BLOCKED: required Dataverse service missing for . Schema action=; app usage=. +``` + ### Step 6b — Publish customizations **Print before starting:** @@ -655,7 +824,8 @@ Only after **every** Step 5 metadata POST and **every** Step 6 `npx power-apps a ```bash node "${CLAUDE_SKILL_DIR}/../../scripts/dataverse-request.js" POST \ "PublishXml" \ - --body "{\"ParameterXml\":\"cr123_table1cr123_table2\"}" + --body "{\"ParameterXml\":\"cr123_table1cr123_table2\"}" \ + --tenant-id '' ``` Build the entity list from all tables that were **created or extended** in Steps 4–5. Skip reused-as-is tables — they don't need republishing. @@ -664,15 +834,16 @@ If the publish call returns a non-2xx status, report the error and stop — do n ### Step 6c — Verify tables exist -For each created or extended table, confirm it is queryable after publish: +Confirm every created or extended table is queryable after publish with **one** filtered query, not one request per table: ```bash node "${CLAUDE_SKILL_DIR}/../../scripts/dataverse-request.js" GET \ - "EntityDefinitions(LogicalName='
')?\\$select=LogicalName,DisplayName" + "EntityDefinitions?\$select=LogicalName,DisplayName&\$filter=LogicalName eq '' or LogicalName eq ''" \ + --tenant-id '' ``` -- **200** → confirmed. -- **404** → table missing after publish — report and stop. +- **Every expected name present in `value[]`** → confirmed. +- **Any expected name missing** → that table did not survive publish — report which ones and stop. ### Step 6d — Write `.datamodel-manifest.json` @@ -808,6 +979,8 @@ Environment : Tables reused : Tables extended: Tables created : +Adapted : +Deferred : Generated services: src/generated/services/
Service.ts × N diff --git a/plugins/mobile-apps/skills/add-dataverse/references/dataverse-reference.md b/plugins/mobile-apps/skills/add-dataverse/references/dataverse-reference.md index ec00ff501..720598c56 100644 --- a/plugins/mobile-apps/skills/add-dataverse/references/dataverse-reference.md +++ b/plugins/mobile-apps/skills/add-dataverse/references/dataverse-reference.md @@ -239,10 +239,14 @@ The lookup POST will 404 unless **both** endpoint tables already exist. Ensure: Lookup properties (`_fieldname_value`) are **read-only**. To set a relationship, use the **single-valued navigation property** with `@odata.bind`: ```typescript -// CORRECT - Use @odata.bind for lookup fields -const newRecord: any = { +// CORRECT for generated services: copy the exact key from the generated model. +type CreateFields = Pick< + Parameters[0], + 'prefix_name' | 'prefix_parentaccountid@odata.bind' | 'prefix_status' +>; +const newRecord: CreateFields = { 'prefix_name': 'My Record', - 'prefix_ParentAccount@odata.bind': `/accounts(${accountGuid})`, + 'prefix_parentaccountid@odata.bind': `/accounts(${accountGuid})`, 'prefix_status': 100000000 }; @@ -250,6 +254,12 @@ const newRecord: any = { { '_prefix_parentaccountid_value': accountGuid } // May fail on create ``` +Raw Dataverse Web API examples often use a case-sensitive navigation-property schema name. Generated Power Apps services instead expose their accepted write key in `src/generated/models/Model.ts`. For generated services, that model declaration is authoritative; never convert it to PascalCase or hide an unverified key inside `Record`. + +For filtering across lookups, avoid `lookupNavigation/relatedColumn eq ...` in generated mobile-service options. Resolve the related row through its generated service, then open the source generated model and copy the exact declared read lookup property corresponding to the planned lookup logical column (for example `_new_systemuserid_value` for `new_systemuserid`). Never substitute generic placeholders such as `_lookup_value` or `_systemuserlookup_value` into generated code. + +Dynamic route IDs used in Dataverse calls must be normalized with the shared `normalizeDataverseGuid` helper from `@/utils`. Do not generate an inline RFC UUID regex: Dataverse sequential GUIDs may not contain RFC version bits even though their hexadecimal `8-4-4-4-12` structure is valid. + The `@odata.bind` value must be an entity set path with the GUID: `/()` ## Alternate Keys (Metadata API) - CRITICAL diff --git a/plugins/mobile-apps/skills/create-mobile-app/SKILL.md b/plugins/mobile-apps/skills/create-mobile-app/SKILL.md index 6e40fda1b..f9620c34f 100644 --- a/plugins/mobile-apps/skills/create-mobile-app/SKILL.md +++ b/plugins/mobile-apps/skills/create-mobile-app/SKILL.md @@ -1312,6 +1312,12 @@ Build two lists from the classification: **Sanity check before writing anything:** if any folder has children but no `index.tsx` row in the Screen Map, STOP and report: `BLOCKED: folder app/(app)// has children () but no index.tsx row in the Screen Map. The screen-planner must emit an index.tsx row for every folder.` This catches a planner mistake that would render the folder unreachable from the outer tab. +Normalize every Screen Map file to its Expo route (strip `.tsx`, collapse trailing `/index`, preserve dynamic segments). If two files normalize to the same route, STOP before writing layouts. In particular, reject `/[id].tsx` together with `/[id]/.tsx`; move the detail contract to `/[id]/index.tsx`. + +```text +BLOCKED: duplicate Expo route from and . Use [id]/index.tsx when a dynamic detail route owns child screens. +``` + #### Step 10b.2 — Write per-folder inner `_layout.tsx` files (if any folders exist) For each entry in the Inner stacks list, create the folder if missing and write `app/(app)//_layout.tsx` with this template: @@ -1334,7 +1340,7 @@ Rules: - `headerShown: false` at the Stack level — each screen sets its own header inline via `` at the top of its component (the Expo Router idiom). - `` is required — without it, the folder root won't render. - `presentation: 'modal'` and `presentation: 'formSheet'` come from the Screen Map's Presentation column. Skip the `options` prop entirely for `default` presentation. -- `name` for `[id].tsx` is literally `[id]` (with brackets). +- `name` for `[id].tsx` is literally `[id]` (with brackets). When `[id]` owns child routes, create `/[id]/_layout.tsx` with `` and child entries; do not register both `[id].tsx` and a `[id]/` folder. - Folder name in the function name is PascalCase (e.g. `InspectionsLayout`). **Why this must run BEFORE Step 11:** screen-builders write their files in parallel, multiple builders may target the same folder, and any of them creating `_layout.tsx` would race. The orchestrator owns these files. @@ -1796,6 +1802,14 @@ Common wave-gate repair classes to batch instead of fixing line-by-line: **After all waves return and the last wave gate is clean**, run one final `npx tsc --noEmit` before Step 12 to catch cross-screen issues that only appear when all screens exist. If it fails, use the same consolidated batch-repair flow. +Then run the canonical route-contract gate from the app root: + +```bash +node "${CLAUDE_SKILL_DIR}/../../scripts/check-routes.js" +``` + +This gate is required even when TypeScript passes. It detects duplicate normalized routes, `[id].tsx` plus `[id]/.tsx` file/folder collisions, and sender/destination parameter drift. If it fails, repair the affected route files or re-spawn their screen builders with the consolidated findings, then rerun once. Do not continue to Step 11.4 or start Metro while route findings remain. + **Sticky tsc/build error policy (run-level).** The first time a `tsc` or `npm run build` failure surfaces in this run, ask the user once: > "tsc found error(s) in . Patch + continue, or stop and let me investigate?" diff --git a/plugins/mobile-apps/skills/edit-app/SKILL.md b/plugins/mobile-apps/skills/edit-app/SKILL.md index 01ad1200f..dda4cc010 100644 --- a/plugins/mobile-apps/skills/edit-app/SKILL.md +++ b/plugins/mobile-apps/skills/edit-app/SKILL.md @@ -447,6 +447,7 @@ Before spawning builders: Navigation/layout algorithm: - Read the approved `## Screens` Screen Map and Navigation Contracts. +- Normalize every target file to its Expo route before editing. Reject duplicate normalized routes, especially `/[id].tsx` together with `/[id]/.tsx`; move the detail contract to `/[id]/index.tsx` before builders run. - For every new route, create the parent folder and inner `_layout.tsx` when the route is nested. - For modal/formSheet/detail routes, add the correct `` in the owning folder layout. - For tab/root changes, patch only the route list in `app/(app)/_layout.tsx`; preserve auth/provider logic and imports not related to route registration. diff --git a/plugins/mobile-apps/skills/setup-datamodel/SKILL.md b/plugins/mobile-apps/skills/setup-datamodel/SKILL.md index 0c34bebae..86106df82 100644 --- a/plugins/mobile-apps/skills/setup-datamodel/SKILL.md +++ b/plugins/mobile-apps/skills/setup-datamodel/SKILL.md @@ -140,7 +140,12 @@ Arguments: `/add-dataverse` creates tables in tier order, runs `npx power-apps add-data-source --api-id dataverse --org-url --resource-name ` per table from the app root, publishes customizations, writes `.datamodel-manifest.json`, and type-checks. Wait for it to return before Phase 6. -**Calculated columns from the screen plan** — if `## Data Model` in the plan includes a `### Cross-entity Reads (auto-derived from screen plan)` subsection (the `data-model-architect`'s Step 6a addendum, approved at the Gate 1 addendum during planning), `/add-dataverse` Step 5c creates each row as a calculated column on the parent table via `scripts/create-calculated-column.js`. No additional action needed in this skill — `/add-dataverse` handles it. If the subsection is absent, Step 5c is silently skipped. See [`shared/references/data-performance.md` § Cross-entity Reads](${PLUGIN_ROOT}/shared/references/data-performance.md#cross-entity-reads) for why this matters. +**Cross-entity reads from the screen plan** — the approved +`### Cross-entity Reads` subsection contains formatted lookups, bounded chained +fetches, or `external-projection-required` blockers. `/add-dataverse` never +synthesizes calculated/formula definitions through code. A user-supplied, +maker-created computed column is validated as an existing dependency during +reconciliation before it can be reused. Skip if Phase 2 chose Path C (no Dataverse).