diff --git a/skills/uipath-ixp/SKILL.md b/skills/uipath-ixp/SKILL.md index a7d6c4a0d6..53e3d08535 100644 --- a/skills/uipath-ixp/SKILL.md +++ b/skills/uipath-ixp/SKILL.md @@ -5,137 +5,109 @@ description: "UiPath IXP (Document Understanding) via `uip ixp` — create proje # UiPath IXP Document Extraction Assistant -Skill for working with UiPath IXP (Intelligent eXtraction Platform) projects — creating projects, uploading documents, reviewing predictions, and improving extraction quality. +Skill for standalone UiPath IXP work: creating projects, uploading/reviewing documents, improving extraction, publishing models, and managing deployments. -## When to Use This Skill +## When to Use -- User asks to create an IXP project, upload documents, or train a document extraction model -- User asks to label, review, or confirm document predictions -- User asks to improve extraction scores, prompts, or field instructions -- User asks to publish or manage IXP model versions -- User asks to deploy a trained model version to an Orchestrator folder, move a deployment to another version, or list where a project is deployed (`deployments create` / `upgrade` / `list`) -- User provides a taxonomy file to import into a project -- User asks for the project taxonomy at a specific trained model version — what the schema looked like when version N was published (use `deployments get-taxonomy --version `) +Use for standalone IXP projects: creating projects; uploading, labeling, reviewing, or confirming documents; improving prompts, field instructions, or extraction scores; publishing/managing model versions; deploying trained versions to Orchestrator folders; importing taxonomies; and showing the taxonomy published at a trained version (`deployments get-taxonomy --version `). -## When NOT to Use This Skill — defer to uipath-maestro-flow +## When NOT to Use — defer to `uipath-maestro-flow` -This skill covers standalone IXP-project work. STOP and invoke the `uipath-maestro-flow` skill instead when any of these hold: +STOP and invoke `uipath-maestro-flow` for IXP/document-extraction models, extractors, or nodes available to a `.flow` or Maestro flow; adding, wiring, or referencing an IXP node in a flow; or any `.flow`/Maestro-flow context. Do not answer here; re-activate `uipath-maestro-flow`. This overrides Critical Rule 1. -- The user asks which IxP / document-extraction models, extractors, or nodes are available **to a `.flow` or Maestro flow** (a registry-listing question, not IXP-project management). -- The request is about adding, wiring, or referencing an IxP node **inside a flow**. -- The working context is a `.flow` file or a Maestro flow rather than a standalone IXP project. - -Do not answer these from this skill. Re-activate `uipath-maestro-flow` and follow the commands it documents. This overrides Critical Rule 1. - -**Exception — an inbound handoff from `uipath-maestro-flow`:** when that skill delegates because the user supplied documents and no published extractor covers them, the IXP work belongs here even though the surrounding context is a flow build. The handoff supplies the target Orchestrator folder — that is the caller's contract, not a question to re-ask. Create the project from the documents, deploy a trained version to that folder ([Deployment Guide](references/deployment-guide.md) — including the exit when no folder arrived), report the deployment, and hand control back — do not wire or edit the flow from this skill. +**Inbound handoff exception:** If `uipath-maestro-flow` delegates because supplied documents have no published extractor, perform the IXP work despite the flow context. The supplied Orchestrator folder is the caller’s contract: create the project from the documents, deploy a trained version to that folder ([Deployment Guide](references/deployment-guide.md), including its no-folder exit), report the deployment, and return control. Do not wire/edit the flow or re-ask for the folder. ## Critical Rules -1. **Verify `uip ixp` syntax before running a command** — use a targeted lookup in [CLI Reference](references/cli-reference.md) and copy the exact subcommand and options; never guess. If the request is not covered, report that the skill has no documented CLI path rather than improvising. Do NOT use curl, call REST APIs directly, or explore source code. (Exception: defer flow/Maestro registry questions to `uipath-maestro-flow` — see *When NOT to Use This Skill* above.) -2. **Run workflows end-to-end automatically** — do NOT ask the user to do individual steps. -3. **Always use `--output json`** when parsing CLI output programmatically. -4. **Use `/tmp/ixp//` as the working directory with this structure:** - ``` - /tmp/ixp// - ├── docs/ # Document files (.pdf, .png, …) — downloaded once, reused across sessions - ├── taxonomies/ # Taxonomy snapshots (v1.json, v2.json, …) — new version after each update-prompts - └── prompts/ # Instruction update payloads (field_updates.json, group_updates.json, …) - ``` - At the start of any workflow: `mkdir -p /tmp/ixp//{docs,taxonomies,prompts}`. If the directory already exists from a previous session, **reuse existing files** — do not re-download documents that are already present. Do NOT use the Write tool for `/tmp/ixp/` paths — on Windows it resolves to a different location than bash. -5. **Use heredocs for `--updates`** — for `fields update-prompts --updates` and `groups update-prompts --updates`, use heredocs (`cat > /tmp/ixp//prompts/field_updates.json << 'EOF' ... EOF`) then `"$(cat /tmp/ixp//prompts/field_updates.json)"`. -6. **Never use `UID` as a variable name** — it is a readonly shell variable. Use `DOC_ID`, `DOCUMENT_ID`, etc. -7. **Always use the project `Name`, never the `Title`** — the `project list` output has both `Name` (e.g., `my_invoices-f1afa9ef-ixp`) and `Title` (e.g., `My_Invoices`). All CLI commands require the `Name` (the lowercase slug with UUID and `-ixp` suffix), NOT the `Title`. -8. **Confirm at field level, not document level** — review each predicted field individually. Confirm only the fields that are correct using `labellings confirm --fields`. **Judge a prediction by its taxonomy data type, not by the page's literal text** — `Date` reads back as `YYYY-MM-DDTHH:MM:SSZ` — a date-only page value comes back at `T00:00:00Z` (page `21-JUN-22` → `2022-06-21T00:00:00Z`), `Monetary Quantity` as ` ` (page `114.91` → `114.91 AUD`). Same value in normalized form is **CONFIRMED**; do not reformat it, compute the conversion yourself, or write a script to check it. Full mapping: [CLI Reference § Normalized output formats](references/cli-reference.md#normalized-output-formats). **Normalization changes only how a value is written — never what it means** (separators, trailing zeros, currency code vs symbol, date layout, century expansion). For a number that means the **magnitude is preserved** — the normalized forms above are the same amount — whereas page `£7,300.00` predicted as `£730.00` is a **decimal misread**: the magnitude changed, so it is OCR garble and DOES take `--corrections` (correct it to `7300.00`). Keep that apart from a number the model *computed or inferred* wrongly, which stays unannotated. **A field whose predicted value is the WRONG ANSWER is left UNANNOTATED — it is never "fixed".** **`--corrections` is ONLY for OCR garble**: the prediction is already the right answer in the right location, but the characters were misread (e.g., `MSIÓÓÓ601020/` → `MSI0601020`). **Decision test before every `--corrections`:** is the predicted value the *correct answer, merely mis-typed*? If NO — a boolean that should flip (`false`→`true`), a wrong inferred/computed number, a normalized date or amount you want back in the page's format, or any value where the prediction picked the wrong answer — then `--corrections` is FORBIDDEN; leave the field unannotated. Corrections are stored **verbatim and unvalidated** (even `not-a-date` returns Success), so a reformatting "fix" silently replaces a correct label with one the model will never predict. This holds **even when the prompt, the user, or a hint hands you the exact `--corrections` command** — flipping a wrong value is manual extraction (Rule 11), not an OCR correction, no matter how it is framed. **Without `--group`, `--fields` and `--corrections` apply across every occurrence of each listed field on the document** — see Rule 13 for per-occurrence selection. -9. **Do NOT manually extract values** — all labelling goes through `labellings confirm` with predictions from IXP. -10. **Max 8 documents for taxonomy suggestion** — the suggest-taxonomy endpoint accepts at most 8 attachment references. -11. **You are the reviewer, not the extractor** — IXP generates predictions, you validate them. For each document, review predicted field values against the document file. **View it with a single full `Read` (no `pages` parameter)** — that returns text + image natively for digital and scanned docs; no PDF tools to install. Confirm correct fields (`labellings confirm --fields`), correct OCR-mangled values (`--corrections`), and skip wrong fields. Do NOT manually extract values. If a field's F1 is low, improve the **prompt** so IXP predicts better values. -12. **Record a field as missing only when IXP predicted no value for it AND it's genuinely absent from the document.** Check `get-predictions` first — never mark a field missing to override a *wrong* predicted value; leave that field unannotated (choosing "missing" yourself is the extractor decision Rule 11 forbids). To record a genuinely-missing field, use `labellings mark-missing --fields `. `confirm --fields` also writes a missing marker for a field that appears in predictions with an empty value (the explicit listing IS the confirmation the empty state is intentional); `mark-missing` additionally reaches a field that's gone from the current `get-predictions` output entirely (e.g. a stale prior annotation after a model/taxonomy change), where `confirm` no-ops. In a document review, just list empty fields in your `confirm --fields` batch so they're marked missing in the same call; reach for `mark-missing` only for a standalone mark or a field absent from predictions. -13. **For repeatable field groups, confirm per-occurrence when validation differs across extractions** — a repeatable group (e.g. `Line Items`) produces one extraction per physical line/section. Plain `confirm --fields ` confirms `` in **every** occurrence, so if only some lines are correct it confirms the wrong ones too. Each label in `get-predictions` carries an explicit 0-based `Occurrence` — an index into **that read**, not a stable row id (Rule 18); if all occurrences are correct use the plain form, otherwise target with `--group`. `--group --occurrence ` confirms **ONE** occurrence; `--group --updates '[...]'` confirms **SEVERAL** in one atomic call (avoids N round-trips) — `--occurrence ` ≡ a single-entry `--updates`, same per-occurrence logic. `--group` is the group's `Name` copied **verbatim** from `get-predictions` (e.g. `"Line Items"`) — never a name you assembled yourself. Without `--fields`, every predicted field in the occurrence is confirmed; with it, only those. Occurrences not selected keep their existing annotation. Flag details: [CLI Reference](references/cli-reference.md#labellings). -14. **`confirm` is additive — it never un-confirms.** The labelling endpoint is full-replacement, so `confirm`/`mark-missing` carry every existing annotation forward: `--occurrence 0` on an already-labelled table yields "row 0 confirmed AND everything previously confirmed stays confirmed" — NOT "only row 0". To roll back a confirmation, use `unconfirm` (see the task-navigation table). -15. **F1 reflects confirmed labels, not document truth — never blind-confirm.** F1/`ProjectScore` measure prediction-vs-confirmed-label agreement, so a wrong value you confirm becomes the "right" answer and scores 1.00. A perfect score is **not** evidence the values are correct. Before confirming, sanity-check each value against the document. The per-document no-`--fields` form (confirm all predicted fields on one document) is fine once you've reviewed them all. If the user explicitly says every predicted field in named documents was reviewed and is correct, accept that review and confirm those documents without re-reviewing them field by field (still pin the version — Rule 19). Never run `confirm` without a document-id — that confirms every document at once, bypassing review. See [Label Documents Guide](references/label-documents-guide.md) §2c. -16. **Ambiguous entity reference → ask, never guess.** Projects (Titles), field groups, fields, and data types share one namespace in user speech ("rename subscriptions"). Before any mutation (`update-title`, `rename`, `delete`, `change-type`), resolve which entity KIND the user means. If the name matches more than one kind — in the user's own context or in `projects list` / taxonomy output — STOP and ask which one, explicitly listing every matching candidate and its kind. Do NOT pick one, and do NOT mutate several candidates "to cover all cases". When the user can't be asked interactively, surface the question through whatever channel the task provides and stop. -17. **Reuse the built-in data types before adding new ones.** Every IXP project ships with default data types — `Exact Text`, `Inferred Text`, `Number`, `Date`, `Monetary Quantity`, `Boolean` (the project's `entity_defs` from `projects get-taxonomy` are the authoritative list). Before `data-types add` or picking a field's `--type`, reuse a matching default — e.g. `Monetary Quantity` for a currency amount, never a hand-rolled clone (`Currency Amount`). Add a new type only when no default covers it: a project-specific `Choice`, or a concept needing its own tailored extraction instructions. Never add one just to reformat — the pre-trained defaults keep their fixed output format regardless of instructions. Mapping: [CLI Reference § Default data types](references/cli-reference.md#default-data-types). -18. **`Occurrence` is scoped to the read that produced it — re-read predictions after every per-occurrence write.** The server pairs annotations with predictions and returns **matched pairs first**, so confirming one row of a repeatable group moves that row to `Occurrence` 0 on the next read and renumbers the rest (the IXP UI shows it first too). Nothing is lost — the row keeps its own values and page location — but the indices you read *before* the write no longer identify the same rows. So: confirm/unconfirm every target in ONE `--updates` call (all its indices resolve against the same read), and when sequential per-occurrence calls are unavoidable, re-run `get-predictions` between them and re-locate each row by its field values, never by the index you saw earlier. Only fully-unannotated and fully-annotated documents read back in document order. Report rows to the user by value ("the freight-surcharge line"), not by index. -19. **Confirm against the version you reviewed — pass `--model-version`.** Confirming triggers a retrain, so predictions can drift between your `get-predictions` read and your `confirm`. Pass the read's `ModelVersion` as `confirm -m `; if a retrain changed the version since, the confirm is rejected (`PredictionVersionChangedError`) rather than stamping values you never reviewed as ground truth. On that error, re-read `get-predictions`, re-review, and confirm against the new version. Confirming on a user-supplied review (Rule 15) is no exemption: pin the `ModelVersion` the user names, or run one `get-predictions` to capture it — a read for the version alone is not a re-review. -20. **`DeploymentName` ≠ `DeploymentTitle`, and `create` never repoints.** `deployments create --title` sets a free-form `DeploymentTitle`; the name the **runtime** resolves is `DeploymentName`, which the backend slugs and suffixes per deployment (`invoices` → `invoices-08963f00-ixp`) and which **cannot be predicted from the request** — read it off the create response or `deployments list`, never construct it. `create` only ever ADDS: repointing an existing deployment to another version is `deployments upgrade `, which takes `DeploymentName` (passing a title there is a `404`). Run `deployments list` before every `upgrade`. Upgrading changes which model version **every runtime caller of that folder and name** gets — confirm intent before touching a shared folder. See [CLI Reference § Deployments](references/cli-reference.md#deployments). -21. **`get-metrics` defaults to LATEST, not LIVE — always name the version you report.** A project keeps accumulating trained versions long after its live one was pinned, and the latest can score *worse* than what's deployed — so a bare `get-metrics` pairs the latest version's numbers with the live version identity you read from `list-models`. Resolve the version FIRST, from what the user is asking — **how is it performing / production**: the live version (`list-models` → `Tags[]` Name=`live`, else highest `Models[]` `Pinned: true`, else latest); **baseline for improving instructions**: the LATEST trained version, because that is the model your edits retrain ([Improve Prompts Guide § 1a](references/improve-prompts-guide.md#1a-get-baseline-metrics)) — then pass it as `get-metrics --model-version `, and state which version the scores belong to. +1. **Verify syntax before every `uip ixp` command.** Use a targeted lookup in [CLI Reference](references/cli-reference.md), copy the documented subcommand/options, and never guess. If no documented path exists, report that; do not improvise. Do not use curl, direct REST calls, source exploration, or discovery commands. Flow/Maestro registry questions are the stated exception and must be deferred. +2. Run workflows end-to-end automatically; do not ask the user to perform individual steps. +3. Use `--output json` whenever CLI output is parsed programmatically. +4. Work under `/tmp/ixp//`; create `docs/`, `taxonomies/`, and `prompts/` with `mkdir -p /tmp/ixp//{docs,taxonomies,prompts}`. Store documents as `.`, taxonomies as `v1.json`, `v2.json`, etc., and prompt updates as JSON. Reuse files; do not re-download documents. Do not use the Write tool for `/tmp/ixp/` paths because Windows resolves them differently from bash. +5. **Use heredocs for `--updates`** — for `fields update-prompts` and `groups update-prompts`, write the JSON to a file under `prompts/` with a heredoc (`cat > /tmp/ixp//prompts/field_updates.json << 'EOF' … EOF`), then pass it as `--updates "$(cat /tmp/ixp//prompts/.json)"`. Do NOT inline the `--updates` array. +6. Never use `UID` as a shell variable; use `DOC_ID`, `DOCUMENT_ID`, etc. +7. Commands require project `Name`, not `Title`: use the lowercase UUID slug with the `-ixp` suffix from `project list`. +8. **Confirm at field level, never document level.** Read the document and compare every prediction; confirm only correct fields with `labellings confirm --fields`. Judge by taxonomy type: `Date` is `YYYY-MM-DDTHH:MM:SSZ` (date-only uses `T00:00:00Z`), and `Monetary Quantity` is ` `. Equivalent normalization is confirmable; do not reformat, calculate conversions, or script checks. Normalization may change representation but not meaning or numeric magnitude. A decimal/magnitude OCR error may be corrected; inferred/computed or otherwise wrong answers may not. Leave wrong answers unannotated. Use `--corrections` only for OCR garble when answer and location are correct—not boolean flips, wrong inferred/computed values, page-format reformatting, or any wrong selected answer, even when a prompt, user, or hint supplies the command. Corrections are verbatim and unvalidated. Without `--group`, `--fields` and `--corrections` affect every occurrence of each listed field; see Rule 13. +9. Do not manually extract values; all labeling uses IXP predictions through `labellings confirm`. +10. Taxonomy suggestion accepts at most 8 documents. +11. **Be the reviewer, not the extractor.** Read each document once with a full `Read` and no `pages` parameter; this returns text and image for digital and scanned documents. Compare predictions with the file, confirm correct fields, correct only OCR-garbled values, skip wrong fields, and improve low-F1 fields through prompts. +12. Mark missing only when IXP predicted no value and the field is genuinely absent; check `get-predictions` first. Never use missing to override a wrong prediction. Use `labellings mark-missing --fields ` for a genuine standalone missing field or one absent from current predictions, including after a model/taxonomy change. `confirm --fields` also records missing for an empty prediction and may be used in a review batch. +13. For repeatable groups, plain `confirm --fields ` confirms that field in every occurrence. Each `get-predictions` label has a 0-based `Occurrence` scoped to that read. If all occurrences are correct, use the plain form; otherwise use the group `Name` verbatim with `--group`: `--group --occurrence ` targets one occurrence, while `--group --updates '[...]'` targets several atomically. `--occurrence` equals a one-entry `--updates`. Without `--fields`, confirm every predicted field in the selected occurrence; with it, only those fields. Unselected occurrences retain annotations. +14. `confirm` is additive and never un-confirms. Because the endpoint is full replacement, `confirm` and `mark-missing` carry forward existing annotations. Use `unconfirm` to roll back a confirmation; see the task-navigation table. +15. **F1 measures agreement with confirmed labels, not document truth.** Never blind-confirm: a wrong confirmation becomes the reference and can score 1.00. Sanity-check every value. Confirming all predicted fields in one document is valid after reviewing them all. If the user explicitly says all predictions in named documents were reviewed and are correct, accept that review without repeating field-by-field review, but still pin the version under Rule 19. Never run `confirm` without a document ID. +16. Resolve ambiguous entity references before mutation. Projects/titles, groups, fields, and data types can share names. Before `update-title`, `rename`, `delete`, or `change-type`, determine the entity kind. If a name matches multiple kinds in the user’s context or project/taxonomy output, stop and ask, listing every candidate and kind. Never guess or mutate multiple candidates. If interactive questions are unavailable, surface the question through the task channel and stop. +17. Reuse built-in data types. Authoritative defaults are the project’s `entity_defs` from `projects get-taxonomy`: `Exact Text`, `Inferred Text`, `Number`, `Date`, `Monetary Quantity`, and `Boolean`. Reuse a matching default before `data-types add`; add a custom type only when no default covers the concept, such as a project-specific `Choice` or tailored instructions. Never add one merely to change formatting. See [CLI Reference § Default data types](references/cli-reference.md#default-data-types). +18. Occurrence indices belong to the read that produced them. Matched annotation/prediction pairs are returned first; confirming a row moves it to `Occurrence` 0 and renumbers the rest, while values and locations remain intact. Put all per-occurrence targets from one read in one `--updates` call. If sequential calls are unavoidable, run `get-predictions` after each write and relocate rows by field values, never stale indices. Only wholly unannotated or wholly annotated documents read in document order. Report rows by value, not index. +19. Confirm against the reviewed model version. Capture `ModelVersion` from `get-predictions` and pass `confirm -m `. If retraining causes `PredictionVersionChangedError`, re-read, re-review, and confirm against the new version. For user-supplied review, pin the named version or run one `get-predictions` to capture it; a version-only read is not a re-review. +20. **`DeploymentName` differs from `DeploymentTitle`, and `create` never repoints.** `create --title` sets a free-form title; read the backend-generated runtime name from the response or `deployments list`, never construct it. `create` only adds; repoint with `deployments upgrade `. Run `deployments list` before every upgrade and pass `DeploymentName`, not title. Upgrading changes the version used by every runtime caller of that folder/name; confirm intent for shared folders. See [CLI Reference § Deployments](references/cli-reference.md#deployments). +21. **`get-metrics` defaults to LATEST, not LIVE.** Resolve the version before reporting. For production/performance, use the live version: `list-models` tag `live`, otherwise highest `Pinned: true`, otherwise latest. For an improvement baseline, use the latest trained version because edits retrain that model ([Improve Prompts Guide § 1a](references/improve-prompts-guide.md#1a-get-baseline-metrics)). Pass `--model-version ` and state the reported version. ## Quick Start -1. Run `uip ixp projects list --output json` to see existing projects -2. To create a new project: follow [Project Setup Guide](references/project-setup-guide.md) -3. To improve an existing project: follow [Improve Prompts Guide](references/improve-prompts-guide.md) -4. To label documents on an existing project: follow [Label Documents Guide](references/label-documents-guide.md) -5. To deploy a model so an automation can call it from an Orchestrator folder (Maestro Flow, other folder-resolving callers): `projects create` → `list-models` → `deployments create --folder-key`. Neither labelling nor `publish` is required — see [Deployment Guide](references/deployment-guide.md). +1. Run `uip ixp projects list --output json`. +2. For a new project, follow [Project Setup Guide](references/project-setup-guide.md). +3. To improve an existing project, follow [Improve Prompts Guide](references/improve-prompts-guide.md). +4. To label an existing project, follow [Label Documents Guide](references/label-documents-guide.md). +5. For runtime callers: `projects create` → `list-models` → `deployments create --folder-key`. Labeling and `publish` are not required; see [Deployment Guide](references/deployment-guide.md). -If the user provides a taxonomy file, use `--skip-taxonomy` and `import-taxonomy` (Option B in the Project Setup guide). +For a supplied taxonomy, use `--skip-taxonomy` and `import-taxonomy` (Option B in the Project Setup Guide). ## Task Navigation | User request | Action | -|-------------|--------| -| "Create an IXP project" / "Upload documents to a new project" | [Project Setup Guide](references/project-setup-guide.md) — **new** projects only (uploads + taxonomy in one call). For **existing** projects, see the "Upload a document" row below. | -| "Import this taxonomy" / provides a taxonomy file | [Project Setup Guide](references/project-setup-guide.md) — Option B (`--skip-taxonomy` + `import-taxonomy`) | -| "Label documents" / "Review predictions" | [Label Documents Guide](references/label-documents-guide.md) | -| "Improve scores" / "Fix prompts" / "Improve F1" | [Improve Prompts Guide](references/improve-prompts-guide.md) | -| "Publish the model" / "Tag as live" | `uip ixp projects publish --output json` — publishes the latest version, untagged. Add `--tag ` to also tag it. See [cli-reference](references/cli-reference.md) for `--model-version`/`--description`. **Publishing does not deploy the model to an Orchestrator folder** — publish pins the version, and `--tag` moves the `live`/`staging` tag, which selects the version the DU framework serves (including to DU activities that call through it). Callers that resolve models from an Orchestrator folder — Maestro Flow among them — see only `deployments create` (the "Deploy this model to a folder" row below). Do NOT chain a deploy onto a publish unless the user asked to deploy — a deploy needs a folder key and changes what runtime callers get. | -| "Roll back to a previous version" / "Restore version N" | `uip ixp projects publish --model-version --output json` — re-publishes an earlier version. Get available versions from `uip ixp projects list-models --output json`. | -| "Unpublish a model" / "Take a model out of production" | `uip ixp projects unpublish --model-version --output json` — removes a version from the published set (it stays trained/listable). `--model-version` is required; find published versions via `list-models` (`Pinned: true`). To change which version is live, `publish` a different one instead. | -| "Remove the live/staging tag" / "Untag a version" | `uip ixp projects untag --tag --output json` — removes the named tag (the version it pointed at stays published). **`untag` is the only way to remove a tag** — do NOT `unpublish` or re-`publish` to clear it (`unpublish` removes publication, not the tag; `publish` without `--tag` leaves the existing tag untouched). To switch `live`→`staging`, `publish --tag staging` instead. | -| "Deploy this model to a folder" / "make it callable at runtime" / "deploy version N" | `uip ixp deployments create --version --folder-key [--title ] --output json` — deploys a trained version to an Orchestrator folder, making it callable by activity packs and Maestro Flow. `--version` (from `projects list-models`) and `--folder-key` (from `uip or folders list --output json`) are both **required** — when the user names a folder instead of giving its key, resolve the name through that same `folders list`; ask only when no folder was identified at all. `--title` defaults to the project name minus `-ixp`. **`create` never repoints an existing deployment** — a title already deployed in that folder on a different version is a `409`; use `upgrade` (next row). Read `DeploymentName` off the response: it is slugged and suffixed, never the title or the project name. See [cli-reference § Deployments](references/cli-reference.md#deployments). | -| "Move a deployment to another version" / "upgrade the deployed model" / "that folder is serving an old version" | `uip ixp deployments upgrade <project-name> <deployment-name> --version <N> --folder-key <guid> --output json` — `<deployment-name>` is the `DeploymentName` from `deployments list`, **not** the title (a title there is a `404`). Changes which version **every runtime caller of that folder and name** gets, so confirm intent on a shared folder. **Not a rollback path** — the target version must still appear in `projects list-models`. See [cli-reference § create vs upgrade](references/cli-reference.md#create-vs-upgrade). | -| "Where is this model deployed?" / "list deployments" / "which folder or version is live at runtime" | `uip ixp deployments list <project-name> --output json` — array of `DeploymentName`, `DeploymentTitle`, `ModelVersion`, `FolderKey`, `DeployedAt`; `[]` for a never-deployed project. **The only reliable source of `DeploymentName`** — run it before any `upgrade`. | -| "Show metrics" / "What are the scores?" | `uip ixp projects get-metrics <project-name> --model-version <N> --output json` — **always name the version.** Without `--model-version` the CLI returns the **latest trained** version, which is not necessarily the published/live one; get the live version from `list-models` first (see "How is this project performing?"). | -| "List projects" | `uip ixp projects list --output json` | -| "Configure the model" | `uip ixp projects configure-model <project-name> [options] --output json` | -| "What model / pre-processing does this project use?" / "Query the model settings" | `uip ixp projects get-taxonomy <project-name> --output json` — the configured extraction model and pre-processing are under `Data.dataset._model_config`: `model_version` is the `--model` value (e.g. `gemini_2_5_flash`), and `input_config` must be inverted to the `none`/`table_mini`/`table` token (`null` = not configured, so report the project default — **not** `none`). There is **no `get-model-config`**, and `configure-model` is a read-modify-write: never call it to find out the current settings, it rewrites them. Do NOT answer from `list-models`' `ModelName` — that's the labeller family (`gemini_ixp`), not a `--model` value, and it says nothing about pre-processing. Inversion table: [CLI Reference § Reading the current model and pre-processing](references/cli-reference.md#reading-the-current-model-and-pre-processing). | -| "Delete a project" / "Remove this project" | `uip ixp projects delete <project-name> -y --output json` — **permanent and irreversible**; removes the project's documents, taxonomy, and trained models. Requires `-y/--yes` (the CLI never prompts). | -| "Upload a document" / "Add documents to an existing project" | `uip ixp documents upload <project-name> <file> --output json` — see [CLI Reference § Uploading documents](references/cli-reference.md#uploading-documents-to-an-existing-project). One file per call; loop for multiple. For brand-new projects use `projects create` instead. | -| "Delete a document" / "Remove a document" | `uip ixp documents delete <project-name> <document-id> -y --output json` — irreversible, triggers retrain. `-y/--yes` is required (the CLI never prompts). To delete by filename, look up the `DocumentId` via `documents list` (the `Filename` field shows the original upload name). | -| "Add / delete / rename a field group" | `uip ixp groups {add,delete,rename} <project-name> --name <name> ... --output json` — see [CLI Reference § Groups](references/cli-reference.md#groups). `groups add` requires `--instructions` and `--fields '<json>'` — pass **all** of the new group's fields in that one `--fields` array (batch); do NOT create the group then add fields one at a time (use `fields add` only for an already-existing group). `delete` requires `-y/--yes` (the CLI never prompts). | -| "Add / edit / rename / delete a data type" | `uip ixp data-types {add,update-instructions,rename,delete} <project-name> --name <name> ... --output json` — see [CLI Reference § Data Types](references/cli-reference.md#data-types). `add` requires `--kind` (text/date/money/number/boolean/choice) and `--instructions`. `--input-value` (exact-match/inferred) is required only for `--kind text` and `--kind choice`; the other kinds don't have this property and the CLI rejects the flag for them. `delete` requires `-y/--yes` (the CLI never prompts); deleting a data type breaks any field referencing it. **Reuse a default data type before adding a custom one (Critical Rule 17)** — most needs map to a built-in (e.g. currency → `Monetary Quantity`). | -| "Add / delete / rename / retype a field" | `uip ixp fields {add,delete,rename,change-type} <project-name> --group <name> --field <name> ... --output json` — see [CLI Reference § Fields](references/cli-reference.md#fields). `delete` requires `-y/--yes`; `change-type` deletes annotations and also requires `-y/--yes`. | -| "Move a field to another group" / "this field belongs under X" | There is **no move command** — `--group` addresses a field, it never reparents one. Read the field's `type` and `instructions` from `projects get-taxonomy`, then `fields add` into the target group, then `fields delete <source> -y`. **Add before deleting**, so a failed add leaves the field where it was. Both groups must already exist; a move never creates one. **IRREVERSIBLE** — the recreated field gets a new `field_id`, so its confirmed labels do NOT follow it; say so before starting. **Never move a field via `get-taxonomy` → edit → `import-taxonomy`**: the import *merges*, so the field ends up in **both** groups. Full recipe: [CLI Reference § Moving a field](references/cli-reference.md#moving-a-field-to-a-different-field-group). | -| "Fix an OCR-garbled value" / "Confirm with a correction" | `uip ixp labellings confirm <project-name> <document-id> --fields <ids> --corrections '[{"field_id":"<id>","value":"<fixed>"}]' --output json` — confirms the listed fields and records the corrected value for the garbled one (include the corrected field's id in `--fields`). **`--corrections` is only for OCR garble** — the prediction is already the right answer in the right location, merely misread (e.g. `MSIÓÓÓ601020/` → `MSI0601020`); never use it to flip a wrong prediction, that field stays unannotated (Critical Rule 8). For one occurrence of a repeatable group add `--group <name> --occurrence <N>`; in the batched `--group <name> --updates '[…]'` form each entry takes `"corrections": {"<field_id>": "<value>"}` (an object keyed by field id, not the flat array). See [CLI Reference](references/cli-reference.md#labellings). | -| "Mark a field as missing for a document" | `uip ixp labellings mark-missing <project-name> <document-id> --fields <ids> --output json` — marks the listed fields missing; use when a field is genuinely absent and IXP predicted no value for it. Listing the field in `labellings confirm --fields` records the same missing marker when the field appears in predictions with an empty value. **Only mark a field missing if IXP also predicted nothing for it** — never to override a wrong prediction. See Critical Rule 12. | -| "Undo / unconfirm a wrong confirmation" | `uip ixp labellings unconfirm <project-name> <document-id> --fields <ids> --output json` — rolls back an earlier `confirm` or `mark-missing` for the listed fields (`confirm` can't un-confirm — Critical Rule 14). Every other annotation on the document is carried forward. With `--fields` alone, a field id shared across occurrences of a repeatable group is removed from **all** of them; to roll back specific occurrences, add `--group` with `--occurrence <N>` or `--updates '[…]'` (mirrors `confirm` — see the row below and Critical Rule 14). | -| "Confirm one line item / extraction" / "Confirm only this occurrence" | `uip ixp labellings confirm <project-name> <document-id> --group <name> --occurrence <N> [--fields <ids>] --output json` — targets one specific extraction of a repeatable field group (0-based index **from the latest `get-predictions`**). Without `--fields`, confirms every predicted field in that occurrence; with `--fields`, only those. Other occurrences untouched. Confirming renumbers the group on the next read (the confirmed row moves to `Occurrence` 0) — so batch multiple occurrences into one `--updates '[…]'` call rather than chaining `--occurrence` calls off a single read. See Critical Rules 13 and 18. | -| "Unconfirm one line item / extraction" / "Roll back only this occurrence" | `uip ixp labellings unconfirm <project-name> <document-id> --group <name> --occurrence <N> [--fields <ids>] --output json` — rolls back one specific extraction of a repeatable field group (0-based index, same as `get-predictions`/`confirm`). Without `--fields`, unconfirms every annotated field in that occurrence; with `--fields`, only those. Other occurrences untouched. **Re-read `get-predictions` first** — on a partly-confirmed group the confirmed rows sort to the front, so the index that confirmed a row is usually not the index that rolls it back (Critical Rule 18). For several occurrences in one call, use `--updates '[…]'` instead. See Critical Rule 14. | -| "Set overall extraction instructions" / "Update project prompt" | `uip ixp projects update-prompt <project-name> --prompt "<text>" --output json` — replaces the taxonomy-wide prompt (the "Overall extraction instructions" field in the IXP UI). Distinct from `fields update-prompts` (per-field) and `groups update-prompts` (per-field-group). | -| "How is this project performing?" / "What's the F1?" | Resolve the live version with `uip ixp projects list-models <project-name> --output json`, then `uip ixp projects get-metrics <project-name> --model-version <live-version> --output json` (Critical Rule 21). If Data is `{ Metrics: null }` the model isn't validated yet — report that and stop. If the call instead returns `Result: Failure` with `ErrorCode: not_found` (a project with no trained model yet, e.g. no confirmed labellings), treat it the same way — report "no metrics yet" and stop. Otherwise Data is flat; report in order: (1) which version the scores belong to + `TrainedTime`; (2) overall `ProjectScore`/`ProjectScoreQuality`; (3) per-group scores from `FieldGroups[]` (F1/Precision/Recall); (4) per-field scores from `Fields[]`, sorted lowest-F1 first — F1 **with** its `Precision`/`Recall` (a low F1 means the opposite fix depending on which side is short), plus `Annotations` (the sample size behind the F1) and `ErrorRate` (= errors/`Annotations` — it counts misses, so it is **not** `1 - Precision`). Ignore the `Quality` labels (derived, inconsistent scales — [Improve Prompts Guide § What get-metrics returns](references/improve-prompts-guide.md#what-get-metrics-returns-and-which-values-decide)). State numbers plainly; no "good enough" judgement unless asked; route low scores to [Improve Prompts Guide](references/improve-prompts-guide.md). Answer from these calls only — no ad-hoc discovery (Critical Rule #1). | -| "Describe this project" / "What's in it?" | Three calls, reported in order: (1) identity — `Title`/`Name` from `uip ixp projects get <project-name> --output json`; (2) current model **version** — live/published + `TrainedTime` from `list-models` (the trained version, not the configured extraction model — for that see the row above); (3) taxonomy — label-group/field counts from `uip ixp projects get-taxonomy <project-name> --output json` (raw artifact: counts live under `Data.dataset.label_groups` and `Data.dataset.entity_defs`, snake_case). Fold in performance (above) only if asked. Do NOT page `documents list` (its `Data` is a paged `{ Documents, Total, Offset, Limit }` envelope — use `Total` for a count) or read deployment bindings. Answer from these calls only (Critical Rule #1). | +|---|---| +| Create a project / upload to a new project | Follow [Project Setup Guide](references/project-setup-guide.md); new-project creation combines uploads and taxonomy. | +| Import a taxonomy | Use Option B of [Project Setup Guide](references/project-setup-guide.md): `--skip-taxonomy` + `import-taxonomy`. | +| Label or review documents | Follow [Label Documents Guide](references/label-documents-guide.md). | +| Improve scores, prompts, or F1 | Follow [Improve Prompts Guide](references/improve-prompts-guide.md). | +| Publish/tag a model | `uip ixp projects publish <project-name> --output json`; add `--tag <live\|staging>` as requested. Publishes latest unless `--model-version` is specified and does not deploy. Do not chain deployment unless requested. | +| Roll back/restore version N | `uip ixp projects publish <project-name> --model-version <N> --output json`; obtain versions with `projects list-models`. | +| Unpublish a model | `uip ixp projects unpublish <project-name> --model-version <N> --output json`; find published versions with `list-models`. To change live, publish another version. | +| Remove a live/staging tag | `uip ixp projects untag <project-name> --tag <live\|staging> --output json`; only `untag` removes a tag. | +| Deploy version N to a folder | Resolve a named folder with `uip or folders list --output json`; ask only if none was identified. Run `uip ixp deployments create <project-name> --version <N> --folder-key <guid> [--title <title>] --output json`. Version and folder key are required; `--title` defaults to the project name without `-ixp`. On conflict, list and upgrade. Read/report `DeploymentName`. | +| Move an existing deployment to version N | First run `uip ixp deployments list <project-name> --output json`; then `uip ixp deployments upgrade <project-name> <deployment-name> --version <N> --folder-key <guid> --output json`, using `DeploymentName`, not title. Target must remain in `list-models`; confirm impact on shared folders. | +| List deployments / find runtime folder or version | `uip ixp deployments list <project-name> --output json`; report `DeploymentName`, `DeploymentTitle`, `ModelVersion`, `FolderKey`, and `DeployedAt`. `[]` means never deployed. | +| Show metrics/scores | Resolve the requested version, then `uip ixp projects get-metrics <project-name> --model-version <N> --output json`; never report an unnamed/default version. | +| List projects | `uip ixp projects list --output json`. | +| Configure the model | `uip ixp projects configure-model <project-name> [options] --output json`. | +| Show model/pre-processing settings | Use `uip ixp projects get-taxonomy <project-name> --output json`. Read `Data.dataset._model_config`: `model_version` is the `--model` value; invert `input_config` to `none`, `table_mini`, or `table`; `null` means project default, not `none`. There is no `get-model-config`; `configure-model` is read-modify-write and must not read. Do not use `list-models.ModelName`, which is the labeller family. See [CLI Reference § Reading the current model and pre-processing](references/cli-reference.md#reading-the-current-model-and-pre-processing). | +| Delete a project | `uip ixp projects delete <project-name> -y --output json`; permanent and irreversible. | +| Upload to an existing project | `uip ixp documents upload <project-name> <file> --output json`; one file per call; loop for multiple. | +| Delete a document | Resolve the filename through `documents list`, then run `uip ixp documents delete <project-name> <document-id> -y --output json`; irreversible and retrains. | +| Add/delete/rename a group | `uip ixp groups {add,delete,rename} <project-name> --name <name> ... --output json`. `groups add` requires `--instructions` and one complete `--fields '<json>'` array; use `fields add` only for an existing group. Delete requires `-y`. | +| Add/update/rename/delete a data type | `uip ixp data-types {add,update-instructions,rename,delete} <project-name> ... --output json`. `add` requires `--kind` and `--instructions`; `--input-value` is allowed/required only for text and choice kinds. Delete requires `-y` and can break referencing fields. Reuse defaults per Rule 17. | +| Add/delete/rename/retype a field | `uip ixp fields {add,delete,rename,change-type} <project-name> --group <name> --field <name> ... --output json`. Delete and `change-type` require `-y`; changing type deletes annotations. | +| Move a field to another group | No move command. Read `type` and `instructions` from `projects get-taxonomy`; add to target, then delete source, adding first. Both groups must exist. This is irreversible: the new field has a new ID and labels do not follow. Never edit/export/import taxonomy; import merges and duplicates fields. See [CLI Reference § Moving a field](references/cli-reference.md#moving-a-field-to-a-different-field-group). | +| Correct an OCR-garbled value | Use `uip ixp labellings confirm <project-name> <document-id> --fields <ids> --corrections '[{"field_id":"<id>","value":"<fixed>"}]' --output json`; include the corrected field in `--fields`. For repeatable groups use `--group`/`--occurrence`, or batched `--updates` with corrections keyed by field ID. See [CLI Reference](references/cli-reference.md#labellings). | +| Mark a field missing | `uip ixp labellings mark-missing <project-name> <document-id> --fields <ids> --output json`; only when predictions contain no value and the document lacks it. An empty prediction may instead be included in `confirm --fields`. | +| Undo/unconfirm | `uip ixp labellings unconfirm <project-name> <document-id> --fields <ids> --output json`. Fields alone affect all occurrences of a shared repeatable field ID; use `--group` plus `--occurrence` or `--updates` for selected occurrences. Carry forward all other annotations. | +| Confirm one occurrence | `uip ixp labellings confirm <project-name> <document-id> --group <name> --occurrence <N> [--fields <ids>] --output json`; indices come from the latest read. Without fields, confirm all predicted fields in that occurrence. Batch several with `--updates`; re-read after sequential writes. | +| Unconfirm one occurrence | `uip ixp labellings unconfirm <project-name> <document-id> --group <name> --occurrence <N> [--fields <ids>] --output json`; re-read first because matched rows renumber; use `--updates` for several. | +| Set overall extraction instructions | `uip ixp projects update-prompt <project-name> --prompt "<text>" --output json`; replaces taxonomy-wide instructions and differs from field/group updates. | +| Ask how the project is performing | Resolve live version with `projects list-models`, then call `get-metrics --model-version <live-version>`. If `Data` is `{ Metrics: null }`, or the result is `Result: Failure` with `ErrorCode: not_found`, report no metrics yet and stop. Otherwise report, in order: (1) version and `TrainedTime`; (2) overall `ProjectScore`/`ProjectScoreQuality`; (3) group scores from `FieldGroups[]` (F1, Precision, Recall); (4) fields from `Fields[]`, lowest F1 first, including F1, Precision, Recall, `Annotations`, and `ErrorRate` (`errors/Annotations`, not `1 - Precision`). Ignore `Quality` labels. State numbers plainly; do not judge unless asked; route low scores to [Improve Prompts Guide](references/improve-prompts-guide.md). Use only documented calls. | +| Describe the project | Make and report three calls in order: (1) identity from `projects get` (`Title`/`Name`); (2) current live/published trained version and `TrainedTime` from `list-models`; (3) taxonomy counts from `projects get-taxonomy` (`Data.dataset.label_groups` and `Data.dataset.entity_defs`). Include performance only if asked. Do not page `documents list` or inspect deployments. | ## Common Pitfalls -| Symptom | Cause | Fix | -|---------|-------|-----| -| Reported score doesn't match what the IXP UI's build page shows | `get-metrics` was called without `--model-version`, so it returned the **latest trained** version while the UI (or your own sentence) named the **live** one | Resolve the version from `list-models` and re-run `get-metrics --model-version <N>`. A project whose live version was pinned a while ago can have many newer trained versions, and the latest may score lower. See Critical Rule 21. | -| Metrics don't change after a prompt update | Re-evaluation hasn't completed | Wait out the retrain — [Improve Prompts Guide § Waiting for retrain](references/improve-prompts-guide.md#waiting-for-retrain). | -| ModelVersion doesn't advance | Retrain still in progress | Any change to model inputs (labellings OR instructions) triggers a full retrain. Re-read metrics under the **bounded** wait in [Improve Prompts Guide § Waiting for retrain](references/improve-prompts-guide.md#waiting-for-retrain) — fixed interval, capped number of checks, then stop. Never poll indefinitely. | -| Field instructions conflict with label_def instructions | `fields update-prompts` only edits per-field instructions, NOT the parent label_def instructions | Before iterating, read the label_def `instructions` and update them with `groups update-prompts` if they contradict the per-field prompts. | -| A confirmed line item now reads back as the first row, or the other rows' `Occurrence` numbers shifted | Expected: the read returns annotation↔prediction matched pairs first, so confirmed rows sort ahead of unconfirmed ones | Nothing to fix — values and page locations are unchanged. Re-run `get-predictions` before the next per-occurrence call and target the row by its values (Critical Rule 18). | -| A second `--occurrence` call landed on the wrong row, or `unconfirm --occurrence N` no-ops | Indices came from a read taken *before* an earlier confirm renumbered the group | Re-read `get-predictions` between per-occurrence writes, or issue them as one `--updates` call. | -| `deployments create` returns `409` | That title is already deployed in that folder on a different version — `create` only ever ADDS | Use `deployments upgrade <project-name> <deployment-name>` instead, taking `<deployment-name>` from `deployments list` (Critical Rule #20). | -| `deployments upgrade` returns `404` | A `DeploymentTitle` was passed where `DeploymentName` is expected — the name is slugged and suffixed (`invoices` → `invoices-08963f00-ixp`), so it cannot be derived from the title | Run `deployments list <project-name> --output json` and pass its `DeploymentName` verbatim. | +| Symptom | Cause and fix | +|---|---| +| Metrics differ from the UI | Latest was used instead of live. Resolve with `list-models`, then pass `--model-version`. | +| Metrics do not change after prompt updates | Retrain/re-evaluation is incomplete; follow [Improve Prompts Guide § Waiting for retrain](references/improve-prompts-guide.md#waiting-for-retrain). | +| `ModelVersion` does not advance | Retraining is still running. Recheck under the guide’s bounded interval and capped checks; never poll indefinitely. Labeling and instruction changes retrain. | +| Field and group instructions conflict | `fields update-prompts` changes only field instructions; read and, if needed, update parent `label_def` instructions with `groups update-prompts`. | +| Confirmed row moves or indices shift | Expected matched-pair ordering. Re-read and relocate by values; values and page locations are unchanged. | +| Sequential occurrence action hits the wrong row or no-ops | The index was stale. Re-read between writes or batch targets in one `--updates` call. | +| `deployments create` returns `409` | The title already exists in that folder on another version. List deployments, then upgrade with `DeploymentName`. | +| `deployments upgrade` returns `404` | A title was supplied instead of backend-generated `DeploymentName`. List deployments and copy the name verbatim. | ## Unsupported Capabilities -These requests fall outside the skill. Recognise the request, reply with the standard response, route the user. Do NOT enter discovery (`uip --help`, grep, source reading) — see Critical Rule #1. - -| User request | Standard response | -|--------------|-------------------| -| "Create a model" / "create a project" | **Documents or a taxonomy supplied →** use the [Project Setup Guide](references/project-setup-guide.md) (this skill creates the project from them). **Otherwise →** "I work on existing IXP projects rather than creating them from scratch. Create one in-product: https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/managing-projects — then I can label, review, and improve it." | -| "Upload these files" / "add documents" | **Project named / already in context →** supported; upload it (see the "Upload a document" row in Task Navigation). **Otherwise →** "Name an existing project and I'll upload it — or upload in-product (e.g. for a new project): https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/building-and-deploying-models." | -| "Push to an environment / another tenant" / "deploy to staging or production" | **Names an Orchestrator folder (a folder literally called `Production`) →** supported; use the "Deploy this model to a folder" row in [Task Navigation](#task-navigation). **Otherwise →** "IXP has no environment or cross-tenant deploy target — a deployment is a (folder, version) pair inside one tenant." Note `projects publish --tag staging\|live` moves the tag the DU framework (and DU activities calling through it) resolve — for those consumers that IS the staging/live switch; it creates no folder deployment. | -| "Give X access" / "share this project" / "change roles or permissions" | "Access, roles, and permissions are managed in-product, not through this skill: https://docs.uipath.com/ixp/automation-cloud/latest/overview/managing-access." | -| "Use this model in my automation / workflow / agent" / "call the extractor from a process" | "Consuming a published model inside an automation is an authoring task outside this skill. See https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/building-and-consuming-a-workflow." | -| "Mine these emails / communications" / "set up Communications Mining" | "Communications Mining is a separate IXP capability this skill doesn't cover (this skill is document extraction). See https://docs.uipath.com/ixp/automation-cloud/latest/cm-user-guide/introduction-to-uipath-communication-mining." | -| "Monitor the deployed model" / "how many docs did it process?" / "runtime throughput or incidents" | "Runtime/operational monitoring of a deployed model lives in Orchestrator, not this skill: https://docs.uipath.com/orchestrator/automation-cloud/latest/user-guide/about-monitoring. For design-time scores use `get-metrics` (see 'Show metrics')." | -| "Edit a data type's values" / "add or remove a Choice option" | "The CLI can rename a data type, change its instructions (`data-types update-instructions`), or delete it — but it can't add or remove the **values** of an existing Choice data type. Do that by hand in-product on the Manage Taxonomy page: https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/managing-projects — then continue here." | +Requests outside this skill include creating a project/model with no documents, cross-tenant/environment deployment, access/roles/permissions, consuming a model inside an automation, Communications Mining, runtime/operational monitoring, and editing `Choice` values. Recognize the request, give the standard response, and route the user; never discover commands with `uip --help`, grep, or source reading. Use [Unsupported Capabilities](references/unsupported-capabilities.md) for exact responses and links. ## Reference Navigation -- [CLI Commands Reference](references/cli-reference.md) — all `uip ixp` commands with options and output formats -- [Project Setup Guide](references/project-setup-guide.md) — create a new project, review and label documents -- [Improve Prompts Guide](references/improve-prompts-guide.md) — iterative optimization loop with regression detection -- [Label Documents Guide](references/label-documents-guide.md) — reusable workflow for reviewing and confirming predictions +- [CLI Commands Reference](references/cli-reference.md) — documented `uip ixp` commands, options, and output formats +- [Project Setup Guide](references/project-setup-guide.md) — new projects, uploads, taxonomy, and initial labeling +- [Improve Prompts Guide](references/improve-prompts-guide.md) — iterative optimization and regression detection +- [Label Documents Guide](references/label-documents-guide.md) — prediction review and confirmation workflow +- [Deployment Guide](references/deployment-guide.md) — deploy a trained version to an Orchestrator folder +- [Unsupported Capabilities](references/unsupported-capabilities.md) — out-of-scope requests and their standard responses diff --git a/skills/uipath-ixp/references/cli-reference.md b/skills/uipath-ixp/references/cli-reference.md index 038a30911f..dc2e973696 100644 --- a/skills/uipath-ixp/references/cli-reference.md +++ b/skills/uipath-ixp/references/cli-reference.md @@ -1,258 +1,223 @@ # CLI Commands Reference -All commands use `uip ixp` prefix. Always append `--output json` when parsing output programmatically. +All commands use `uip ixp`. Always append `--output json` when parsing output programmatically. > **Destructive commands require `-y, --yes`.** Every irreversible `uip ixp` command (all `delete`s and `fields change-type`) gates on `-y/--yes`; the CLI never prompts. Always pass `-y/--yes`. ## Projects | Command | Description | -|---------|-------------| -| `uip ixp projects list [-l <limit>] [--offset <n>] --output json` | List IXP projects — returns a paged envelope `Data: { Projects: [{ Id, Name, Title, CreatedAt }], Total, Offset, Limit }` (rows under `Projects`, **not** a bare array). `-l, --limit` defaults 50 (range 1-10000); `--offset` defaults 0 to page. | -| `uip ixp projects get <project-name> --output json` | Get a project | -| `uip ixp projects create "<name>" <folder-path> [-d "<description>"] [--skip-taxonomy] --output json` | Create project and upload supported docs in `<folder-path>` (top-level only — sub-folders are not scanned; see [Supported document files](#supported-document-files)). By default suggests+imports taxonomy. `-d` provides context for better taxonomy suggestion. Use `--skip-taxonomy` to create a blank project (import taxonomy separately). Use `ProjectName` from output. | -| `uip ixp projects import-taxonomy <project-name> <file> --output json` | Import taxonomy from a local JSON file. Accepts `{ field_types, label_group }` or `{ entity_defs, label_groups }` format. **Merges — it never replaces**: entries you omit are kept and a posted `field_id` is ignored, so it cannot remove, move, or replace anything (a re-imported edit returns `{"status":"ok"}` and silently leaves duplicates). Use it to seed a project that has no taxonomy; change an existing one with the targeted `groups`/`fields`/`data-types` commands. | -| `uip ixp projects update-title <project-name> "<new-title>" --output json` | Update the display title of a project | -| `uip ixp projects update-prompt <project-name> --prompt "<text>" --output json` | Update the project's **Overall extraction instructions** — the taxonomy-wide prompt the model sees on every extraction (the field at the top of the IXP UI's Manage Taxonomy page). Distinct from per-field-group prompts (`groups update-prompts`) and per-field prompts (`fields update-prompts`). Replaces the existing value. | -| `uip ixp projects get-taxonomy <project-name> --output json` | Export the raw IXP taxonomy artifact. Data is `{ status, dataset: { entity_defs, label_groups } }` — read `entity_defs` and `label_groups` under `dataset`. Intended for re-import (see `import-taxonomy`), not a human-readable view. `dataset` also carries `_model_config`, the only read path for the configured extraction model and pre-processing — see [Reading the current model and pre-processing](#reading-the-current-model-and-pre-processing). | -| `uip ixp projects get-metrics <project-name> [--model-version <N>] --output json` | Get validation metrics. **Validated model →** flat Data: `ProjectScore`, `ProjectScoreQuality`, `ValidatedDocuments`, `ModelVersion`, plus per-group `FieldGroups[]` (`FieldGroup`, `F1`, `Precision`, `Recall`, `ErrorRate`, `Documents`) and per-field `Fields[]` (`FieldGroup`, `FieldId`, `Name`, `F1`, `Precision`, `Recall`, `ErrorRate`, `Documents`, `Annotations`, `Quality`). `Name` is the field's display name resolved from the taxonomy — report on it, but compare on `FieldId`, which is the stable key; it is `null` when the service could not resolve it (e.g. the field was deleted after that version was scored). Display names are unique only within a group, so qualify as `<FieldGroup> / <Name>` when two fields share one. Scores are surfaced at the backend's own precision — long tails like `0.824999988079071` are its float32 arithmetic widened to double, not extra accuracy; round when you display them, and compare the raw values. **Trained but not yet validated →** Data is `{ Metrics: null }` (not an error). **No trained model yet (e.g. a project with no confirmed labellings) →** the call returns a failure envelope `Result: Failure` with `ErrorCode: not_found` (no `Data`), NOT `{ Metrics: null }` — treat it as "no metrics yet". **Defaults to the LATEST TRAINED version, which is NOT necessarily the published/live one** — resolve the version from `list-models` and pass it as `--model-version <N>` whenever you report a score, so the numbers and the version identity match (SKILL.md Critical Rule 21). **Any version the backend ever scored is readable**, including older ones `list-models` no longer lists — that is what makes a version-to-version comparison possible; `not_found` on a version means the backend never scored it, not that it aged out. Field semantics — which values decide and which are derived — are in [Improve Prompts Guide § What get-metrics returns](improve-prompts-guide.md#what-get-metrics-returns-and-which-values-decide). `ErrorRate` is `errors / Annotations` (it counts misses — not `1 - Precision`); the `Quality`/`ProjectScoreQuality` labels use inconsistent scales — never gate on them. | -| `uip ixp projects configure-model <project-name> [options] --output json` | Configure extraction model. Options: `--model` (gemini_2_5_flash/gemini_2_5_pro/gpt_4o_2024_05_13) and `--preprocessing` (none/table_mini/table). To read the current settings, see [Reading the current model and pre-processing](#reading-the-current-model-and-pre-processing). | -| `uip ixp projects list-models <project-name> --output json` | List all model versions and tags. Returns `Models[]` (`Version`, `ModelName`, `Pinned`, `TrainedTime`, `Description`), `Tags[]` (`Name`, `Version`, `UpdatedAt`), and `MaxPublished`. **The only read path for the project's live version** — `Tags[]` entry Name=`live`, else the highest `Models[]` with `Pinned: true`; which version a **folder** serves at runtime is a different question — [Deployments](#deployments). `ModelName` is the trained labeller's **family** (e.g. `gemini_ixp`, `gemini_pro_ixp`) — it is never a `--model` value like `gemini_2_5_flash`, so it does not answer "which extraction model is configured" (see [Reading the current model and pre-processing](#reading-the-current-model-and-pre-processing)). | -| `uip ixp projects publish <project-name> [--model-version <N>] [--tag <live\|staging>] --output json` | Publish a model version — defaults to the latest; pass `-m, --model-version <N>` to pick a specific one. `-d, --description "<text>"` sets a description; `--tag <live\|staging>` tags the published version. | -| `uip ixp projects unpublish <project-name> --model-version <N> --output json` | Unpublish a model version — it stays trained and listable; only its published status is removed. `--model-version` is **required**. Errors if the version isn't found or isn't currently published. To change which version is live, `publish` a different one instead. | -| `uip ixp projects untag <project-name> --tag <live\|staging> --output json` | Remove a tag by **name** (`--tag` is **required**; tag names are unique within a project, so this is unambiguous even when one version holds several tags). The version the tag pointed at stays published; only that tag is cleared. Errors if no version carries the tag. Only `untag` removes a tag — `publish` without `--tag` leaves the existing tag untouched. To switch `live`→`staging`, `publish --tag staging` instead. | -| `uip ixp projects delete <project-name> -y --output json` | **Permanently** delete a project — its documents, taxonomy, and trained models. **Irreversible.** `-y, --yes` is **required**; the command refuses to run without it (the CLI never prompts). | +|---|---| +| `uip ixp projects list [-l <limit>] [--offset <n>] --output json` | List projects. Returns paged `Data: { Projects: [{ Id, Name, Title, CreatedAt }], Total, Offset, Limit }`; rows are under `Projects`, not a bare array. `-l, --limit`: default 50, range 1-10000. `--offset`: default 0. | +| `uip ixp projects get <project-name> --output json` | Get a project. | +| `uip ixp projects create "<name>" <folder-path> [-d "<description>"] [--skip-taxonomy] --output json` | Create a project and upload supported documents from the top level of `<folder-path>`; do not scan sub-folders. By default suggest+import taxonomy. `-d` supplies taxonomy context; `--skip-taxonomy` creates a blank project. Use `ProjectName` from output. See [Supported document files](#supported-document-files). | +| `uip ixp projects import-taxonomy <project-name> <file> --output json` | Import local JSON in `{ field_types, label_group }` or `{ entity_defs, label_groups }` format. **Merges; never replaces**: omitted entries remain and posted `field_id` is ignored, so this cannot remove, move, or replace definitions. Re-imported edits can return `{"status":"ok"}` while leaving duplicates. Run it only to seed a project without taxonomy; modify existing taxonomy with targeted `groups`/`fields`/`data-types` commands. | +| `uip ixp projects update-title <project-name> "<new-title>" --output json` | Update display title. | +| `uip ixp projects update-prompt <project-name> --prompt "<text>" --output json` | Replace **Overall extraction instructions**, distinct from `groups update-prompts` and `fields update-prompts`. | +| `uip ixp projects get-taxonomy <project-name> --output json` | Export raw taxonomy. Read `Data.dataset.entity_defs` and `Data.dataset.label_groups`; use it for re-import, not human-readable viewing. `dataset` also contains `_model_config`, the only read path for configured extraction model and pre-processing; see [Reading the current model and pre-processing](#reading-the-current-model-and-pre-processing). | +| `uip ixp projects get-metrics <project-name> [--model-version <N>] --output json` | Get validation metrics. Validated: flat `Data` with `ProjectScore`, `ProjectScoreQuality`, `ValidatedDocuments`, `ModelVersion`, `FieldGroups[]` (`FieldGroup`, `F1`, `Precision`, `Recall`, `ErrorRate`, `Documents`) and `Fields[]` (`FieldGroup`, `FieldId`, `Name`, `F1`, `Precision`, `Recall`, `ErrorRate`, `Documents`, `Annotations`, `Quality`). **Compare on `FieldId`** (stable key); **report on `Name`** (current display name), which is `null` when unresolved (e.g. the field was deleted after that version was scored)—fall back to `FieldId`. Names are unique only within a group, so qualify as `<FieldGroup> / <Name>` when two share one. Scores carry the backend's float32 precision (e.g. `0.824999988079071`)—round on display, compare raw. Trained but unvalidated: `Data: { Metrics: null }`. No trained model: failure envelope `Result: Failure`, `ErrorCode: not_found`, no `Data`; treat as no metrics. Default is latest trained, not necessarily published/live. Resolve the version with `list-models` and pass `--model-version <N>` whenever reporting a score so score and version match (SKILL.md Critical Rule 21). **Any version the backend ever scored is readable**, including older ones `list-models` no longer lists—`not_found` on a version means it was never scored, not that it aged out. `ErrorRate` is `errors / Annotations`, not `1 - Precision`; `Quality` and `ProjectScoreQuality` use inconsistent scales—never gate on them. See [Improve Prompts Guide § What get-metrics returns](improve-prompts-guide.md#what-get-metrics-returns-and-which-values-decide). | +| `uip ixp projects configure-model <project-name> [options] --output json` | Configure model: `--model` (`gemini_2_5_flash/gemini_2_5_pro/gpt_4o_2024_05_13`) and `--preprocessing` (`none/table_mini/table`). Read settings below. | +| `uip ixp projects list-models <project-name> --output json` | List `Models[]` (`Version`, `ModelName`, `Pinned`, `TrainedTime`, `Description`), `Tags[]` (`Name`, `Version`, `UpdatedAt`), and `MaxPublished`. Read the live version only from `Tags[]` Name=`live`; otherwise use the highest `Models[]` with `Pinned: true`. Folder runtime version is separate; see [Deployments](#deployments). `ModelName` is the trained labeller family (for example, `gemini_ixp`, `gemini_pro_ixp`), never a `--model` value and not the configured extraction model. | +| `uip ixp projects publish <project-name> [--model-version <N>] [--tag <live\|staging>] --output json` | Publish a trained version; defaults latest. `-m, --model-version <N>` selects one. `-d, --description "<text>"` sets description; `--tag <live\|staging>` tags it. | +| `uip ixp projects unpublish <project-name> --model-version <N> --output json` | Remove published status while retaining the trained version. `--model-version` is required; errors if missing or not published. Publish another version to change live. | +| `uip ixp projects untag <project-name> --tag <live\|staging> --output json` | Remove a required tag. Tags are unique within a project; errors if no version has it. Only `untag` removes a tag; publish without `--tag` leaves tags unchanged. Publish with `--tag staging` to switch `live` to `staging`. | +| `uip ixp projects delete <project-name> -y --output json` | **Permanently and irreversibly** delete project documents, taxonomy, and trained models. `-y, --yes` is required. | ### Reading the current model and pre-processing -There is **no `get-model-config` command** — `configure-model` only writes. The configured extraction model and pre-processing are in the taxonomy artifact under `Data.dataset._model_config`: +There is **no `get-model-config` command**. Run: ```bash uip ixp projects get-taxonomy <project-name> --output json ``` -**Model** — `_model_config.model_version` holds the `--model` value verbatim (e.g. `gemini_2_5_flash`). Report that one. Do **not** report `list-models`' `ModelName`: that is the trained labeller's family (`gemini_ixp`) and carries no pre-processing information at all. - -**Pre-processing** — `_model_config.input_config` stores the underlying mode, not the `none|table_mini|table` token, so invert it: +Read `Data.dataset._model_config.model_version` and report it verbatim as the configured `--model`; never report `list-models` `ModelName`. Invert `Data.dataset._model_config.input_config`: | `input_config` | `--preprocessing` | -|----------------|-------------------| -| `null` | never configured — report it as *not configured* (the project uses the IXP default), **not** as `none` | +|---|---| +| `null` | never configured; report *not configured*, not `none` | | `{"mode": "image_only"}` | `none` | | `{"mode": "text_plus_image", "text_config": {"kind": "uipath_cv_table_only"}}` | `table_mini` | | `{"mode": "text_plus_image", "text_config": {"kind": "gemini_table_only"}}` | `table` | -The remaining `_model_config` keys (`kind`, `flags`, `attribution_method`, `temperature`, `top_p`, `seed`, `system_prompt_override`, `iterative_config`) have no `uip ixp` flag — mention them only if the user asks. - -`_model_config` reflects the project's **current** setting, not the setting a given trained version was built with — so report it as the project's configuration, not as a property of the published version. +`kind`, `flags`, `attribution_method`, `temperature`, `top_p`, `seed`, `system_prompt_override`, and `iterative_config` have no `uip ixp` flag; mention them only if asked. `_model_config` is the project's current setting, not the setting used by a trained version; report it as project configuration. ## Documents | Command | Description | -|---------|-------------| -| `uip ixp documents list <project-name> [-l <limit>] [--offset <n>] --output json` | List documents — returns a paged envelope `Data: { Documents: [{ DocumentId, AttachmentRef, Filename }], Total, Offset, Limit }` (rows are under `Documents`, **not** a bare array). `AttachmentRef`/`Filename` may be `null`; `Filename` is the original upload filename. `-l, --limit` defaults 50 (range 1-10000); `--offset` defaults 0 (range 0-1000000). | -| `uip ixp documents download <project-name> <document-id> -o <path> --output json` | Download the original document file (PDF/PNG/JPG/etc.) to exactly the `-o` path you give. **The CLI does not append or correct a file extension** — the file is written verbatim to `-o`, and `Data.ContentType` is commonly the generic `application/octet-stream` rather than the true MIME type — so include the correct extension in `-o` yourself (e.g. `-o invoice.pdf`). `Data.Path` echoes the path you provided. | -| `uip ixp documents upload <project-name> <file> --output json` | Upload a single document file to an existing project. See [Uploading documents](#uploading-documents-to-an-existing-project) below for validation, output shape, and the multi-file loop pattern. | -| `uip ixp documents delete <project-name> <document-id> -y --output json` | Delete a document (and its labellings) from a project. Irreversible — triggers a retrain. `-y, --yes` is **required** (the CLI never prompts; without it the command refuses and exits 1). | +|---|---| +| `uip ixp documents list <project-name> [-l <limit>] [--offset <n>] --output json` | List documents. Returns `Data: { Documents: [{ DocumentId, AttachmentRef, Filename }], Total, Offset, Limit }`; rows are under `Documents`, not a bare array. `AttachmentRef`/`Filename` may be `null`; `Filename` is the original upload filename. `-l, --limit`: default 50, range 1-10000. `--offset`: default 0, range 0-1000000. | +| `uip ixp documents download <project-name> <document-id> -o <path> --output json` | Download the original file exactly to `-o`; the CLI does not append or correct extensions. Include the extension. `Data.ContentType` is commonly `application/octet-stream`; `Data.Path` echoes the supplied path. | +| `uip ixp documents upload <project-name> <file> --output json` | Upload one document; see [Uploading documents to an existing project](#uploading-documents-to-an-existing-project). | +| `uip ixp documents delete <project-name> <document-id> -y --output json` | Irreversibly delete a document and labellings; triggers retrain. `-y, --yes` is required. | ### Supported document files -Both `projects create` (bulk folder upload) and `documents upload` (single file) validate against the same extension whitelist, case-insensitive: - -`.pdf`, `.png`, `.jpg`, `.jpeg`, `.gif`, `.tif`, `.tiff`, `.bmp` - -Validation differs by command: +`projects create` and `documents upload` accept, case-insensitively: `.pdf`, `.png`, `.jpg`, `.jpeg`, `.gif`, `.tif`, `.tiff`, `.bmp`. -- `documents upload` rejects an unsupported file with `Unsupported file type "<ext>"` before any network call. -- `projects create` scans only the top level of `<folder-path>` (sub-folders are ignored), silently skips unsupported files, and fails only when **no** supported files exist (`No supported documents found in <folder>`). +- `documents upload` rejects unsupported extensions before network access with `Unsupported file type "<ext>"`. +- `projects create` scans only the folder's top level, silently skips unsupported files, and fails only when none remain: `No supported documents found in <folder>`. -Each upload triggers a retrain — wait it out before reading metrics or predictions for new docs, under the bounded wait in [Improve Prompts Guide § Waiting for retrain](improve-prompts-guide.md#waiting-for-retrain). +Each upload triggers retrain. Wait under the bounded wait in [Improve Prompts Guide § Waiting for retrain](improve-prompts-guide.md#waiting-for-retrain) before reading metrics or predictions for new documents. ### Uploading documents to an existing project -`uip ixp documents upload <project-name> <file> --output json` pushes one document to an existing project. +Run `uip ixp documents upload <project-name> <file> --output json` once per file. Supported files return `{ ProjectName, Filename, AttachmentRef, DocumentId }` (`Code: IxpDocumentsUpload`); capture `DocumentId` for download or `labellings confirm`. -For supported extensions, validation error strings, and retrain timing, see [Supported document files](#supported-document-files) above. - -Returns `{ ProjectName, Filename, AttachmentRef, DocumentId }` (Code: `IxpDocumentsUpload`). Capture `DocumentId` for later `documents download` or `labellings confirm` calls. - -**Multiple files** — one file per call; loop the command: +For multiple files, run one call per file: ```bash cd "<folder-with-docs>" for f in *.pdf *.png *.jpg *.jpeg *.gif *.tif *.tiff *.bmp; do - [ -e "$f" ] || continue # skip unmatched glob patterns + [ -e "$f" ] || continue uip ixp documents upload <project-name> "$f" --output json done ``` -**When NOT to use this:** for filling a brand-new project, prefer `projects create <name> <folder-path>` — uploads the whole folder and suggests a taxonomy in one call. +For a new project, prefer `projects create <name> <folder-path>` to upload the folder and suggest taxonomy together. ## Data Types -Manage the reusable type definitions (entity_defs) that fields reference via `field_type_id`. In the IXP UI, these are the project's "Data Types". +Data types are reusable `entity_defs` referenced by fields through `field_type_id`; the UI calls them Data Types. | Command | Description | -|---------|-------------| -| `uip ixp data-types add <project-name> --name <name> --kind <text\|date\|money\|number\|boolean\|choice> --instructions <text> [--input-value <exact-match\|inferred>] [--choices <json>] --output json` | Create a new data type. `--kind` selects the underlying data shape; `text` is the default Text type. `--input-value` is **required for `--kind text` and `--kind choice`, forbidden for `date`, `money`, `number`, and `boolean`** — those kinds don't expose the "Exact match" / "Inferred" radio in the IXP UI, so the CLI rejects the flag when the kind doesn't support it. `exact-match` marks the value as appearing verbatim in the document; `inferred` is for computed/derived values that don't have a visible location. `--choices` is **required when `--kind choice`** and forbidden otherwise. JSON array of `{"value":"<canonical>","alternates":["<alt1>",...]}`; `value` is the canonical display name (model output); `alternates` is optional (defaults to `[]`) and lists alternate spellings the model maps to `value`. | -| `uip ixp data-types update-instructions <project-name> --name <name> --instructions <text> --output json` | Replace the instructions on an existing data type. Name, kind, and input-value stay the same. | -| `uip ixp data-types rename <project-name> --name <name> --new-name <name> --output json` | Rename a data type. Existing field references (via `field_type_id`) stay intact. | -| `uip ixp data-types delete <project-name> --name <name> -y --output json` | Delete a data type. **IRREVERSIBLE** — any field referencing it via `field_type_id` will break. `-y, --yes` is **required** (the CLI never prompts). | +|---|---| +| `uip ixp data-types add <project-name> --name <name> --kind <text\|date\|money\|number\|boolean\|choice> --instructions <text> [--input-value <exact-match\|inferred>] [--choices <json>] --output json` | Add a type. `--input-value` is required for `text` and `choice`, forbidden for `date`, `money`, `number`, `boolean`; `exact-match` means verbatim and `inferred` means computed/not visibly located. `--choices` is required for `choice`, forbidden otherwise; JSON is `[ {"value":"<canonical>","alternates":["<alt1>",...]} ]`, with optional `alternates` defaulting to `[]`. | +| `uip ixp data-types update-instructions <project-name> --name <name> --instructions <text> --output json` | Replace instructions; name, kind, and input-value remain. | +| `uip ixp data-types rename <project-name> --name <name> --new-name <name> --output json` | Rename while preserving `field_type_id` references. | +| `uip ixp data-types delete <project-name> --name <name> -y --output json` | **Irreversibly** delete a type; referencing fields break. `-y, --yes` is required. | ### Default data types -Every IXP project ships with the built-in data types below (the project's `entity_defs` from `projects get-taxonomy` are the authoritative list). **Before `data-types add`, or before choosing a field's `--type` (in `fields add` / `groups add`), check the existing `entity_defs` and reuse a matching default.** A redundant custom type (e.g. a `Currency Amount` when `Monetary Quantity` already exists) splits annotations across two types and forfeits the default's pre-trained model. Add a new data type only when it carries something no default does — a `Choice`, or a reusable concept that needs its own tailored extraction instructions — not as a clone of a default. +Before `data-types add` or selecting `--type` in `fields add`/`groups add`, run `projects get-taxonomy`, inspect authoritative `entity_defs`, and reuse a matching default. Do not create redundant types: they split annotations and lose pretrained behavior. | Default type | `--kind` | `--input-value` | Reuse for | -|--------------|----------|-----------------|-----------| -| `Exact Text` | `text` | `exact-match` | Text copied verbatim from the document — names, IDs, addresses, codes | -| `Inferred Text` | `text` | `inferred` | Text derived/computed, not appearing verbatim in the document | +|---|---|---|---| +| `Exact Text` | `text` | `exact-match` | Verbatim names, IDs, addresses, codes | +| `Inferred Text` | `text` | `inferred` | Derived/computed text | | `Number` | `number` | — | Counts, quantities, plain numbers | | `Date` | `date` | — | Dates | -| `Monetary Quantity` | `money` | — | Any currency / monetary amount — total, subtotal, tax, unit price, freight | -| `Boolean` | `boolean` | — | True / false values | +| `Monetary Quantity` | `money` | — | Currency amounts: total, subtotal, tax, unit price, freight | +| `Boolean` | `boolean` | — | True/false | -`Date`, `Number`, `Monetary Quantity`, and `Boolean` carry pre-trained models with a fixed output format (below) — instructions cannot change their formatting, so a hand-rolled equivalent is strictly worse. `Choice` is the only `--kind` with no default: choice types are always project-specific (`data-types add --kind choice --choices …`). +`Date`, `Number`, `Monetary Quantity`, and `Boolean` have pretrained fixed output formats; hand-rolled equivalents are worse. `Choice` has no default and is always project-specific. ### Normalized output formats -`get-predictions` reports these types in the type's normalized form, never the page's literal text. A plain `confirm` stores that same normalized string as the label. +`get-predictions` and plain `confirm` use normalized values, not literal page text: -| Type | `FormattedValue` | Page → prediction | -|------|------------------|-------------------| -| `Date` | `YYYY-MM-DDTHH:MM:SSZ` — a date-only page value comes back at `T00:00:00Z` | `21-JUN-22` → `2022-06-21T00:00:00Z` | -| `Monetary Quantity` | `<amount> <ISO-4217 code>` — no thousands separator, decimals as written on the page (not fixed to 2), currency appended even when the page shows none | `114.91` → `114.91 AUD`; `8.0700` → `8.0700 USD` | -| `Number` | bare numeric string, no unit or separator | `29311577` → `29311577` | +| Type | `FormattedValue` | Example | +|---|---|---| +| `Date` | `YYYY-MM-DDTHH:MM:SSZ`; date-only values use `T00:00:00Z` | `21-JUN-22` → `2022-06-21T00:00:00Z` | +| `Monetary Quantity` | `<amount> <ISO-4217 code>`; no thousands separator, decimals as written, currency appended if absent | `114.91` → `114.91 AUD`; `8.0700` → `8.0700 USD` | +| `Number` | Bare numeric string, without unit or separator | `29311577` → `29311577` | | `Boolean` | `True` / `False` | — | -`--corrections` neither normalizes nor validates — the string you send is stored verbatim (`21-JUN-22`, even `not-a-date`, all return Success). Sending the page's format replaces a correct label with one the model will never predict and drops the field's F1. Reformatting is never a reason to use `--corrections` (Critical Rule 8). +`--corrections` neither normalizes nor validates; it stores the supplied string verbatim, including `21-JUN-22` or `not-a-date`, and returns Success. Use it only for OCR-mangled values; never reformat page values with it (Critical Rule 8). ## Groups -Manage field groups (label_defs) — the document type containers for fields. To edit fields **inside** an existing group, use the `fields` subject below. +Field groups (`label_defs`) contain fields. | Command | Description | -|---------|-------------| -| `uip ixp groups add <project-name> --name <group-name> --instructions <text> --fields <json> --output json` | Create a new field group with its fields. `--instructions` describes what document/section the group covers (the model sees it during extraction). `--fields` is a JSON array `[{"name":"...","type":"<type-name>","instructions":"..."}]` — **put ALL of the new group's fields in this one array (batch); do NOT create the group then add its fields one at a time.** Every entry must include `name`, `type`, and a non-empty `instructions`. `type` resolves against the project's `entity_defs` — reuse a [default data type](#default-data-types) before inventing a new one. To add a field to an **already-existing** group, use `fields add` instead. | -| `uip ixp groups delete <project-name> --name <group-name> -y --output json` | Delete a field group. **IRREVERSIBLE** — deletes all annotations on all fields in the group. `-y, --yes` is **required** (the CLI never prompts). | -| `uip ixp groups rename <project-name> --name <group-name> --new-name <name> --output json` | Rename a field group. Preserves all fields and annotations. | -| `uip ixp groups update-prompts <project-name> --updates <json> --output json` | Bulk-update field group (label_def) instructions. `--updates` is a JSON array `[{"name":"<group>","instructions":"..."}]` matched by group name. Existing fields are preserved. Unmatched names are reported in the response without failing the command. | +|---|---| +| `uip ixp groups add <project-name> --name <group-name> --instructions <text> --fields <json> --output json` | Add a group and **all** its fields in one batch. `--instructions` describes the document/section. `--fields` is `[ {"name":"...","type":"<type-name>","instructions":"..."} ]`; every entry requires non-empty `name`, `type`, and `instructions`. Resolve `type` in `entity_defs` and reuse [default data types](#default-data-types). Use `fields add` for an existing group. | +| `uip ixp groups delete <project-name> --name <group-name> -y --output json` | Irreversibly delete the group and all field annotations. `-y, --yes` is required. | +| `uip ixp groups rename <project-name> --name <group-name> --new-name <name> --output json` | Rename while preserving fields and annotations. | +| `uip ixp groups update-prompts <project-name> --updates <json> --output json` | Bulk replace group instructions with `[ {"name":"<group>","instructions":"..."} ]`. Fields remain; unmatched names are reported without failing. | ## Fields -Structural edits to a field within an existing field group. For instruction-only edits use `fields update-prompts` (see below). To create the group itself, use `groups add` above. - | Command | Description | -|---------|-------------| -| `uip ixp fields add <project-name> --group <field-group-name> --field <name> --type <type-name> --instructions <text> --output json` | Add a new field to an **existing** field group. `--type` is the name of an entity_def in the project's taxonomy (see `projects get-taxonomy`) — reuse a [default data type](#default-data-types) (e.g. `Monetary Quantity` for a currency amount) before adding a custom one. `--instructions` is required — describe what to extract and where it appears. | -| `uip ixp fields delete <project-name> --group <field-group-name> --field <name> -y --output json` | Remove a field from a field group. `-y, --yes` is **required** (the CLI never prompts). | -| `uip ixp fields rename <project-name> --group <field-group-name> --field <name> --new-name <name> --output json` | Rename a field. Preserves `field_id` and existing annotations. | -| `uip ixp fields change-type <project-name> --group <field-group-name> --field <name> --type <type-name> -y --output json` | Change a field's type. **IRREVERSIBLE** — the server creates a new field under the hood, so all existing annotations for that field are deleted. `-y, --yes` is **required** (the CLI never prompts). | -| `uip ixp fields update-prompts <project-name> --updates <json> --output json` | Bulk-update per-field extraction instructions. `--updates` is a JSON array `[{"name":"<field>","instructions":"..."}]` matched by `moon_form` field name (across all field groups). Existing field definitions are preserved. Unmatched names are reported in the response without failing the command. | +|---|---| +| `uip ixp fields add <project-name> --group <field-group-name> --field <name> --type <type-name> --instructions <text> --output json` | Add a field to an existing group. `--type` names an `entity_defs` entry; reuse [default data types](#default-data-types). `--instructions` is required and must say what to extract and where. | +| `uip ixp fields delete <project-name> --group <field-group-name> --field <name> -y --output json` | Remove a field; `-y, --yes` is required. | +| `uip ixp fields rename <project-name> --group <field-group-name> --field <name> --new-name <name> --output json` | Rename while preserving `field_id` and annotations. | +| `uip ixp fields change-type <project-name> --group <field-group-name> --field <name> --type <type-name> -y --output json` | **Irreversibly** change type; the server creates a new field and deletes annotations. `-y, --yes` is required. | +| `uip ixp fields update-prompts <project-name> --updates <json> --output json` | Bulk replace per-field instructions with `[ {"name":"<field>","instructions":"..."} ]`, matched by `moon_form` field name across groups. Definitions remain; unmatched names are reported without failing. | ### Moving a field to a different field group -There is **no move/reparent command**. Every field command takes its group as `--group`, which only addresses the field — it cannot change which group owns it. A move is two `fields` calls against the existing groups, in this order: +There is no move/reparent command. Tell the user before starting, then: -1. `uip ixp projects get-taxonomy <project-name> --output json` — read the field's current `type` and `instructions` so they can be carried over. In `Data.dataset`, the field is a `moon_form` entry under its group's `label_def`; its type is the `entity_defs[]` entry whose `id` matches the entry's **`field_type_id`** (NOT its `field_id`, which is the field's own identity and matches no `entity_def`). -2. `uip ixp fields add <project-name> --group <target-group> --field <name> --type <type-name> --instructions <text> --output json` — recreate it in the target group. -3. `uip ixp fields delete <project-name> --group <source-group> --field <name> -y --output json` — remove it from the source group. +1. Run `uip ixp projects get-taxonomy <project-name> --output json`; in `Data.dataset`, read the field's `moon_form` entry under its group's `label_def`. Resolve its type through the `entity_defs[]` entry whose `id` matches `field_type_id`, **not** `field_id`. +2. Run `uip ixp fields add <project-name> --group <target-group> --field <name> --type <type-name> --instructions <text> --output json`. +3. After the add succeeds, run `uip ixp fields delete <project-name> --group <source-group> --field <name> -y --output json`. -Add before deleting: if the add fails, the field is still in its original group. Both groups must already exist — a move never creates one. Creating the target group first, if the user asked for a group that isn't there yet, is a separate `groups add` step you should confirm with them. +Both groups must already exist; creating a target group is a separate `groups add` step requiring confirmation. The add mints a new `field_id`, so labels do not follow and documents require re-review. -**IRREVERSIBLE** — `fields add` mints a new `field_id`, so the field's confirmed labels do not follow it into the new group. Tell the user before starting; documents must be re-reviewed for that field. - -**Do NOT move a field by editing the taxonomy and re-importing it.** `projects import-taxonomy` **merges** — it does not replace. Fields you omit from a posted group are kept, and a posted `field_id` is ignored (the backend mints a new one), so the import returns `{"status":"ok"}` while leaving the field in **both** groups as two separate fields. Do not use `groups delete` + `groups add` either: that destroys every other field in the group along with its annotations. +Do **not** edit and re-import taxonomy to move a field: `import-taxonomy` merges, retains omitted fields, ignores posted `field_id`, and can leave duplicate fields in both groups while returning `{"status":"ok"}`. Do not use `groups delete` + `groups add`; it destroys every other field and its annotations. ## Labellings | Command | Description | -|---------|-------------| -| `uip ixp labellings get-predictions <project-name> <document-id> --output json` | Get IXP model predictions for one document. Returns `Data: { ProjectName, TotalDocuments, DocumentsWithPredictions, Predictions[] }`. Each `Predictions[]` entry is one document `{ DocumentId, Labels[] }`; each label is `{ Name, Occurrence, Fields[] }`; each field is `{ FieldId, FieldName, FormattedValue }`. This is the model's **prediction** layer, not the confirmed/annotation layer. Each label carries an explicit `Occurrence` (the value for `--occurrence`/`--updates`); it is 0-based and usually runs 0..N-1 in document order, but do NOT assume it is contiguous or starts at 0 — a single-occurrence group can come back as `Occurrence` 1 with no 0. Always target the actual `Occurrence` value reported here, never a positional guess. **The order is not stable across writes**: the server lists annotation↔prediction matched pairs first, so confirmed rows of a repeatable group sort to the front and the rest renumber — see [Occurrence numbering and read order](#occurrence-numbering-and-read-order). Each document also carries `ModelVersion` (the model version that produced its predictions) — capture it and pass it to `confirm -m/--model-version` to guard against a mid-review retrain. | -| `uip ixp labellings confirm <project-name> <document-id> [--fields <ids>] [--corrections <json>] [--model-version <version>] --output json` | Confirm predictions for a document. (`--fields` has short alias `-f`; `--corrections` has short alias `-c`.) Without `--fields`, confirms every predicted field that has content. `--fields "a7c3e9105f2b4d86,b2f8a01c7d3e6940"` confirms only those fields, and applies a **single uniform rule**: listed fields with content get confirmed; listed fields whose IXP prediction is empty get a missing marker (the explicit listing IS the confirmation that the empty state is intentional — see Critical Rule 12). `--corrections '[{"field_id":"...","value":"..."}]'` is **only for OCR-mangled values** — same field, same location, garbled bytes. Do NOT use `--corrections` to flip wrong booleans, fix wrong inferred values, or override any non-OCR mistake; those fields must be left unannotated. See Critical Rule 8. Existing missing markers and other annotations carry forward across calls. `-m, --model-version <N>` pins the model version you reviewed (the `ModelVersion` from `get-predictions`); if a retrain produced a newer version since, the confirm is rejected (`PredictionVersionChangedError`) instead of stamping drifted values — re-read predictions and review again. | -| `uip ixp labellings confirm <project-name> <document-id> --group <name> --occurrence <N> [--fields <ids>] [--corrections <json>] [--model-version <version>] --output json` | **Single-occurrence form** — confirm ONE occurrence. The ergonomic choice for a single line. `--occurrence` is the 0-based index of the target extraction within `--group`, as reported by the **latest** `get-predictions`. Without `--fields`, confirms every predicted field in that one occurrence; with `--fields`, confirms only those fields there. Other occurrences are untouched. Requires `--group` (Critical Rule 13). Mutually exclusive with `--updates`. The call renumbers the group for subsequent reads ([Occurrence numbering and read order](#occurrence-numbering-and-read-order)), so use `--updates` for more than one row instead of chaining these off one read. | -| `uip ixp labellings confirm <project-name> <document-id> --group <name> --updates <json> [--model-version <version>] --output json` | **Batched form** — confirm SEVERAL occurrences in ONE atomic call (one request; avoids N round-trips, e.g. a 10-line invoice). `--updates` is a JSON array `[{"occurrence":<0-based-index>,"fields"?:["<field_id>",…],"corrections"?:{"<field_id>":"<value>"}}]`. Per entry: **omit `"fields"`** to confirm every predicted field in that occurrence (same default as `--occurrence` without `--fields`), or list specific IDs; un-selected fields in a selected occurrence carry forward any existing annotation. **`--updates` is the superset** — `--occurrence <N>` ≡ `--updates` with one entry; both share the same per-occurrence logic. Use `--occurrence` for a single line, `--updates` for several together. Mutually exclusive with `--fields`/`--corrections`/`--occurrence`. | -| `uip ixp labellings unconfirm <project-name> <document-id> --fields <ids> --output json` | Roll back confirmations on a document (`--fields` has short alias `-f`) — the listed fields go back to un-annotated state. Use when an earlier `confirm` was a mistake (confirm can't un-confirm — Critical Rule 14). Every other annotation on the document is carried forward. **With `--fields` alone, a field id shared across occurrences of a repeatable group is removed from all of them**; to scope the roll-back to specific occurrences, add `--group` (see the two rows below). Returns `Unmatched` for IDs that weren't annotated to begin with. | -| `uip ixp labellings unconfirm <project-name> <document-id> --group <name> [--occurrence <N>] [--fields <ids>] --output json` | **Per-occurrence form** — roll back specific occurrences of a repeatable group instead of every occurrence a field id appears in. `--group` alone unconfirms every occurrence of the group; add `--occurrence <N>` (0-based, same index as `get-predictions`/`confirm`, taken from a **fresh** read — on a partly-confirmed group the index that confirmed a row is usually not the index that rolls it back) to roll back ONE occurrence. Without `--fields`, unconfirms every annotated field in the targeted occurrence(s); with `--fields`, only those there. Other occurrences are untouched. Mutually exclusive with `--updates`. Mirrors `confirm`'s `--group`/`--occurrence` flags. | -| `uip ixp labellings unconfirm <project-name> <document-id> --group <name> --updates <json> --output json` | **Batched form** — roll back SEVERAL occurrences in ONE atomic call. `--updates` is a JSON array `[{"occurrence":<0-based-index>,"fields"?:["<field_id>",…]}]`. Per entry: omit `"fields"` to unconfirm every annotated field in that occurrence, or list specific IDs. Occurrences not listed are left as-is. Mutually exclusive with `--fields`/`--occurrence`. | -| `uip ixp labellings mark-missing <project-name> <document-id> --fields <ids> --output json` | Mark the listed fields as missing (`--fields` has short alias `-f`; annotated with no value and no location) — use when a field is genuinely absent from the document and IXP predicted no value for it. Unlike `confirm --fields`, it also marks a field that's gone from the current predictions entirely (e.g. a stale prior annotation after a model/taxonomy change), which `confirm` can't reach. **Only for fields where IXP predicted no value** — if IXP predicted a *wrong* value, leave the field unannotated instead. Returns `Unmatched` for any IDs not found in the document's annotation OR prediction. | +|---|---| +| `uip ixp labellings get-predictions <project-name> <document-id> --output json` | Return `Data: { ProjectName, TotalDocuments, DocumentsWithPredictions, Predictions[] }`. Each prediction document is `{ DocumentId, Labels[] }`; each label `{ Name, Occurrence, Fields[] }`; each field `{ FieldId, FieldName, FormattedValue }`. `Occurrence` is explicit, 0-based, may be non-contiguous or start at 1; use it, never a positional guess. Read order changes after writes; see [Occurrence numbering and read order](#occurrence-numbering-and-read-order). Capture each document's `ModelVersion` and pass it to `confirm -m/--model-version`. | +| `uip ixp labellings confirm <project-name> <document-id> [--fields <ids>] [--corrections <json>] [--model-version <version>] --output json` | Without `--fields`, confirm every predicted field with content. `--fields`/`-f` confirms listed fields with content and marks listed empty predictions missing; listing is intentional confirmation of empty state (Critical Rule 12). `--corrections`/`-c` is only for OCR-mangled values at the same location; do not use it for wrong booleans, inferred values, or other non-OCR errors—leave those unannotated (Critical Rule 8). Existing annotations carry forward. Pass reviewed `ModelVersion` with `-m, --model-version`; a newer retrain causes `PredictionVersionChangedError`; reread and review. | +| `uip ixp labellings confirm <project-name> <document-id> --group <name> --occurrence <N> [--fields <ids>] [--corrections <json>] [--model-version <version>] --output json` | Confirm one occurrence. Use the 0-based value from the latest `get-predictions`; without `--fields`, confirm all predicted fields in that occurrence, otherwise only listed fields. Other occurrences remain untouched. Requires `--group` (Critical Rule 13). Mutually exclusive with `--updates`. A write renumbers subsequent reads; use `--updates` for several rows. | +| `uip ixp labellings confirm <project-name> <document-id> --group <name> --updates <json> [--model-version <version>] --output json` | Atomically confirm several occurrences. JSON: `[ {"occurrence":<0-based-index>,"fields"?:["<field_id>",…],"corrections"?:{"<field_id>":"<value>"}} ]`. Omit `fields` to confirm all predicted fields; selected occurrences' unselected fields retain annotations. `--updates` is equivalent to one-entry `--occurrence`; mutually exclusive with `--fields`/`--corrections`/`--occurrence`. | +| `uip ixp labellings unconfirm <project-name> <document-id> --fields <ids> --output json` | Roll back listed fields to unannotated while carrying other annotations. Without `--group`, a shared field ID is removed from all repeatable occurrences. Returns `Unmatched` for IDs not initially annotated. Use when confirm was mistaken; confirm cannot un-confirm (Critical Rule 14). | +| `uip ixp labellings unconfirm <project-name> <document-id> --group <name> [--occurrence <N>] [--fields <ids>] --output json` | Scope rollback to a group or one occurrence. `--group` alone targets every occurrence; add `--occurrence <N>` from a fresh prediction read to target one. Without `--fields`, target all annotated fields; with it, only those IDs. Other occurrences remain. On partly confirmed groups, reread because indices shift. Mutually exclusive with `--updates`. | +| `uip ixp labellings unconfirm <project-name> <document-id> --group <name> --updates <json> --output json` | Atomically roll back several occurrences. JSON: `[ {"occurrence":<0-based-index>,"fields"?:["<field_id>",…]} ]`; omit `fields` for all annotated fields. Unlisted occurrences remain. Mutually exclusive with `--fields`/`--occurrence`. | +| `uip ixp labellings mark-missing <project-name> <document-id> --fields <ids> --output json` | Mark listed fields missing with no value/location; use only when IXP predicted no value. Unlike `confirm --fields`, reaches fields absent from current predictions, including stale annotations after model/taxonomy changes. Do not use for wrong predictions; leave them unannotated. Returns `Unmatched` for IDs absent from both annotations and predictions. | ### Occurrence numbering and read order -`Occurrence` is the row's position in the read that reported it, not a stable row id — a repeatable group's rows have no per-row identifier in the contract (`field_group.id` is the taxonomy group id and is identical for every row). - -The server pairs annotations with predictions and returns the **matched pairs first**, then the unmatched predictions. Confirming one row therefore moves it to `Occurrence` 0 on the next read and shifts the others down (the IXP UI shows it first too). The row's values and page location are unchanged — only its position in the read moves. Document order holds only for a group with no annotations, or one where every row is annotated. +`Occurrence` is a read position, not a stable row ID; repeatable groups have no per-row identifier (`field_group.id` is identical for every row). The server returns annotation↔prediction matched pairs first, then unmatched predictions. Confirming a row moves it to occurrence 0 and shifts others; values and page locations do not change. Document order is reliable only when no rows, or all rows, are annotated. -So an `Occurrence` value is invalidated by any write to its group: +Any write to a group invalidates occurrence values: -- confirm/unconfirm every target row in ONE `--updates` call — all indices in a call resolve against the same read; -- between sequential per-occurrence calls, re-run `get-predictions` and re-locate each row by its field values; -- never carry an index across a write. +- Confirm/unconfirm all target rows in one `--updates` call; indices in that call use one read. +- Between sequential per-occurrence calls, run `get-predictions` and relocate each row by field values. +- Never carry an index across a write. ## Deployments -Publishing a version (`projects publish`) makes it usable **inside** the project. Deploying it to an Orchestrator folder is the separate step that makes it callable **at runtime** — activity packs and Maestro Flow address a model by the `{FolderKey, DeploymentName}` pair. +`projects publish` makes a version usable inside the project. Deploying it to an Orchestrator folder makes it callable at runtime; callers address `{FolderKey, DeploymentName}`. | Command | Description | -|---------|-------------| -| `uip ixp deployments create <project-name> --version <N> --folder-key <guid> [--title <title>] --output json` | Deploy a trained model version to an Orchestrator folder. **Only ever adds** — never repoints an existing deployment (see [create vs upgrade](#create-vs-upgrade)). `--version` and `--folder-key` are both **required**; `--title` defaults to the project name minus its `-ixp` suffix. Returns `ProjectName`, `ModelVersion`, `FolderKey`, `DeploymentTitle`, `DeploymentName` (Code: `IxpDeploymentsCreate`). | -| `uip ixp deployments upgrade <project-name> <deployment-name> --version <N> --folder-key <guid> --output json` | Move an existing deployment to another trained model version. `<deployment-name>` is a positional argument and takes the **`DeploymentName`** from `deployments list` — NOT the title (Code: `IxpDeploymentsUpgrade`). | -| `uip ixp deployments list <project-name> --output json` | List the project's deployments across every version and folder. **The only reliable source of `DeploymentName`.** `Data` is an array — `[]` for a never-deployed project, never a `{Message: ...}` object, so iterate unconditionally. Each entry carries `DeploymentName`, `DeploymentTitle`, `ModelVersion`, `FolderKey`, `DeployedAt` (Code: `IxpDeploymentsList`). | -| `uip ixp deployments get-taxonomy <project-name> --version <N> --output json` | Get the project taxonomy (data types + field groups) at a specific trained model version. `--version` is **required** (non-negative integer; 0 is valid; no short alias) — get the number from `projects list-models`. Like `projects get-taxonomy`, the body is the raw IXP dataset artifact in snake_case, under `Data.dataset` (`entity_defs[]` + `label_groups[]`), bound to the snapshot the version was trained on (Code: `IxpDeploymentsGetTaxonomy`). | +|---|---| +| `uip ixp deployments create <project-name> --version <N> --folder-key <guid> [--title <title>] --output json` | Add a deployment; never repoint an existing one. `--version` and `--folder-key` are required. `--title` defaults to project name minus `-ixp`. Returns `ProjectName`, `ModelVersion`, `FolderKey`, `DeploymentTitle`, `DeploymentName` (`Code: IxpDeploymentsCreate`). | +| `uip ixp deployments upgrade <project-name> <deployment-name> --version <N> --folder-key <guid> --output json` | Move an existing deployment to another trained version. Positional `<deployment-name>` must be `DeploymentName` from `deployments list`, not title (`Code: IxpDeploymentsUpgrade`). | +| `uip ixp deployments list <project-name> --output json` | List deployments across versions/folders. `Data` is an array, `[]` when never deployed, never `{Message: ...}`. Entries contain `DeploymentName`, `DeploymentTitle`, `ModelVersion`, `FolderKey`, `DeployedAt` (`Code: IxpDeploymentsList`). | +| `uip ixp deployments get-taxonomy <project-name> --version <N> --output json` | Get version-specific taxonomy. `--version` is required, a non-negative integer; 0 is valid; no short alias. Read version from `projects list-models`. Raw snake_case artifact is under `Data.dataset` with `entity_defs[]` and `label_groups[]` (`Code: IxpDeploymentsGetTaxonomy`). | ### create vs upgrade -Two commands, not one. `create` only adds; `upgrade` moves an existing deployment. - -| Existing deployment in the folder | `create` | `upgrade` | +| Existing deployment in folder | `create` | `upgrade` | |---|---|---| | none | deploys | `404 [DeploymentNotFoundError]` | -| same model version | no-op, exit `0` | no-op, exit `0` — `DeployedAt` does not move | +| same model version | no-op, exit `0` | no-op, exit `0`; `DeployedAt` does not move | | different model version | `409 [DeploymentAlreadyExistsError]` | repoints | -Both verbs are no-ops at the same version, so both are safe to re-run from CI. `create` has **no `--force`** — use `upgrade` to repoint. - -`upgrade` changes which model version **every runtime caller of that folder and name** gets. Confirm the intent before running it against a shared folder. - -`upgrade`'s response echoes the *requested* version without re-reading. Call `list` to prove the move landed. +Both are safe to rerun at the same version. `create` has no `--force`; use `upgrade` to repoint. Confirm before upgrading a shared folder. `upgrade` echoes the requested version without rereading; run `list` to prove the move landed. ### DeploymentName vs DeploymentTitle -Distinct fields. Confusing them is the failure mode this command split exists to prevent. - -- `--title` sets `DeploymentTitle` — free-form, returned verbatim. -- `DeploymentName` is the name the **runtime** resolves: the backend slugs the title and appends a per-deployment suffix (`invoices` → `invoices-08963f00-ixp`). -- The suffix is generated per deployment and **cannot be predicted from the request** — two deployments of the same project in the same folder get different suffixes, and it matches neither the project name's suffix nor the folder key. Read `DeploymentName` off the create response or from `list`; never construct it. -- A deploy with no `--title` still gets its own suffix. `DeploymentName` is never just the project name. -- When `create` returns `DeploymentName: null` (the backend had not yet listed the new deployment), get it from `list`. -- `upgrade` takes `DeploymentName`. Passing a title lands a `404 [DeploymentNotFoundError]`. +`--title` sets free-form `DeploymentTitle`. `DeploymentName` is runtime-resolved: the backend slugs the title and appends a per-deployment suffix. The suffix is generated per deployment and cannot be predicted; never construct it. A deployment without `--title` still has a suffix, and `DeploymentName` is never just the project name. If create returns `DeploymentName: null`, run `list`. `upgrade` requires `DeploymentName`; passing a title returns `404 [DeploymentNotFoundError]`. ### --folder-key -Required on both `create` and `upgrade`; passed in the body, never as a path. The same name can be deployed in several folders, so the folder is part of the deployment identity — there is no tenant-level or default-folder deploy. Get keys from `uip or folders list --output json`. There is no `--folder-path` form, and the key format is not validated client-side (the backend owns what a valid key is), so a malformed key fails server-side. +`--folder-key` is required on both `create` and `upgrade`, passed in the body, never the path. The same name may exist in several folders; there is no tenant-level/default-folder deployment and no `--folder-path`. The key is not client-validated; malformed keys fail server-side. Get keys by running `uip or folders list --output json`. -When filtering the folder list, pass an explicit `--limit` — `--output-filter` without one is rejected on current CLIs (older builds silently filter a single page): +When filtering folders, pass an explicit `--limit`; `--output-filter` without one is rejected on current CLIs (older builds silently filter one page): ```bash uip or folders list --limit 500 --output json --output-filter "[?Path=='Shared'].Key" ``` -Omitting either required option fails locally with exit `3` / `Result: ValidationError` before any auth or backend call. `--version 0` is valid — versions are 0-based. +Omitting either required option fails locally with exit `3` / `Result: ValidationError`, before auth or backend access. `--version 0` is valid; versions are 0-based. ### Deployment errors | Surfaced error | Meaning | Fix | |---|---|---| -| `409 [DeploymentAlreadyExistsError]` on `create` | The title is already deployed in that folder on a **different** version | Run `upgrade` with the `DeploymentName` from `list` — NOT the title the backend's message quotes | -| `404 [DeploymentNotFoundError]` on `upgrade` | Name was never deployed, is deployed only in **another** folder, or a *title* was passed where `DeploymentName` belongs | Re-read `DeploymentName` from `list`; verify `--folder-key` | -| `404 [ModelVersionNotFoundError]` on `upgrade` | `--version` is not deployable (the version is checked before the deployment is looked up) | Pick a version from `projects list-models <project-name> --output json` | -| `408 Timed out waiting for new model version` on `upgrade` | Upstream IXP timeout. The CLI surfaces it without retrying, and the outcome is **unknown** — the write may or may not have landed | Run `list` and read the version actually being served before retrying. Do not assume either outcome | -| `409 [AmbiguousDeploymentError]` | More than one deployment matches in the folder | Disambiguate from `list`; carries the same `create` hint as the conflict above | - -**Rejected writes are no-ops** — every `409`/`404` above leaves the deployment on its original version with `DeployedAt` untouched. - -**Do not branch on `ErrorCode` for these.** A `409` surfaces as `ErrorCode: invalid_argument`, because `400`/`409`/`422` map alike. Branch on `Context.HttpStatus` or the bracketed backend error name. +| `409 [DeploymentAlreadyExistsError]` on `create` | Title already deployed in folder at a different version | Run `upgrade` with `DeploymentName` from `list`, not the quoted title | +| `404 [DeploymentNotFoundError]` on `upgrade` | Name absent, exists only in another folder, or title was supplied | Reread `DeploymentName`; verify `--folder-key` | +| `404 [ModelVersionNotFoundError]` on `upgrade` | Version is not deployable | Choose one from `projects list-models <project-name> --output json` | +| `408 Timed out waiting for new model version` on `upgrade` | Outcome unknown; write may have landed | Run `list`, inspect served version, then decide whether to retry | +| `409 [AmbiguousDeploymentError]` | Multiple deployments match in folder | Disambiguate from `list`; use the same `create` conflict guidance | -**`upgrade` is not a rollback path.** Versions leave the deployable list as a project retrains, so a deployment can be serving a version it can no longer be moved back to. Verify the target is in `projects list-models` first. +Rejected `409`/`404` writes leave the original version and `DeployedAt` unchanged. Do not branch on `ErrorCode`: these statuses map to `ErrorCode: invalid_argument`; branch on `Context.HttpStatus` or bracketed backend error name. -After a successful deploy, the folder-scoped runtime API takes roughly 15 seconds to resolve the new deployment. A runtime lookup immediately after `create` can miss it. +`upgrade` is not a rollback path: retraining can remove versions from the deployable list. Verify the target in `projects list-models` first. After successful deployment, folder-scoped runtime resolution takes roughly 15 seconds; an immediate lookup may miss it. diff --git a/skills/uipath-ixp/references/improve-prompts-guide.md b/skills/uipath-ixp/references/improve-prompts-guide.md index 2adeaf58b1..2529740b70 100644 --- a/skills/uipath-ixp/references/improve-prompts-guide.md +++ b/skills/uipath-ixp/references/improve-prompts-guide.md @@ -1,103 +1,82 @@ # Improve Prompts Guide -Iterative optimization loop for improving extraction quality on an existing IXP project. Runs multiple iterations automatically, rolling back if scores regress. +Iteratively improve extraction quality on an existing IXP project for up to the requested iterations (default: 3). Retrain after changes and roll back regressions. ## What Prompts CAN and CANNOT Fix -Before starting, understand the limits of prompt iteration: +Prompts can fix wrong values (precision), missed values (recall), and ambiguous candidate selection through clearer instructions, location hints, negative examples, and disambiguation. -**Prompts CAN fix:** +Prompts and reviewing cannot fix: -- Fields where the model extracts the wrong value (precision problems) — better instructions clarify what to extract -- Fields where the model misses the value entirely (recall problems) — location hints help the model find the field -- Ambiguous fields where the model picks the wrong candidate — negative examples and disambiguation rules help +- **OCR quality issues** — if OCR consistently garbles a field, correct OCR-mangled predictions during review with `labelling confirm --corrections` (keeps the reference, fixes the text). If many fields across documents are affected, report a data-quality issue instead of using iterations. +- **Missing fields** — instructions cannot create a value absent from documents. -**Neither prompts nor reviewing can fix:** +## How Prompt Updates Work -- **OCR quality issues** — if the OCR consistently garbles a field's text, no instruction will fix it. However, during the review step, OCR-mangled predictions can be corrected using `labelling confirm --corrections` (keeps the reference, fixes the text). If many fields are OCR-mangled across multiple documents, report this to the user as a data quality issue rather than burning prompt iterations. -- **Missing fields** — if a field simply doesn't exist in the documents, no instruction will conjure it. +Run these separate commands: -## How prompt updates work +- `uip ixp fields update-prompts <project> --updates <json>` — per-field instructions matched by field name. +- `uip ixp groups update-prompts <project> --updates <json>` — label_def/group instructions matched by label_def name. -Prompts live at two levels and are edited by two separate commands: - -- **`uip ixp fields update-prompts <project> --updates <json>`** — per-field instructions (e.g., "Invoice Number", "Invoice Date"). Match by field name. -- **`uip ixp groups update-prompts <project> --updates <json>`** — field group (label_def) instructions (e.g., "Invoice", "Line Items"). Match by label_def name. - -Each command sends one server-side call; the server matches by name and writes per affected label_def, preserving every definition you didn't change. To update both field and group instructions in the same iteration, run the two commands back-to-back. - -**Aligning group and field instructions.** Each label_def (e.g., "Invoice") has its OWN `instructions` field that the model sees alongside per-field instructions. If the group instruction says "Extract only fields visible on the first page" but a per-field instruction says "Found in the summary table on page 2", the model gets contradictory signals. When updating field instructions, also update the parent group instruction with `groups update-prompts` if it contradicts. +Each command makes one server-side call and preserves definitions omitted from the update. Run both back-to-back when changing fields and groups. Keep group and field instructions consistent: each label_def has its own `instructions` shown alongside field instructions. If they conflict, update the parent group with `groups update-prompts` too. ## Before Starting -The user may specify a max number of iterations (default: 3). Track: - -- **Baseline metrics** — the `get-metrics` payload before any changes, and its `ModelVersion` — a trained version's metrics can be re-read at any time with `--model-version <N>`, so keeping the version number is enough to recover anything. The values that drive the loop are mapped in [What `get-metrics` returns](#what-get-metrics-returns-and-which-values-decide); the rest is reported once or ignored. -- **Previous iteration metrics** — the same, for the last successful iteration's version -- **Previous instructions** — the per-field (field) instructions from the last successful iteration (for rollback) - -Do NOT re-read the taxonomy or sample documents between iterations — use what you already have. Only re-read metrics after each instruction update + retrain cycle. This assumes no one modifies the taxonomy or documents externally during the loop. If the user mentions changes were made in the web UI, re-fetch the taxonomy and document list before continuing. +Track baseline metrics and `ModelVersion` from `get-metrics`, previous-iteration metrics and `ModelVersion`, and previous instructions for rollback. -## What `get-metrics` returns, and which values decide +A trained version's metrics can be reread with `--model-version <N>`; retaining the version is sufficient. Do NOT reread taxonomy or sample documents between iterations. Reread only metrics after each update/retrain cycle. If the user reports web-UI changes, refetch taxonomy and document list before continuing. -The values `get-metrics` returns are neither independent nor interchangeable — using the wrong one silently changes the loop's behaviour. +## What `get-metrics` Returns and Which Values Decide -| Value | Level | Role in this loop | -|-------|-------|-------------------| -| `F1` | field, group | **Decision variable.** Targeting (2a), regression (2f), stopping (2f). | -| `Precision` | field, group | **Decision variable.** Splits a low `F1` into PRECISION vs RECALL (2a) — that split picks which rewrite to attempt. | -| `Recall` | field, group | **Decision variable.** Same split, plus the `< 0.5` labelling-gap probe (2a-check). | -| `Annotations` | field | **Decision variable.** Reviewed **extractions** for that field (not documents) — the sample `F1` is computed over, so it sets that field's regression threshold (2f). | -| `Documents` | field, group | **Decision variable.** How many documents this field (or group) was reviewed in — a per-field count, not a project total. `0` → SKIP (2a): no evidence to evaluate a rewrite against. Below `ValidatedDocuments` → some reviewed documents carry no label for this field (2f). | -| `ProjectScore` | project | **Report only** — the headline number, an average of the per-field `F1` values. Never gate on it (2f diffs the fields directly). | -| `ValidatedDocuments` | project | **Decision variable.** How many labelled documents the metrics are computed over — project-level only, and the ceiling for every per-field `Documents`. Below the project's total document count → unlabelled documents exist; label them before looping (1e). | -| `ModelVersion` | project | **Decision variable.** Retrain completion ([Waiting for retrain](#waiting-for-retrain)). | -| `ErrorRate` | field, group | **Report — independent of `Precision`.** Wrong extractions over `Annotations`. A wrong value counts **once** (not as a false positive plus a false miss), and a miss counts even though it cannot lower `Precision` — so `Precision` 1.00 can still carry `ErrorRate` 0.20. Report it as the manual-correction burden; diagnose direction from `Precision`/`Recall`. | -| `Quality` | field | **Ignore.** A coarse label derived from the numbers, on a scale inconsistent with `ProjectScoreQuality` (an `F1` of 1.00 still reads `good` while a `ProjectScore` of 0.91 reads `excellent`). Never gate on it and don't report it per field — if the user asks about the UI's label, explain the scales differ. | -| `ProjectScoreQuality` | project | **Report on the project line only** (the label the UI shows beside the score) — different scale from field `Quality` (above). | -| `FieldGroup`, `FieldId`, `Name` | field | Identity. Compare on `FieldId` (stable); report on `Name` (the current display name — `null` for a deleted field, fall back to `FieldId`, see 1a). | +| Value | Level | Role | +|---|---|---| +| `F1` | field, group | Decision variable for targeting (2a), regression, and stopping (2f). | +| `Precision` | field, group | Decision variable for precision/recall diagnosis (2a). | +| `Recall` | field, group | Decision variable for diagnosis and the `< 0.5` labelling-gap probe (2a-check). | +| `Annotations` | field | Reviewed **extractions**, not documents; sets that field's regression threshold (2f). | +| `Documents` | field, group | Reviewed documents for that field/group, not project total. `0` → SKIP (2a). Below `ValidatedDocuments` → some reviewed documents carry no label for the field (2f). | +| `ProjectScore` | project | Report only. Averages per-field `F1`; never gate on it. | +| `ValidatedDocuments` | project | Decision variable: labelled documents used for metrics and the ceiling for per-field `Documents`. If below total document count, unlabelled documents exist (1e). | +| `ModelVersion` | project | Decision variable for retrain completion. | +| `ErrorRate` | field, group | Report only and independent of `Precision`: wrong extractions count once, and misses count even when they do not lower `Precision`. Report manual-correction burden; diagnose direction with `Precision`/`Recall`. | +| `Quality` | field | Ignore. It is a coarse scale inconsistent with `ProjectScoreQuality`; never gate or report it per field. If asked, explain that the scales differ. | +| `ProjectScoreQuality` | project | Report only on the project line, using the UI's project label. | +| `FieldGroup`, `FieldId`, `Name` | field | Identity. Compare on `FieldId` (stable); report on `Name` (current display name — `null` for a deleted field, fall back to `FieldId`; see 1a). | +## Waiting for Retrain -## Waiting for retrain +Every input change—labellings, instructions, document upload/delete, or taxonomy edits—triggers a full retrain. Never compare metrics read mid-retrain. -Every change to model inputs — labellings, instructions, document upload/delete, taxonomy edits — triggers a full retrain. Metrics read mid-retrain are *pre*-change scores and corrupt every downstream comparison, so wait before each metrics read. +1. Record `ModelVersion` from the last metrics read **before** the change. +2. Wait 2 minutes, then run `uip ixp projects get-metrics <project-name> --output json`. +3. If `ModelVersion` is greater than the recorded value, proceed. Any increment counts; queued changes can skip versions. +4. Otherwise repeat step 2 with 2 minutes between checks, for 5 checks total (10 minutes). Do not use one long sleep, shorten the interval, or escalate. +5. If unchanged after the fifth check, stop polling and report that retraining did not complete. Mark carried-forward metrics as pre-change; never present them as post-change or roll back against them. -**Bounded wait. Poll on a fixed interval; never poll indefinitely:** +A failed read counts against the five-check budget and does not restart it. -1. Record `ModelVersion` from the last metrics read BEFORE the change. -2. Wait 2 minutes, then read `uip ixp projects get-metrics <project-name> --output json`. -3. `ModelVersion` **greater than** the recorded value → retrain is done, proceed. Any increment counts. Do NOT wait for a specific number: queued input changes can bump the version by more than one, so waiting for exactly *N*+1 polls until the budget dies when the server jumps straight to *N*+2. -4. Otherwise repeat step 2 — **2 minutes between checks, 5 checks in total** (10 minutes). Do NOT use a single long sleep, and do NOT escalate or shorten the interval between checks. -5. Still unchanged after the 5th check → **stop polling** and report that the retrain did not complete. Metrics you carry forward predate the change: label them as such, never present them as the post-change measurement, and never roll back instructions on a comparison against them. +## Step 1 — Setup (once) -A read that fails counts against the budget like any other attempt, and never restarts it. +### 1a. Get Baseline Metrics -## Step 1 — Setup (once, before the loop) - -### 1a. Get baseline metrics - -If documents were just labelled (or uploaded, or the taxonomy was edited), wait out the resulting retrain before reading metrics — apply the bounded wait in [Waiting for retrain](#waiting-for-retrain). +If documents were labelled, uploaded, or the taxonomy was edited, wait using [Waiting for retrain](#waiting-for-retrain), then run: ```bash mkdir -p /tmp/ixp/<project-name>/{docs,text,taxonomies,prompts} uip ixp projects get-metrics <project-name> --model-version latest --output json ``` -`--model-version latest` is deliberate: the baseline is the latest trained version — the model your instruction edits retrain — not the `live` tag (Critical Rule 21: the version follows the question). - -Note the `ModelVersion` from this baseline read — later iterations check that it advances after each `fields update-prompts` / `groups update-prompts` (see step 2e). If the value here looks identical to a known pre-labelling version, the retrain may still be in flight; re-fetch under the bounded wait in [Waiting for retrain](#waiting-for-retrain), then proceed with whatever it returns. - -Save the full per-field `Fields` array as `baseline_metrics`. This is the starting point you compare against. (For a validated model, get-metrics Data is flat — `Fields`/`FieldGroups`/`ValidatedDocuments` are top-level. An unvalidated model returns `Data: { Metrics: null }` instead — re-fetch under the bounded wait above.) +Use `--model-version latest`, not `live`: the baseline must be the latest trained version that instruction edits retrain (Critical Rule 21: the version follows the question). Record `ModelVersion` and save the complete per-field `Fields` array as `baseline_metrics`. If the result is unvalidated (`Data: { Metrics: null }`), refetch under the bounded wait. If it resembles a known pre-labelling version, refetch under the bounded wait and use the returned result. -**Field names:** each `Fields` entry carries both `FieldId` and `Name`, so report and compare fields straight from the metrics — do NOT fetch the taxonomy to build an id→name map. Three rules: +**Field names:** each `Fields` entry carries both `FieldId` and `Name`, so report and compare straight from the metrics — do NOT fetch the taxonomy to build an id→name map. Three rules: -- **Compare on `FieldId`, report on `Name`.** `FieldId` is stable; `Name` reflects the taxonomy as it is now, so a field renamed since an older version was scored reads back under its current name. -- **`Name` is null** when the service could not resolve it (e.g. the field was deleted after that version was scored). Fall back to `FieldId` — never skip the field. -- **When two fields share a `Name`, qualify it with `FieldGroup`.** Display names are unique only *within* a group, so the same label can sit under two of them — print those rows as `<FieldGroup> / <Name>` or the reader cannot tell which one a score belongs to. This changes how you print the row, nothing else: the comparison still keys on `FieldId`. +- **Compare on `FieldId`, report on `Name`.** `FieldId` is stable; `Name` reflects the current taxonomy, so a field renamed since an older version was scored reads back under its current name. +- **`Name` is `null`** when the service could not resolve it (e.g. the field was deleted after that version was scored). Fall back to `FieldId`; never skip the field. +- **When two fields share a `Name`, qualify it as `<FieldGroup> / <Name>`.** Display names are unique only within a group. This changes only how you print the row — the comparison still keys on `FieldId`. -### 1b. Check model configuration +### 1b. Check Model Configuration -If many fields have low scores across the board, the model configuration may be wrong (e.g., no table pre-processing for table-heavy documents). View sample documents and check if the current config matches the document type. If not, reconfigure: +If many fields score poorly, inspect sample documents and verify configuration, especially table preprocessing. If wrong, run: ```bash uip ixp projects configure-model <project-name> \ @@ -106,19 +85,21 @@ uip ixp projects configure-model <project-name> \ --output json ``` -See the [Project Setup Guide](project-setup-guide.md) Step 2 for the decision table. +Use the decision table in [Project Setup Guide](project-setup-guide.md) Step 2. -### 1c. Get taxonomy +### 1c. Get Taxonomy + +Run: ```bash uip ixp projects get-taxonomy <project-name> --output json ``` -Save to `/tmp/ixp/<project-name>/taxonomies/v1.json`. Output is `{ status, dataset: { entity_defs, label_groups } }` (raw snake_case); each `dataset.label_groups[]` holds `label_defs` with their fields and current `instructions`. These per-field instructions are what you'll be iterating on. Increment the version after each prompt update (v2, v3, …). +Save the raw snake_case response (`{ status, dataset: { entity_defs, label_groups } }`) to `/tmp/ixp/<project-name>/taxonomies/v1.json`. Each `dataset.label_groups[]` contains `label_defs`, fields, and current `instructions`. Increment the taxonomy version after each prompt update. Pass the field `name` to `fields update-prompts`. -The field `name` (e.g., `"Invoice Number"`, `"Description"`) is what you pass to `fields update-prompts --updates`. +### 1d. Read Sample Documents -### 1d. Read sample documents (2-3 documents) +Read 2–3 documents. Run: ```bash uip ixp documents list <project-name> --output json @@ -127,105 +108,77 @@ uip ixp documents list <project-name> --output json uip ixp documents download <project-name> <document-id> -o /tmp/ixp/<project-name>/docs/sample --output json ``` -The `download` command auto-detects format and appends the correct extension — read the resolved `Path` from the response. View the document with the **Read tool** — one full Read per document, **no `pages` parameter** (returns text + image natively for PDF/PNG/JPG). Files persist across sessions — check for existing files before downloading. - -### 1e. Check for unlabelled documents +The download command detects format and appends the extension. Read the resolved `Path`; use the Read tool once per document, with no `pages` parameter, so PDF/PNG/JPG text and image are returned natively. Reuse existing files. -Compare the document list against the metrics. If the metrics show fewer `ValidatedDocuments` than the total document count, some documents have no confirmed labellings (e.g., newly added documents). Review and label them first using the [Label Documents Guide](label-documents-guide.md), then re-fetch metrics under the bounded wait in [Waiting for retrain](#waiting-for-retrain) before starting the loop. +### 1e. Check for Unlabelled Documents ---- +Compare the document list with metrics. If `ValidatedDocuments` is below the total document count, label unlabelled documents first using [Label Documents Guide](label-documents-guide.md), then wait and refetch metrics under [Waiting for retrain](#waiting-for-retrain). ## Step 2 — Optimization Loop -Repeat the following for each iteration (up to max iterations): - -### 2a. Diagnose fields and field groups - -Use the current metrics (baseline on first iteration, post-relabel metrics on subsequent iterations). The metrics include both `FieldGroups` (per-group scores) and `Fields` (per-field scores). - -**Field group diagnosis:** Check `FieldGroups` first. If an entire group has low F1, the group-level instructions may need updating with `--groups` rather than fixing individual fields. - -**Per-field diagnosis:** Identify individual fields with F1 < 0.7 as targets. Diagnose each: - -1. **Classify the action:** - - `Documents = 0` AND `F1 = 0` → **SKIP** - - `Documents < 1` → **SKIP** - - Otherwise → **REFINE** - -2. **Diagnose the problem type** from the `Precision`/`Recall` split — `F1` says *how bad*, the split says *what to write*: - - `Precision < Recall` significantly → **PRECISION** — model extracts wrong values - - `Recall < Precision` significantly → **RECALL** — model misses the field - - Otherwise → **BOTH** — rewrite entirely +Repeat through the maximum iteration count. -3. **Record the field's `Annotations` count** next to the diagnosis. It does not change the classification, but it sets how much of the following delta you are entitled to believe (2f), so carry it forward rather than re-fetching it later. +### 2a. Diagnose Fields and Groups -Print a diagnosis summary with one row per field — name, `F1`, `Precision`, `Recall`, `ErrorRate`, `Annotations`, `Documents`, diagnosis — plus the group rows and the project line (`ProjectScore` / `ProjectScoreQuality` / `ValidatedDocuments` / `ModelVersion`). Ignore the `Quality` labels ([What `get-metrics` returns](#what-get-metrics-returns-and-which-values-decide)). +Use baseline metrics on iteration 1 and post-relabel metrics thereafter. Check `FieldGroups` first; a low-scoring group may require `--groups`. Target fields with `F1 < 0.7`: -If no fields need REFINE, stop — the project is already at target quality. +1. `Documents = 0` AND `F1 = 0` → **SKIP**. +2. `Documents < 1` → **SKIP**. +3. Otherwise → **REFINE**. +4. For REFINE fields, classify the `Precision`/`Recall` split: + - `Precision < Recall` significantly → **PRECISION** (wrong values). + - `Recall < Precision` significantly → **RECALL** (misses). + - Otherwise → **BOTH** (full rewrite). +5. Record `Annotations`; it does not alter diagnosis but sets the field's regression threshold in 2f. -### 2a-check. Check for labelling gaps (before writing instructions) +Print one row per field with name, `F1`, `Precision`, `Recall`, `ErrorRate`, `Annotations`, `Documents`, and diagnosis; include group rows and the project line (`ProjectScore` / `ProjectScoreQuality` / `ValidatedDocuments` / `ModelVersion`). Ignore field `Quality`. Stop if no fields need REFINE. -For each REFINE field with **Recall < 0.5**, check whether the problem is a bad prompt or a missing label: +### 2a-check. Check Labelling Gaps -1. Look at the sample document images you already have from Step 1d -2. For each low-recall field, check: **can you see this field's value in the document?** - - If yes, the model may have predicted it correctly but it wasn't confirmed in a previous round → re-fetch predictions and review those fields again - - If the field is genuinely not visible in the document → it's a prompt/recall issue, handle with instruction changes +For every REFINE field with `Recall < 0.5`, inspect already-downloaded sample images and determine whether the value is visible. If visible, refetch predictions and review the field; a correct prediction may previously have been skipped. If not visible, treat it as a prompt/recall issue. -**If you find previously skipped predictions that are actually correct**, confirm them now using `labelling confirm --fields` for those specific documents and fields, then re-fetch metrics under the bounded wait in [Waiting for retrain](#waiting-for-retrain) before continuing. +If skipped predictions are correct, confirm only those documents and fields with `labelling confirm --fields`, then wait and refetch metrics under [Waiting for retrain](#waiting-for-retrain) before writing instructions. If no gap exists, write instructions directly. -**If no labelling gaps are found**, proceed directly to writing instructions. +### 2b. Write Improved Instructions -### 2b. Write improved instructions +Rewrite each REFINE field: -For each field marked REFINE, rewrite its `instructions`: +- **PRECISION**: specify what to extract and what not to extract. +- **RECALL**: describe where to find it. +- **BOTH**: rewrite what, where, and what to avoid. -- **PRECISION** → Be more specific about WHAT to extract and what NOT to extract -- **RECALL** → Better describe WHERE to find the field -- **BOTH** → Full rewrite — what, where, what to avoid +Requirements: -**Instruction quality standards:** +- Focus on what and where; do not specify format because the entity_def type handles it. +- Use at least 120 characters. +- Include a location hint: section, header, table, top of, labeled, or near. +- Include a real document value when visible; use no example when not visible. For example: `Example: '2106732'`. +- Disambiguate similar fields, including what NOT to extract. +- Do NOT include format patterns such as `Format: MM/DD/YYYY`. +- Reference one field only. +- NEVER reference page numbers; use headings or labels. +- From iteration 2 onward, do not repeat a failed instruction; change wording, location hints, length, or negative examples. -Focus on **what** to extract and **where** to find it. Do NOT specify format — the entity_def (field type) already handles that. +### 2c. Update Instructions -- **Minimum length**: 120+ characters. Short instructions like "Extract the date" are too vague. -- **Location hint**: describe WHERE in the document (section, header area, table, near a label). Keywords: "section", "header", "table", "top of", "labeled", "near". -- **Real example**: include an actual value from the documents (e.g., "Example: '2106732'", "Example: 'SINV0077023'"). -- **Disambiguation**: if similar fields exist, clarify what NOT to extract (e.g., "Do NOT confuse with PO Number"). -- **No format patterns**: do NOT include "Format: MM/DD/YYYY" or similar — the entity_def type (Date, Monetary, Text) already defines the format. Adding format in instructions creates conflicting signals. - -**Good instruction** (145 chars): -> "The unique invoice identifier, found in the header area near the top-right, labeled 'Invoice #' or 'Invoice Number'. Example: '2106732'." - -**Bad instruction** (25 chars): -> "Extract the invoice number" - -**For fields visible in documents** — include location and a real example from the actual documents. -**For fields NOT visible** — use a generic instruction with no example: "Extract [what] from this document, as it appears on the page." - -**Additional rules:** - -1. NEVER reference specific page numbers — use section headings or labels -2. Each instruction targets one specific field (e.g., "Invoice Number", "Invoice Date") -3. On iteration 2+, do NOT repeat the same instruction that failed last time — try a different approach (different wording, different location hints, add negative examples) - -### 2c. Update instructions - -Use **field names** for `--fields` and **label_def names** for `--groups`: +Use field names with `--fields` and label_def names with `--groups`. Run: ```bash cat > /tmp/ixp/<project-name>/prompts/field_updates.json << 'FIELDS_EOF' [ {"name": "Invoice Number", "instructions": "The unique document identifier, found in the header area top-right. Example: 2106732, QC006."}, - {"name": "Invoice Date", "instructions": "The date the invoice was issued. Use the exact format as written in the document. Found near the invoice number."} + {"name": "Invoice Date", "instructions": "The date the invoice was issued. Found near the invoice number."} ] FIELDS_EOF uip ixp fields update-prompts <project-name> \ --updates "$(cat /tmp/ixp/<project-name>/prompts/field_updates.json)" \ --output json +``` + +If group instructions also need changing, run: -# If group instructions also need updating, run a second command. +```bash cat > /tmp/ixp/<project-name>/prompts/group_updates.json << 'GROUPS_EOF' [ {"name": "Invoice", "instructions": "General invoice header fields including number, dates, payment terms, and totals."} @@ -237,71 +190,61 @@ uip ixp groups update-prompts <project-name> \ --output json ``` -The second call is optional — skip it if the group instructions don't need changing. - -**Post-update verification:** After the update, re-fetch the taxonomy, save it as the next version, and verify that field counts per label_def are unchanged: +After updating, wait as required, then run: ```bash uip ixp projects get-taxonomy <project-name> --output json > /tmp/ixp/<project-name>/taxonomies/v<N>.json ``` -Compare the number of fields in each updated label_def against the previous version. If any fields are missing, **STOP the workflow immediately** and report to the user — the taxonomy was corrupted and needs manual restoration. The previous taxonomy version has the old instructions for rollback. +Verify field counts in every updated label_def are unchanged from the previous version. If any field is missing, STOP immediately, report taxonomy corruption, and restore manually using the previous taxonomy version. -### 2d. Review and confirm predictions for all documents +### 2d. Review and Confirm All Documents -Wait out the retrain triggered by the updated instructions ([Waiting for retrain](#waiting-for-retrain)), then review predictions for all documents using the [Label Documents Guide](label-documents-guide.md). The updated prompts should produce better predictions — review each document's predictions against the actual content and confirm the correct ones. Documents with incorrect predictions are skipped (their old labels remain). +Wait for the instruction-triggered retrain using [Waiting for retrain](#waiting-for-retrain). Review every document's predictions against its content using [Label Documents Guide](label-documents-guide.md), confirming correct predictions and skipping incorrect ones so old labels remain. -### 2e. Wait and get new metrics +### 2e. Wait and Get Metrics -Wait out the retrain triggered by the new labellings ([Waiting for retrain](#waiting-for-retrain)), then: +Wait for the labelling-triggered retrain using [Waiting for retrain](#waiting-for-retrain), then run: ```bash uip ixp projects get-metrics <project-name> --output json ``` -If `ModelVersion` hasn't advanced since the last check, keep re-reading under that same bounded budget. When the budget runs out, record the metrics you have and move on to step 2f — do NOT stall the iteration waiting for a version bump. +If `ModelVersion` has not advanced, continue under the same bounded budget. If the budget expires, record available metrics and proceed to 2f; do not stall. -### 2f. Compare and decide +### 2f. Compare and Decide -Compare the new metrics against the **previous iteration** at both levels — the fields you touched, and the project as a whole. +Compare the complete new payload with the previous iteration at touched-field and project-wide levels. -#### Regression noise floor +#### Regression Noise Floor -With few `Annotations`, `F1` moves in jumps: a single annotation flipping by chance jumps it as far as a genuinely worse instruction would, and the number alone cannot tell the two apart. The rollback threshold therefore scales with the sample: +For each field: ```text regression_threshold = max(0.1, 1 / Annotations) ``` -That is 0.2 at `Annotations` = 5 — one flipped annotation is not evidence — and the flat 0.1 from `Annotations` = 10 up. Fields whose `Annotations` differ get different thresholds in the same iteration; that is intended, not an inconsistency. - -**Below the threshold is not "no change" — it is "not measurable yet".** Do not report a sub-threshold move as an improvement either. If a field keeps drifting sub-threshold across iterations and its `Annotations` is small, no prompt rewrite can be evaluated — but which remedy to report depends on *why* the sample is small. - -**A small `Annotations` has two causes with opposite remedies.** `Annotations` counts reviewed **extractions**, not documents — one document can contribute several — so it cannot be compared against a document count directly. Compare the field's own `Documents` against the project-level `ValidatedDocuments`: +A single annotation can move `F1` sharply. A move below the field's threshold is **not measurable**, not an improvement. If sub-threshold drift continues, report the appropriate remedy based on sample size. -- **`Documents` equal to `ValidatedDocuments`** → this field already has evidence on every labelled document; the sample is as large as the data allows. Tag it **UPLOAD**. -- **`Documents` below `ValidatedDocuments`** → some labelled documents carry no evidence for this field, and the payload cannot say why — never reviewed there, or reviewed and skipped because the prediction was wrong. Tag it **REVIEW** — the review pass ([Label Documents Guide](label-documents-guide.md)) shows which in seconds, and 2a-check's `Recall < 0.5` gate would never trigger it. Even when the review finds nothing to add, confirming that costs a glance, while an unreviewed document left unfound caps the field for good. +`Annotations` counts reviewed extractions, not documents. Compare the field's `Documents` with project-level `ValidatedDocuments`: -Both tags are **final-report lines, not loop actions**: the loop runs on to its normal stopping criteria — never pause mid-run to ask for documents or to review — and the report then says plainly that a tagged field's score cannot rise further until its sample grows. +- Equal → tag **UPLOAD**: evidence covers every labelled document, so the sample can grow only with more documents. +- Below → tag **REVIEW**: some labelled documents have no label for this field. Review them using [Label Documents Guide](label-documents-guide.md), whether or not new labels are found. -`Annotations / Documents` is the average number of extractions per document — about 1 for a single-value field, higher under a repeatable group. +These are final-report tags, not loop actions. Continue normally and report that the score cannot rise until its sample grows. `Annotations / Documents` is the average extractions per document. -**Selective regression check:** For each field you updated this iteration, compare its `F1` drop against **that field's** `regression_threshold`: +#### Selective Regression and Collateral Checks -- **Regressed fields** (drop > their threshold): roll back ONLY those fields' instructions to the previous iteration's version. Keep the improved instructions for fields that gained or held steady. -- **Improved/unchanged fields**: keep their new instructions. +For each field updated this iteration, compare its `F1` drop with its own threshold: -**Collateral check (fields you did NOT touch):** per-field checks only cover the fields you edited, but a `groups update-prompts` edit rewrites the parent `label_def` and so moves every field under it. +- Drop greater than threshold → roll back only that field. +- Improved or within threshold → keep it. -Do **not** gate this on `ProjectScore`. It is an average over fields — observed to be the unweighted mean of the per-field `F1` values — so it carries nothing the `Fields[]` array does not, and it divides a single field's move by the field count, burying a real regression below its own noise. Diff **every** field against the previous iteration instead, each against **its own** `regression_threshold`: +Diff every field, not `ProjectScore`, against its own threshold. For an unedited field that regresses beyond threshold, report its name and delta. Roll it back only if it shares a field group changed by `groups update-prompts` this iteration; otherwise keep the iteration and recheck next round because two reads cannot prove causation. -- An **edited** field regressed beyond its threshold → roll that field back, as above. -- An **unedited** field regressed beyond its threshold → collateral damage. Report it by name with its delta. Roll it back only when it shares a field group with a `groups update-prompts` edit you made this iteration — that is the one interaction with a mechanical cause. Otherwise **keep the iteration and re-check next round**: two metric reads cannot establish that your edit caused the move, and discarding edits that individually passed destroys work on a guess. - -If any fields regressed, do a selective rollback: +For selective rollback, run: ```bash -# Only include the regressed fields, not the whole iteration cat > /tmp/ixp/<project-name>/prompts/rollback.json << 'FIELDS_EOF' [{"name": "Vendor Address", "instructions": "previous instruction for this field only"}] FIELDS_EOF @@ -311,23 +254,19 @@ uip ixp fields update-prompts <project-name> \ --output json ``` -Wait out the retrain ([Waiting for retrain](#waiting-for-retrain)). On the next iteration, try a **different approach** for the regressed fields only (different wording, shorter instruction, fewer examples). - -**Rollback caveat:** Rollback restores the previous instructions but the model needs to retrain. Expect only **partial recovery** — prefer small-scope iterations (few fields at a time). +Wait for retrain. On the next iteration, use a different approach for regressed fields only. Rollback restores instructions, but retraining may yield only partial recovery; prefer small-scope iterations. -**No regression:** Accept the iteration. Update `previous_metrics` (the complete payload again, not just F1) and `previous_instructions` with the new values. +If no regression occurs, accept the iteration and update `previous_metrics` with the complete payload and `previous_instructions` with the new values. -**Stopping criteria — stop the loop if:** +Stop when: -- All fields meet the user's target F1 (default: 0.7) -- Max iterations reached -- No fields improved by more than their own `regression_threshold` in the last 2 consecutive iterations (diminishing returns — a run of sub-threshold moves is not progress) - ---- +- All fields meet the user's target F1 (default: 0.7). +- The maximum iteration count is reached. +- No field improves by more than its own `regression_threshold` for 2 consecutive iterations. ## Step 3 — Final Report -After the loop ends, print a summary: +Print: ```text Optimization complete after N iterations. Model version V1 -> V2. @@ -347,6 +286,4 @@ Fields whose regression_threshold sits above the flat 0.1 (too few Annotations t Labelling gaps fixed: [list any fields re-labelled in 2a-check] ``` -`ErrorRate` is the manual-correction burden left; `Annotations` tells a real plateau from an unmeasurable one. - -If fields still need work, suggest the user run another round with more iterations. For any field in the *too-few-`Annotations`* list, say plainly that its score cannot rise further until its sample grows, and which remedy grows it — **UPLOAD** (more documents) or **REVIEW** (the documents where it carries no label). +Report `ErrorRate` as remaining manual-correction burden and `Annotations` as evidence of whether a plateau is measurable. If fields remain below target, suggest another round with more iterations. For each too-few-`Annotations` field, state that its score cannot rise until its sample grows and identify **UPLOAD** or **REVIEW** as the remedy. diff --git a/skills/uipath-ixp/references/label-documents-guide.md b/skills/uipath-ixp/references/label-documents-guide.md index 5c4a92ce3c..626b63ee53 100644 --- a/skills/uipath-ixp/references/label-documents-guide.md +++ b/skills/uipath-ixp/references/label-documents-guide.md @@ -5,95 +5,81 @@ Reusable workflow for labelling documents in an IXP project. Used by: - [Project Setup](project-setup-guide.md) — initial labelling after creating a project - [Improve Prompts](improve-prompts-guide.md) — reviewing predictions during optimization -You act as a **reviewer** — IXP generates predictions, you validate them field-by-field against the document. Only fields that are correct get confirmed. Fields that are wrong are left unannotated. Fields where the prediction found the right location but the value is OCR-mangled get corrected. +Act as a **reviewer**: IXP generates predictions; validate them field-by-field against the document. Confirm only correct fields. Leave wrong fields unannotated. Correct only OCR-mangled values when IXP found the right location. ## Step 1 — Get Documents and Taxonomy +Run: + ```bash mkdir -p /tmp/ixp/<project-name>/{docs,text,taxonomies,prompts} uip ixp documents list <project-name> --output json uip ixp projects get-taxonomy <project-name> --output json ``` -Save the taxonomy to `/tmp/ixp/<project-name>/taxonomies/v1.json` (increment the version on each re-fetch). - -From the taxonomy (raw snake_case: field groups/fields under `Data.dataset.label_groups`, types under `Data.dataset.entity_defs`), review the field groups and field types so you understand what each predicted field represents. +Save the taxonomy as `/tmp/ixp/<project-name>/taxonomies/v1.json`; increment the version on each re-fetch. Review raw snake_case field groups and fields under `Data.dataset.label_groups`, and types under `Data.dataset.entity_defs`. ## Step 2 — Process Each Document -For each document from the list, process one at a time: get predictions, download image/text, review, confirm. +Process documents one at a time: get predictions, download or reuse the file, review every field, and confirm only approved fields. + +### 2a. Get predictions -### 2a. Get predictions for this document +Run: ```bash uip ixp labellings get-predictions <project-name> <document-id> --output json ``` -This returns `Data: { ProjectName, TotalDocuments, DocumentsWithPredictions, Predictions[] }`. Each `Predictions[]` entry is `{ DocumentId, Labels[] }` (for a single-document call, `Predictions[0]`). Each label is `{ Name, Occurrence, Fields[] }`, and each field has `FieldId`, `FieldName`, `FormattedValue`. `Occurrence` is the explicit 0-based index used for `--occurrence`/`--updates`, valid **for this read only** — see [Occurrence numbers are read-scoped](#occurrence-numbers-are-read-scoped). +Read `Data: { ProjectName, TotalDocuments, DocumentsWithPredictions, Predictions[] }`. Each `Predictions[]` entry is `{ DocumentId, Labels[] }` (for a single-document call, `Predictions[0]`). Each label is `{ Name, Occurrence, Fields[] }`; each field has `FieldId`, `FieldName`, `FormattedValue`. -The response also carries `ModelVersion` — the model version that produced these predictions. Note it: pass it to `confirm --model-version` in step 2d so a retrain mid-review can't silently change the values `confirm` stamps. +Treat `Occurrence` as the explicit 0-based index for `--occurrence`/`--updates`, valid **for this read only**; see [Occurrence numbers are read-scoped](#occurrence-numbers-are-read-scoped). Record `ModelVersion` and pass it to every `confirm --model-version` call so a retrain cannot silently change reviewed values. -### 2b. Download the document file +### 2b. Download the document -- **If the file already exists** in `/tmp/ixp/<project-name>/docs/` from a previous session, reuse it — do NOT re-download. -- **Otherwise, download:** +If the file exists in `/tmp/ixp/<project-name>/docs/`, reuse it and do NOT re-download. Otherwise run: ```bash uip ixp documents download <project-name> <document-id> -o /tmp/ixp/<project-name>/docs/<document-id> --output json ``` -Use the document ID as the filename. Pass `-o` **without an extension** — the CLI detects the actual format (PDF, PNG, JPG, …) from the file content and appends the correct extension. Read the resolved `Path` from the response and use that for the next step. Files persist across sessions — check for existing files before downloading. +Pass `-o` without an extension. Read the response's resolved `Path` and use it next; the CLI detects the content format and appends the correct extension. Files persist across sessions. + +### 2c. Review every predicted field + +Use the **Read tool** once on the whole file with no `pages` parameter. A full Read returns text + image natively for digital and scanned PDF/PNG/JPG documents; do not install PDF tools. Understand the layout, then evaluate every predicted field. -### 2c. Review predictions field-by-field +Use exactly one verdict per field: -Use the **Read tool** to view the document file (read the whole document in one call, no `pages` parameter — a full Read returns text + image natively for digital and scanned PDF/PNG/JPG docs; no PDF tools to install), then review each predicted field against the document: +- **CONFIRMED** — the value matches literally or in the data type's normalized form. Capitalization, whitespace, and type normalization are acceptable. For example, a `Date` may read back as `2022-06-21T00:00:00Z` for `21-JUN-22`, and a `Monetary Quantity` as `114.91 AUD`. Compare by reading; do not write a conversion/checking script. See [CLI Reference § Normalized output formats](cli-reference.md#normalized-output-formats). +- **CORRECTED** — OCR mangling only: the prediction found the right field and location, the bytes-on-page are correct, and only the transcription is garbled. A magnitude difference is OCR garble, never type normalization (Rule 8). Do not correct formatting, wrong booleans, wrong inferred/computed values, or wrong source selection. +- **MISSING** — IXP predicted an empty `FormattedValue` **and** the field is genuinely absent. Both conditions are required. A non-empty prediction for an absent field is NOT CONFIRMED, never MISSING (Critical Rule 12). +- **NOT CONFIRMED** — every other error: wrong literal value, wrong source, hallucination, wrong boolean, wrong inferred/computed value, or a value not present in the document. Leave it unannotated and do not use `--corrections`; improve the prompt instead. `--corrections` is OCR-only (Critical Rule 8). -1. **Look at the document** to understand the layout and where field values appear. -2. **For each predicted field**, assign one of four verdicts: - - **CONFIRMED** — the predicted value matches what is in the document, literally or in its data type's normalized form. Minor OCR-level differences (capitalization, whitespace) are acceptable, as is any difference that is purely the type's normalization — a `Date` reads back as `2022-06-21T00:00:00Z` for a page showing `21-JUN-22`, a `Monetary Quantity` as `114.91 AUD` for a page showing `114.91`. Compare the values by reading them; do not write a script to convert or check formats. See [CLI Reference § Normalized output formats](cli-reference.md#normalized-output-formats). - - **CORRECTED** — **OCR-mangled values only.** The prediction found the right field in the right location, the bytes-on-page are correct, but the text was garbled in transcription (e.g., `MSIÓÓÓ601020/` instead of `MSI0601020`, `lNGRAM` instead of `INGRAM`, or a misread digit in a number — page `£7,300.00` predicted as `£730.00`). The reference is correct, only the literal characters need fixing. **A number whose magnitude differs from the page is OCR garble, never type normalization** (Rule 8). Do NOT use CORRECTED to restore a page's date or amount formatting, for booleans that came back with the wrong answer, inferred/computed values that came back wrong, or any case where IXP picked the wrong source on the page — those are NOT CONFIRMED. - - **MISSING** — IXP predicted **no value** (empty `FormattedValue`) AND the field is genuinely absent from the document. Both conditions must hold. If IXP predicted a value but the field isn't actually in the document, that's NOT CONFIRMED, not MISSING — Critical Rule 12 forbids overriding a non-empty prediction with "missing". - - **NOT CONFIRMED** — the prediction is wrong for any reason other than OCR mangling. Covers: wrong literal value on the right field, wrong-source extraction, hallucinated value, boolean came back with the wrong answer, inferred/computed value came back wrong, predicted a value the document doesn't contain. Left unannotated. Do NOT try to "fix" these with `--corrections` — `--corrections` is OCR-only (see Critical Rule 8). Improve the prompt instead. -3. **Report your verdict for every field.** Print a table per document: +Report every field in a table per document: ```text Document: <document-id> Field | Verdict | Reason -------------------------|---------------|----------------------------------------------- -Invoice Number | CORRECTED | OCR mangled "MSIÓÓÓ601020/" → "MSI0601020", top-right of page 1 -Invoice Date | CONFIRMED | Predicted "2018-02-28" matches document -Vendor Address | NOT CONFIRMED | Predicted "123 Main St" but actual is "456 Oak Ave", top-left of page 1 -Has Signature | NOT CONFIRMED | Predicted "false" but signature visible bottom-right (boolean came back wrong — NOT CORRECTED) -Total After Tax | NOT CONFIRMED | Predicted "$1100.00" but Subtotal+Tax = "$1210.00" (inferred value wrong — NOT CORRECTED) -Terms of Payment | MISSING | IXP predicted no value AND field not visible in document -Discount | MISSING | IXP predicted no value AND no discount section on the page -Line Items > Description | CONFIRMED | Predicted "Widget A" matches row 1 in the table +<field> | <verdict> | <comparison, correction, or absence evidence> ``` -**Repeatable field groups produce one extraction per row.** `get-predictions` returns one label per row, each with an explicit 0-based `Occurrence` — `Line Items` on a multi-line invoice has N entries indexed 0..N-1. Read `Occurrence` directly — on an unlabelled document it matches document order, but once part of the group is confirmed it does not (see [Occurrence numbers are read-scoped](#occurrence-numbers-are-read-scoped)), so match rows to the document by their values, not by index. When validation differs across rows, give per-occurrence verdicts: +For CORRECTED, state the mangled value, corrected value, and location; the error must be character-level in the same field and location. For MISSING, state that the prediction was empty and how absence was verified. For NOT CONFIRMED, state the predicted value, actual value if visible, and location. -```text -Line Items > Description (occurrence 0) | CONFIRMED | "Widget A" matches line 1 -Line Items > Description (occurrence 1) | CONFIRMED | "Widget B" matches line 2 -Line Items > Description (occurrence 3) | NOT CONFIRMED | Predicted "Widget D" but line 4 shows "Widget Z" -``` +Repeatable field groups produce one extraction per row. `get-predictions` returns one label per row with a 0-based `Occurrence`; read it directly, but match rows to the document by values, not index. On an unlabelled document it matches document order; after partial confirmation it may not. Report differing rows separately. -For **CORRECTED** fields: state the mangled predicted value, the corrected value, and where it appears. The mistake must be at the character level — same field, same location, garbled bytes. -For **MISSING** fields: state that the prediction was empty AND describe how you verified the field is absent (e.g., "no payment-terms section anywhere in the document"). -For **NOT CONFIRMED** fields: state the predicted value, the actual value (if visible) and location. Includes any non-OCR mistake — wrong source, wrong boolean, wrong inferred value, hallucination, value the document doesn't contain. **Do NOT use `--corrections` to fix these** — improve the field's prompt instructions instead. +Build: -4. **Build two lists from the table:** - - **Submit field IDs** — all CONFIRMED + CORRECTED + MISSING fields (one combined list — the CLI applies the right semantic per field based on IXP's prediction) - - **Corrections JSON** — only CORRECTED fields: `[{"field_id":"...","value":"corrected text"}]` +- **Submit field IDs**: all CONFIRMED, CORRECTED, and MISSING fields in one combined list. +- **Corrections JSON**: CORRECTED fields only, `[{'field_id':'...','value':'corrected text'}]` (use valid JSON with double quotes in the command). ### 2d. Confirm and correct -Submit confirmed, corrected, and missing fields for this document — all in one `confirm` call. +Pass the reviewed `ModelVersion` as `-m <model_version>` to every `confirm` call, including narrowed `--occurrence`/`--updates` calls. If confirmation returns `PredictionVersionChangedError`, run 2a again, re-review, and confirm again. -**Pass the version you reviewed.** Add `-m <model_version>` (the `ModelVersion` from step 2a) to every `confirm` call below — the narrowed `--occurrence`/`--updates` forms included. If a retrain landed since you read the predictions, the confirm is rejected with `PredictionVersionChangedError` instead of stamping values you never saw — re-run step 2a, re-review this document, then confirm again. - -**If there are corrections:** +With corrections, run: ```bash uip ixp labellings confirm <project-name> <document-id> \ @@ -103,24 +89,22 @@ uip ixp labellings confirm <project-name> <document-id> \ --output json ``` -The `--fields` list includes CONFIRMED, CORRECTED, and MISSING field IDs together — the CLI writes the right annotation per field based on IXP's prediction (content → confirm, content with override → correct, empty → missing marker). The `--corrections` JSON overrides the predicted value for corrected fields while keeping their document references (bounding boxes). - -**If there are no corrections (all approved fields are exact matches):** +Without corrections, run: ```bash uip ixp labellings confirm <project-name> <document-id> \ --fields "<field_id_1>,<field_id_2>,<field_id_3>" -m <model_version> --output json ``` -If ALL predicted fields for a document are correct with no corrections needed, you can omit `--fields` to confirm every predicted field on **that one document** in a single call: +`--fields` may contain CONFIRMED, CORRECTED, and MISSING IDs. The CLI confirms content, applies a correction override while retaining document references/bounding boxes, or writes a missing marker when the prediction is empty. If every reviewed prediction is correct, run the per-document form: ```bash uip ixp labellings confirm <project-name> <document-id> -m <model_version> --output json ``` -This per-document form is fine **once you've reviewed the document and every field is correct** (2c). What you must NOT do is run `confirm` **without a `<document-id>`** — that confirms every document in the project at once, bypassing the per-document review loop. Confirming unreviewed predictions bakes wrong values into the labels, and because F1 compares predictions against those labels, **the metric reports 1.00 even when the confirmed values are wrong**. F1 alone is never evidence the values are correct. +Never run `confirm` without `<document-id>`: that confirms every project document without review. Wrong labels can make F1 report 1.00, so F1 alone is never evidence of correctness. -**If there are missing fields**, include their IDs in the same `--fields` list as the CONFIRMED and CORRECTED IDs. The `confirm` command applies one uniform rule per listed field: if IXP predicted content, the content is confirmed; if IXP predicted nothing, a missing marker is written. No separate call needed. +Include a MISSING field only when `get-predictions` shows no value (Critical Rule 12). For mixed approved fields, run: ```bash uip ixp labellings confirm <project-name> <document-id> \ @@ -130,11 +114,15 @@ uip ixp labellings confirm <project-name> <document-id> \ --output json ``` -**Only include a field in the `--fields` list for the MISSING case when IXP itself predicted nothing for it** — see Critical Rule 12. If IXP predicted a wrong value, omit the field entirely (don't list it). +Run this when a missing marker must be written directly, including when `confirm --fields` cannot reach a field with a prior annotation: -Use `labellings mark-missing <project-name> <document-id> --fields <ids>` to record a genuinely-missing field. It marks the listed fields directly, so it also handles the case where `confirm --fields` no-ops — a field with a prior annotation that the current prediction no longer includes (e.g., model behavior changed after a retrain), which `confirm` can't reach. Either records the missing marker; only do so when `get-predictions` shows IXP predicted no value for the field — never to override a wrong prediction. +```bash +uip ixp labellings mark-missing <project-name> <document-id> --fields <ids> +``` + +Use it only when IXP predicted no value; never override a wrong prediction. -**Per-occurrence confirm for repeatable groups.** When a repeatable group's verdicts differ across occurrences (some lines correct, some not), `confirm --fields a7c3e9105f2b4d86` is the wrong shape — it confirms `a7c3e9105f2b4d86` in **every** occurrence, including the wrong ones. Target each correct occurrence by index: +For repeatable groups with differing verdicts, do not use unscoped `--fields`; it applies to every occurrence. Target occurrences by index: ```bash # All predicted fields in occurrence 0: @@ -146,9 +134,7 @@ uip ixp labellings confirm <project-name> <document-id> \ --group "Line Items" --occurrence 2 --fields c4e1907a3b8f25d6 -m <model_version> --output json ``` -Occurrences not targeted carry forward whatever annotation they already had (so wrong predictions in untouched occurrences stay unannotated). - -**Confirm all the correct rows in one `--updates` call, not one `--occurrence` call per row** — every index in a single call resolves against the same read, whereas the second of two sequential calls is working from indices the first one invalidated ([Occurrence numbers are read-scoped](#occurrence-numbers-are-read-scoped)): +Confirm all correct rows in one `--updates` call, not separate `--occurrence` calls: ```bash uip ixp labellings confirm <project-name> <document-id> \ @@ -157,9 +143,9 @@ uip ixp labellings confirm <project-name> <document-id> \ --output json ``` -See [CLI Reference § Labellings](cli-reference.md#labellings) and Critical Rule 13. +Every index in one call resolves against the same read. See [CLI Reference § Labellings](cli-reference.md#labellings) and Critical Rule 13. -**Per-occurrence unconfirm.** `unconfirm` takes the same `--group`/`--occurrence`/`--updates` flags, so a wrong confirmation can be rolled back at the same granularity. `unconfirm --fields a7c3e9105f2b4d86` (no `--group`) removes `a7c3e9105f2b4d86` from **every** occurrence; scope it to one line with `--group "Line Items" --occurrence 2`, or several at once with `--group "Line Items" --updates '[…]'`, using the same 0-based indices. Without `--fields`, every annotated field in the targeted occurrence(s) is rolled back; with `--fields`, only those. See Critical Rule 14. +Run `unconfirm` with the same `--group`/`--occurrence`/`--updates` scope to roll back an incorrect confirmation. Unscoped `unconfirm --fields a7c3e9105f2b4d86` affects that field in every occurrence; scope it to one or several occurrences to limit the rollback. Without `--fields`, roll back every annotated field in the target occurrence(s). Use a fresh `get-predictions` index because partial confirmation can reorder occurrences. See Critical Rule 14. ```bash # Roll back only occurrence 2 of Line Items (every field in that line): @@ -167,59 +153,46 @@ uip ixp labellings unconfirm <project-name> <document-id> \ --group "Line Items" --occurrence 2 --output json ``` -Take the index from a **fresh** `get-predictions`: on a partly-confirmed group, the index that confirmed a row is usually not the index that rolls it back. - ### 2e. Move to the next document -Repeat steps 2a–2d for all documents in the list. +Repeat steps 2a–2d for every document in the list. ### Occurrence numbers are read-scoped -`get-predictions` does not return a repeatable group's rows in a fixed order. The server pairs each annotation with its prediction and lists the **matched pairs first**, then the still-unmatched predictions. So on a partly-confirmed group: +`get-predictions` lists matched annotation/prediction pairs first, then unmatched predictions. Confirmed rows therefore move to the front, as in the IXP UI; values and page locations are retained. Only a group with no annotations or with every row annotated reads in document order. -- confirmed rows sort to the front — confirm the third row of four and it reads back as `Occurrence` 0, with the other three shifted to 1, 2, 3; -- the same holds in the IXP UI, which shows the confirmed row first; -- nothing is lost or mis-assigned: the confirmed row keeps its own values and page location, it is only positioned differently in the read. +An `Occurrence` is valid only for the read that produced it, and any write invalidates it: -Only two states read back in document order: a document with no annotations on the group, and one where every row is annotated. - -The consequence for labelling: **an `Occurrence` value is only valid for the read that produced it, and any write to that group invalidates it.** So - -- put every occurrence you want to confirm (or unconfirm) in ONE `--updates` call — all of its indices resolve against the same read; -- if sequential per-occurrence calls are unavoidable, re-run `get-predictions` between them and re-locate each row by its field values; -- never reuse an index across a write, and report rows to the user by value ("the row with Description `Widget B`"), not as "row 3". +- put all target occurrences in one `--updates` call; +- if sequential calls are unavoidable, run `get-predictions` between them and re-locate rows by field values; +- never reuse an index after a write; +- report rows by value, not as a row number. ### Removing a document from the project -If a document is unusable (wrong document type, corrupted, duplicate), delete it instead of confirming or skipping: +For a wrong type, corruption, or duplicate, run deletion instead of confirming or skipping: ```bash uip ixp documents delete <project-name> <document-id> -y --output json ``` -`-y/--yes` is required (the CLI never prompts). `<document-id>` is the `DocumentId` from `documents list` (e.g., `3453547f3538febd.1fc885607f2aac621f8f2d3ef1847f22`). Pass it whole. Do NOT pass the AttachmentRef or the Filename. +`-y/--yes` is required. Use the whole `DocumentId` from `documents list`, not AttachmentRef or Filename. Deletion is irreversible and triggers model retraining; do not delete merely to skip labelling. -**Finding the DocumentId:** +Find the `DocumentId` as follows: | You have | How to get the DocumentId | |----------|---------------------------| -| Filename (e.g., `invoice-001.pdf`) | `uip ixp documents list <project-name> --output json --output-filter "Documents[?Filename=='invoice-001.pdf'].DocumentId \| [0]" --output plain` (rows are under `Documents` — the list is a paged envelope) | -| A distinctive predicted field value (e.g., Invoice Number `MSI0601020`) | `uip ixp documents list <project-name> --output json` for the ids, then `uip ixp labellings get-predictions <project-name> <document-id> --output json` per id until a `Labels[].Fields[].FormattedValue` matches. One call per document, so stop at the first match. | -| Nothing — need to find by content | `uip ixp documents list <project-name> --output json`, then `documents download` candidates and read with the Read tool | +| Filename | Run `uip ixp documents list <project-name> --output json --output-filter "Documents[?Filename=='invoice-001.pdf'].DocumentId \| [0]" --output plain` (rows are under `Documents` in a paged envelope). | +| A distinctive predicted field value | Run `uip ixp documents list <project-name> --output json`, then run `uip ixp labellings get-predictions <project-name> <document-id> --output json` per ID until `Labels[].Fields[].FormattedValue` matches; stop at the first match. | +| Nothing — need to find by content | Run `uip ixp documents list <project-name> --output json`, download candidates with `documents download`, and read them with the Read tool. | -`documents list` returns `Filename` alongside `DocumentId` (the original upload filename, or `null` if none was sent at upload time). When filenames aren't unique within the project, the JMESPath filter returns multiple IDs — review them with `documents download` before deleting. - -Deletion is irreversible and triggers a model retrain. Do NOT use deletion to skip documents you simply don't want to label — leave those unconfirmed instead. +`documents list` returns `Filename` with `DocumentId`; it may be the original upload filename or `null`. If filenames are not unique, review all IDs returned before deleting. ## Step 3 — Summary -After processing all documents, track progress and errors: - -- Do NOT stop on the first error — continue with remaining documents -- If a download or text fetch fails, skip the document and note the failure -- If confirmation fails, log the error and UID, then continue +Do not stop at the first error. Continue with remaining documents. If download or text fetch fails, skip the document and record the failure. If confirmation fails, log the error and UID, then continue. -At the end, report a full summary: +Report: ```text Labelling complete. @@ -228,15 +201,14 @@ Documents: N processed, M confirmed, K skipped (no predictions) Fields: X confirmed, Y corrected, W marked missing, Z not confirmed OCR Corrections Applied: - Doc <uid-1>: Invoice Number "MSIÓÓÓ601020/" → "MSI0601020" - Doc <uid-1>: Vendor Name "INGRAM NTCRO INC" → "INGRAM MICRO INC" - Doc <uid-3>: Bill-To Address "123 Mam St" → "123 Main St" + Doc <uid-1>: <field> "<mangled>" → "<corrected>" + Doc <uid-3>: <field> "<mangled>" → "<corrected>" Marked Missing (IXP predicted empty AND field absent from document): - Doc <uid-2>: Terms of Payment - Doc <uid-4>: Discount + Doc <uid-2>: <field> + Doc <uid-4>: <field> Not Confirmed (skipped): - Doc <uid-3>: Total Amount — predicted "500.00" but actual is "5000.00" (bottom-right, page 1) - Doc <uid-5>: Vendor Address — predicted "123 Main St" but actual is "456 Oak Ave" + Doc <uid-3>: <field> — predicted "<value>" but actual is "<value>" (<location>) + Doc <uid-5>: <field> — predicted "<value>" but actual is "<value>" ``` diff --git a/skills/uipath-ixp/references/project-setup-guide.md b/skills/uipath-ixp/references/project-setup-guide.md index 2251445ee1..8b3501d2f2 100644 --- a/skills/uipath-ixp/references/project-setup-guide.md +++ b/skills/uipath-ixp/references/project-setup-guide.md @@ -1,46 +1,40 @@ # Project Setup Guide -Complete workflow for creating a **new** IXP project, labelling all documents, and getting initial metrics. Run all steps end-to-end automatically. Deployment is a separate, optional final step — [Deployment Guide](deployment-guide.md). +Create a **new** IXP project, label its documents, and obtain initial metrics end-to-end automatically. Deployment is optional and separate: [Deployment Guide](deployment-guide.md). > **Wrong page if the project already exists.** Use `uip ixp documents upload <project-name> <file>` — see [CLI Reference § Uploading documents](cli-reference.md#uploading-documents-to-an-existing-project). ## Step 1 — Create the Project -If the user provides a name, use it. If not, generate a temporary name (e.g., `ixp_project_NNNN` with a random number) — the project will be renamed in Step 3 after the taxonomy reveals the document type. +Use the user-provided name. If none is provided, generate a temporary name such as `ixp_project_NNNN` with a random number; rename it in Step 3 after the taxonomy reveals the document type. `<folder-path>` is filtered to supported document files — see [CLI Reference § Supported document files](cli-reference.md#supported-document-files). -`<folder-path>` is filtered to supported document files — see [CLI Reference § Supported document files](cli-reference.md#supported-document-files). - -**Option A — Auto-suggest taxonomy (default):** +For auto-suggested taxonomy, run: ```bash uip ixp projects create "<name>" <folder-path> --output json ``` -If the user specified what to extract, add `-d` for a better taxonomy suggestion: +If the user specified what to extract, run: ```bash uip ixp projects create "<name>" <folder-path> -d "<what to extract>" --output json ``` -This uploads documents and auto-suggests a taxonomy based on the document content (and the description if provided). - -**Option B — Blank project + import taxonomy from file:** +These commands upload documents and auto-suggest a taxonomy from their content and, when provided, the description. -If the user provides a taxonomy file, create a blank project and import separately: +If the user provides a taxonomy file, run: ```bash uip ixp projects create "<name>" <folder-path> --skip-taxonomy --output json uip ixp projects import-taxonomy <project-name> <taxonomy-file> --output json ``` -The taxonomy file can be in either format — the CLI auto-detects based on which keys are present: - -- `{ "field_types": [...], "label_group": {...} }` — use when importing a taxonomy suggested by a previous `project create` run -- `{ "entity_defs": [...], "label_groups": [...] }` — use when importing a taxonomy file provided by the user, or cloning from an existing project. `projects get-taxonomy` returns these under a `dataset` wrapper (`{ status, dataset: { entity_defs, label_groups } }`); `import-taxonomy` reads `entity_defs`/`label_groups` at the **top level**, so pass the inner `dataset` object (e.g. `jq .Data.dataset`), not the whole response +The CLI detects either format from its keys: -Use the `ProjectName` from the create output for all subsequent commands. This is the lowercase slug with UUID and `-ixp` suffix (e.g., `my_invoices-f1afa9ef-ixp`), NOT the Title. +- `{ "field_types": [...], "label_group": {...} }` — taxonomy suggested by a previous `project create` run. +- `{ "entity_defs": [...], "label_groups": [...] }` — user-provided taxonomy or one cloned from an existing project. `projects get-taxonomy` returns these under a `dataset` wrapper (`{ status, dataset: { entity_defs, label_groups } }`); `import-taxonomy` requires `entity_defs`/`label_groups` at the **top level**, so pass the inner `dataset` object (for example, `jq .Data.dataset`), not the whole response. -Create the working directory using the returned `ProjectName`: +Use `ProjectName` from the create output for all subsequent commands. It is the lowercase slug with UUID and `-ixp` suffix, not the Title. Run: ```bash mkdir -p /tmp/ixp/<project-name>/{docs,text,taxonomies,prompts} @@ -48,23 +42,23 @@ mkdir -p /tmp/ixp/<project-name>/{docs,text,taxonomies,prompts} ## Step 2 — Configure the Model -Before labelling, configure the extraction model based on what the documents look like. Download 2-3 sample document images and view them: +Before labelling, inspect 2–3 sample document images. Run: ```bash uip ixp documents list <project-name> --output json uip ixp documents download <project-name> <document-id> -o /tmp/ixp/<project-name>/docs/sample --output json ``` -View with the **Read tool** — one full Read per document, **no `pages` parameter** (returns text + image natively). Then decide: +View each sample with the **Read tool**, using one full Read per document and **no `pages` parameter**; it returns text and image natively. Select configuration as follows: | Document characteristics | Pre-processing | Model | -|--------------------------|---------------|-------| +|---|---|---| | Simple documents, no tables | `none` | `gemini_2_5_flash` | -| Documents with simple tables or multiple tables | `table_mini` | `gemini_2_5_flash` | -| Complex nested tables, merged cells, multi-page tables | `table` | `gemini_2_5_flash` | +| Simple tables or multiple tables | `table_mini` | `gemini_2_5_flash` | +| Complex nested tables, merged cells, or multi-page tables | `table` | `gemini_2_5_flash` | | Very long documents (100+ pages) | `none` or `table_mini` | `gemini_2_5_pro` | -Apply the configuration: +The default is `--model gemini_2_5_flash --preprocessing table_mini`. Apply the configuration by running: ```bash uip ixp projects configure-model <project-name> \ @@ -73,25 +67,25 @@ uip ixp projects configure-model <project-name> \ --output json ``` -**Default recommendation:** `--model gemini_2_5_flash --preprocessing table_mini` — works well for most invoice/document types. - ## Step 3 — Name the Project -Based on the taxonomy from Step 1 (e.g., if it has "Invoice Details", "Line Items", "Bill-To" → it's an invoices project), give the project a descriptive title: +Use the Step 1 taxonomy to choose a descriptive title. For example, "Invoice Details", "Line Items", and "Bill-To" indicate an invoices project. Run: ```bash uip ixp projects update-title <project-name> "Vendor Invoices" --output json ``` -Skip this step if the user already provided a meaningful name in Step 1. +Skip this step when the user provided a meaningful name in Step 1. ## Step 4 — Label All Documents -**Default:** follow the [Label Documents Guide](label-documents-guide.md) to label every document in the project. +**Default:** follow the [Label Documents Guide](label-documents-guide.md) and label every document. + +Labelling is optional. It produces the **project score** (`get-metrics` reports nothing until documents are confirmed) but is not required for a callable model; a trained version appears on its own within seconds of `projects create`. -Labelling is optional — it produces the **project score** (`get-metrics` reports nothing until documents are confirmed) and is not required for a callable model (a trained version appears on its own within seconds of `projects create`). Skip it only when: +Skip labelling only when: -- **The model unblocks a larger build in this session** — the deliverable is something else (a flow, an automation) waiting on a callable model, and no score, metrics, or accuracy target was named. Deploy per the [Deployment Guide](deployment-guide.md) and resume the build. Canonical case: the inbound `uipath-maestro-flow` handoff (see *When NOT to Use This Skill* in [SKILL.md](../SKILL.md)). -- **The user opts out** — says to skip labelling, or that somebody else will handle labelling. Stop after Step 3 and hand over the project name; labelling can happen later, in-product or via the [Label Documents Guide](label-documents-guide.md). +- **The model unblocks a larger build in this session:** the deliverable is something else, such as a flow or automation, waiting on a callable model; no score, metrics, or accuracy target was named. Deploy per the [Deployment Guide](deployment-guide.md) and resume the build. The canonical case is the inbound `uipath-maestro-flow` handoff (see *When NOT to Use This Skill* in [SKILL.md](../SKILL.md)). +- **The user opts out:** the user says to skip labelling or that somebody else will handle it. Stop after Step 3 and hand over the project name; labelling can occur later in-product or via the [Label Documents Guide](label-documents-guide.md). -Skipping is never silent: state that the model is unscored and that labelling is the fix if fields come back wrong. +Never skip silently: state that the model is unscored and that labelling is the fix if fields come back wrong. diff --git a/skills/uipath-ixp/references/unsupported-capabilities.md b/skills/uipath-ixp/references/unsupported-capabilities.md new file mode 100644 index 0000000000..8ff61f7fdf --- /dev/null +++ b/skills/uipath-ixp/references/unsupported-capabilities.md @@ -0,0 +1,14 @@ +# Unsupported Capabilities + +These requests fall outside the skill. Recognise the request, reply with the standard response, route the user. Do NOT enter discovery (`uip --help`, grep, source reading) — see Critical Rule #1. + +| User request | Standard response | +|--------------|-------------------| +| "Create a model" / "create a project" | **Documents or a taxonomy supplied →** use the [Project Setup Guide](project-setup-guide.md) (this skill creates the project from them). **Otherwise →** "I work on existing IXP projects rather than creating them from scratch. Create one in-product: https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/managing-projects — then I can label, review, and improve it." | +| "Upload these files" / "add documents" | **Project named / already in context →** supported; upload it (see the "Upload a document" row in Task Navigation). **Otherwise →** "Name an existing project and I'll upload it — or upload in-product (e.g. for a new project): https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/building-and-deploying-models." | +| "Push to an environment / another tenant" / "deploy to staging or production" | **Names an Orchestrator folder (a folder literally called `Production`) →** supported; use the "Deploy this model to a folder" row in [Task Navigation](../SKILL.md#task-navigation). **Otherwise →** "IXP has no environment or cross-tenant deploy target — a deployment is a (folder, version) pair inside one tenant." Note `projects publish --tag staging\|live` moves the tag the DU framework (and DU activities calling through it) resolve — for those consumers that IS the staging/live switch; it creates no folder deployment. | +| "Give X access" / "share this project" / "change roles or permissions" | "Access, roles, and permissions are managed in-product, not through this skill: https://docs.uipath.com/ixp/automation-cloud/latest/overview/managing-access." | +| "Use this model in my automation / workflow / agent" / "call the extractor from a process" | "Consuming a published model inside an automation is an authoring task outside this skill. See https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/building-and-consuming-a-workflow." | +| "Mine these emails / communications" / "set up Communications Mining" | "Communications Mining is a separate IXP capability this skill doesn't cover (this skill is document extraction). See https://docs.uipath.com/ixp/automation-cloud/latest/cm-user-guide/introduction-to-uipath-communication-mining." | +| "Monitor the deployed model" / "how many docs did it process?" / "runtime throughput or incidents" | "Runtime/operational monitoring of a deployed model lives in Orchestrator, not this skill: https://docs.uipath.com/orchestrator/automation-cloud/latest/user-guide/about-monitoring. For design-time scores use `get-metrics` (see 'Show metrics')." | +| "Edit a data type's values" / "add or remove a Choice option" | "The CLI can rename a data type, change its instructions (`data-types update-instructions`), or delete it — but it can't add or remove the **values** of an existing Choice data type. Do that by hand in-product on the Manage Taxonomy page: https://docs.uipath.com/ixp/automation-cloud/latest/user-guide/managing-projects — then continue here." |