Repository navigation
Release v1.1.0: fix payer eligibility and published tool paths #53
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
58316ce
a75ddb6
7ab81de
7f9dfe5
3d0095a
72e1fa9
6b4cbaf
f562ec3
0e9130d
d181168
4c812c8
f6efd75
3f94258
926e2fa
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,56 @@ | ||
| # Changelog | ||
|
|
||
| ## 1.1.0 — 2026-10-06 | ||
|
|
||
| ### Fixes | ||
|
|
||
| - Restore the published tool-reference path in the Docker image. The directory | ||
| alias `/opt/healthverse-task-assets` points to `/opt/chi-bench-task-assets`, so | ||
| existing task instructions see the selected per-task tool reference. | ||
| - Allow a completed payer intake with failed member eligibility to end in the | ||
| existing `denied` state with reason `ELIG_TERMED`. The agent generates and | ||
| delivers a denial letter. Approval and clinical routing are blocked after a | ||
| failed eligibility check. | ||
| - Add coverage status and known coverage dates to the existing request-form | ||
| template. The data repair restores pa_t035's termination date from the original | ||
| synthesis facts and replaces its unreachable `requirement_checked` expectation | ||
| with denial and a denial letter. No separate evidence document is added. | ||
| - Provide scripts to repair downloaded pa_t035 task copies and regenerate the | ||
| 25 saved payer request forms and their marathon copies before rebuilding. | ||
| Remove rendering timestamps from regenerated PDFs so repeat runs preserve | ||
| their hashes when content is unchanged. | ||
| - Align the local competition Rules and Overview references with the published | ||
| Kaggle pages. | ||
|
|
||
| ### Upgrade | ||
|
|
||
| After obtaining handbook access and downloading the task data: | ||
|
|
||
| ```bash | ||
| uv sync --extra dev | ||
| uv run playwright install chromium | ||
| uv run python scripts/repair_pa_t035.py --data-dir data | ||
| uv run python scripts/regenerate_payer_request_forms.py --data-dir data | ||
| uv run cb data verify | ||
| uv run cb docker build | ||
| ``` | ||
|
|
||
| On Linux, use `uv run playwright install --with-deps chromium` if browser system | ||
| libraries are missing. | ||
|
|
||
| This is runtime version **1.1.0**. The downloaded dataset remains pinned to | ||
| **chi-bench-v1.0.0**; the scripts apply local corrections. This change does not | ||
| publish a new Hugging Face dataset revision. Preserve earlier trial results and | ||
| record the runtime commit and image digest for new runs. | ||
|
|
||
| ### P2P clarification | ||
|
|
||
| The P2P workflow is unchanged. The original `MdCaseDecision` records the referral | ||
| to P2P; the completed P2P request supplies the outcome, and | ||
| `determination.finalize` records the final determination. No second physician | ||
| submission is required. | ||
|
|
||
| Among the six public payer tasks requiring P2P, t026 and t032 expect the referral | ||
| to have `signed_off=false`. Tasks t016, t019, t031, and t036 exclude MD review | ||
| from their scored stages. A direct-decision task may require signed physician | ||
| review even if an agent chooses P2P instead. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -99,6 +99,9 @@ COPY data/marathon /opt/chi-bench/tasks/marathon | |
| COPY data/skills/managed-care-operations-handbook /workspace/skills/managed-care-operations-handbook | ||
| RUN mkdir -p /opt/chi-bench-task-assets | ||
| COPY data/care_management/shared/tool_reference.md /opt/chi-bench-task-assets/tool_reference.md | ||
| # Published task instructions still use the healthverse asset path. Alias the | ||
| # directory so per-task selection and tool-name rewriting remain visible there. | ||
| RUN ln -s /opt/chi-bench-task-assets /opt/healthverse-task-assets | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
This only fixes images built from Useful? React with 👍 / 👎. |
||
|
|
||
| COPY docker/entrypoint.sh /usr/local/bin/entrypoint.sh | ||
| RUN chmod +x /usr/local/bin/entrypoint.sh | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These commands repair only a user's local checkout. The published-task image built by
docker/Dockerfile.harborstill downloads the unchangedchi-bench-v1.0.0dataset and copies it directly without running either script, so a fresh Harbor export retains pa_t035's missing termination evidence and unreachablerequirement_checkedexpectation. Invoke the repair during that build or publish and pin a repaired dataset revision before releasing this behavior.Useful? React with 👍 / 👎.