Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
0c1fb4b
Acquire verified DINOv2 models on first use
OliverZhaohaibin Aug 22, 2026
1147ec3
Fix Pets model acquisition fallback and publication
OliverZhaohaibin Aug 22, 2026
39750c5
Fix DINOv2 torch hub test stub
OliverZhaohaibin Aug 22, 2026
423fa8e
Keep network permission errors out of storage fallback
OliverZhaohaibin Aug 22, 2026
0becd6f
Test network permission errors do not trigger model fallback
OliverZhaohaibin Aug 22, 2026
ec668f9
Close model temp files before network cleanup
OliverZhaohaibin Aug 22, 2026
7ef88fa
docs: clarify Pets model publish and sidecar contract
OliverZhaohaibin Aug 22, 2026
359b25d
fix: stabilize pinned DINOv2 first-use acquisition
OliverZhaohaibin Aug 22, 2026
1590fdb
fix: load verified DINOv2 checkpoint directly
OliverZhaohaibin Aug 22, 2026
3e5dae9
fix: trigger pet backfill after startup scan
OliverZhaohaibin Aug 22, 2026
992b4ea
test: cover startup pet backfill scheduling
OliverZhaohaibin Aug 22, 2026
c1eb8e0
fix: harden DINOv2 first-use acquisition
OliverZhaohaibin Aug 22, 2026
0580928
refactor: remove pets pipeline module substitution
OliverZhaohaibin Aug 22, 2026
744ea8a
refactor: separate verified cache build from device activation
OliverZhaohaibin Aug 22, 2026
172f3b0
test: target explicit pets hardening boundaries
OliverZhaohaibin Aug 22, 2026
0bd5fef
test: preserve pets facade monkeypatch compatibility
OliverZhaohaibin Aug 22, 2026
becfd9a
test: support pytest 9 monkeypatch compatibility
OliverZhaohaibin Aug 22, 2026
d94bcbb
test: remove pets facade monkeypatch compatibility
OliverZhaohaibin Aug 22, 2026
0749dbf
Serialize DINOv2 first-use cache builds
OliverZhaohaibin Aug 22, 2026
388ba38
Keep DINO cache cleanup inside acquisition lock
OliverZhaohaibin Aug 22, 2026
b087b72
Test concurrent DINOv2 cache acquisition
OliverZhaohaibin Aug 22, 2026
1aea0aa
Avoid destructive DINOv2 cache cleanup from resolver
OliverZhaohaibin Aug 23, 2026
42da9f0
Test DINOv2 resolver during cache publication
OliverZhaohaibin Aug 23, 2026
072a7a8
Use fixed DINOv2 Release artifacts
OliverZhaohaibin Aug 25, 2026
cb4e808
Relax deterministic trace tolerance
OliverZhaohaibin Aug 25, 2026
41b8257
Install NumPy in DINO validation jobs
OliverZhaohaibin Aug 25, 2026
f5120aa
Pin verified Pets model artifact
OliverZhaohaibin Aug 25, 2026
8805b10
Publish only validated model artifacts
OliverZhaohaibin Aug 25, 2026
d1fdcc8
Record fixed Pets model release evidence
OliverZhaohaibin Aug 25, 2026
f90bf74
Exercise production model download contract
OliverZhaohaibin Aug 25, 2026
5091a3a
Install published model contract imports
OliverZhaohaibin Aug 25, 2026
9ba3b6b
Serialize Face runtime import before QThread start
OliverZhaohaibin Aug 25, 2026
ceb24fa
Close Pets acquisition and release provenance gaps
OliverZhaohaibin Aug 25, 2026
d21a0bc
Pin repaired v1 artifact and benchmark policy
OliverZhaohaibin Aug 25, 2026
32b5e1d
Add guarded one-time v1 repair job
OliverZhaohaibin Aug 25, 2026
e91920f
Freeze repaired v1 and harden recognition benchmarks
OliverZhaohaibin Aug 25, 2026
08864a8
Record PR 917 recognition performance evidence
OliverZhaohaibin Aug 25, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 98 additions & 0 deletions .github/workflows/pets-dino-release-publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,98 @@
name: Pets DINO Release publish

on:
workflow_dispatch:
inputs:
artifact_run_id:
description: Successful Pets DINO candidate workflow run ID
required: true
type: string

permissions:
contents: read

env:
CANDIDATE_WORKFLOW_ID: "340207631"
CANDIDATE_WORKFLOW_PATH: ".github/workflows/pets-dino-source-contract.yml"

jobs:
publish-release:
runs-on: ubuntu-latest
environment: pets-model-release
permissions:
actions: read
contents: write
steps:
- name: Require an explicit artifact run
run: |
if [ -z "${{ inputs.artifact_run_id }}" ]; then
echo "artifact_run_id is required" >&2
exit 1
fi

- name: Resolve candidate run provenance
id: candidate
env:
GH_TOKEN: ${{ github.token }}
run: |
gh api "repos/${GITHUB_REPOSITORY}/actions/runs/${{ inputs.artifact_run_id }}" > candidate-run.json
head_sha=$(python -c 'import json; print(json.load(open("candidate-run.json"))["head_sha"])')
echo "head_sha=${head_sha}" >> "${GITHUB_OUTPUT}"

- name: Check out the exact builder commit
uses: actions/checkout@v4
with:
ref: ${{ steps.candidate.outputs.head_sha }}
path: builder

- name: Check out the release manifest commit
uses: actions/checkout@v4
with:
ref: ${{ github.sha }}
path: release

- uses: actions/download-artifact@v4
with:
name: pet-models-v1-candidate
path: model-dist
run-id: ${{ inputs.artifact_run_id }}
github-token: ${{ github.token }}

- name: Verify artifact, manifest, and builder provenance
run: |
builder_commit=$(git -C builder rev-parse HEAD)
python release/tools/model_release_provenance.py \
--run-json candidate-run.json \
--build-manifest model-dist/dinov2_vits14.build.json \
--builder-commit "${builder_commit}" \
--expected-repository "${GITHUB_REPOSITORY}" \
--expected-workflow-id "${CANDIDATE_WORKFLOW_ID}" \
--expected-workflow-path "${CANDIDATE_WORKFLOW_PATH}"
python release/tools/validate_dinov2_torchscript.py \
model-dist/dinov2_vits14.pt \
model-dist/dinov2_vits14.pt.metadata.json \
--manifest release/src/iPhoto/pets/model_manifest.json \
--metadata-only

- name: Refuse overwrite of an immutable v1
env:
GH_TOKEN: ${{ github.token }}
run: |
if gh release view pet-models-v1 --repo "${GITHUB_REPOSITORY}" >/dev/null 2>&1; then
echo "pet-models-v1 already exists; refusing to overwrite it" >&2
exit 1
fi

- name: Create v1 when no Release exists
env:
GH_TOKEN: ${{ github.token }}
run: >-
gh release create pet-models-v1
model-dist/dinov2_vits14.pt
model-dist/dinov2_vits14.pt.metadata.json
model-dist/dinov2_vits14.build.json
model-dist/SHA256SUMS
--repo "${GITHUB_REPOSITORY}"
--target "${{ steps.candidate.outputs.head_sha }}"
--title "iPhotron Pets models v1"
--notes "Pinned DINOv2 ViT-S/14 TorchScript artifact validated on Ubuntu, macOS, and Windows."
120 changes: 120 additions & 0 deletions .github/workflows/pets-dino-source-contract.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
name: Pets DINO Release contract

on:
workflow_dispatch:
pull_request:
branches: [main, edit-base]

permissions:
contents: read

env:
PYTHON_VERSION: "3.12"
TORCH_VERSION: "2.12.1"
TORCHVISION_VERSION: "0.27.1"
DINO_REVISION: "7764ea0f912e53c92e82eb78a2a1631e92725fc8"
DINO_WEIGHTS_URL: "https://dl.fbaipublicfiles.com/dinov2/dinov2_vits14/dinov2_vits14_pretrain.pth"

jobs:
build-release-candidate:
runs-on: ubuntu-latest
timeout-minutes: 30
steps:
- uses: actions/checkout@v4
with:
ref: ${{ github.event.pull_request.head.sha || github.sha }}
- uses: actions/checkout@v4
with:
repository: facebookresearch/dinov2
ref: ${{ env.DINO_REVISION }}
path: .model-build/dinov2
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
cache: pip
cache-dependency-path: pyproject.toml
- name: Install pinned CPU build runtime
run: >-
python -m pip install torch==${TORCH_VERSION}
torchvision==${TORCHVISION_VERSION}
--index-url https://download.pytorch.org/whl/cpu
- name: Download pinned official checkpoint
run: >-
curl --fail --location --retry 3 "${DINO_WEIGHTS_URL}"
--output .model-build/dinov2_vits14_pretrain.pth
- name: Build one verified TorchScript candidate
env:
XFORMERS_DISABLED: "1"
IPHOTO_MODEL_BUILD_COMMIT: ${{ github.event.pull_request.head.sha || github.sha }}
run: >-
python tools/convert_dinov2_torchscript.py
model-dist/dinov2_vits14.pt
--source-dir .model-build/dinov2
--checkpoint .model-build/dinov2_vits14_pretrain.pth
--runtime-metadata model-dist/dinov2_vits14.pt.metadata.json
--build-manifest model-dist/dinov2_vits14.build.json
- name: Record artifact checksum
run: sha256sum model-dist/dinov2_vits14.pt > model-dist/SHA256SUMS
- uses: actions/upload-artifact@v4
with:
name: pet-models-v1-candidate
path: model-dist/*
if-no-files-found: error
retention-days: 14

validate-release-candidate:
needs: build-release-candidate
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, macos-latest, windows-latest]
runs-on: ${{ matrix.os }}
timeout-minutes: 20
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install pinned CPU runtime on Linux and Windows
if: runner.os != 'macOS'
run: |
python -m pip install numpy
python -m pip install torch==${{ env.TORCH_VERSION }} --index-url https://download.pytorch.org/whl/cpu
- name: Install pinned runtime on macOS
if: runner.os == 'macOS'
run: python -m pip install numpy torch==${{ env.TORCH_VERSION }}
- uses: actions/download-artifact@v4
with:
name: pet-models-v1-candidate
path: model-dist
- name: Validate identical release bytes and inference
shell: bash
run: >-
python tools/validate_dinov2_torchscript.py
model-dist/dinov2_vits14.pt
model-dist/dinov2_vits14.pt.metadata.json
| tee "model-dist/validation-${RUNNER_OS}.json"
- uses: actions/upload-artifact@v4
with:
name: pet-models-v1-validation-${{ runner.os }}
path: model-dist/validation-*.json
if-no-files-found: error

published-release-contract:
if: github.event_name == 'pull_request'
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with:
python-version: ${{ env.PYTHON_VERSION }}
- name: Install pinned published-artifact runtime
run: |
python -m pip install certifi numpy pillow pytest
python -m pip install torch==${TORCH_VERSION} --index-url https://download.pytorch.org/whl/cpu
- name: Validate the immutable public Release asset
env:
PYTHONPATH: src
IPHOTO_RUN_PETS_DINO_RELEASE_CONTRACT: "1"
run: python -m pytest -q -s tests/contracts/test_pets_dino_source_contract.py
14 changes: 8 additions & 6 deletions docs/development.md
Original file line number Diff line number Diff line change
Expand Up @@ -235,12 +235,14 @@ YOLOX release asset and can be overridden with:
export IPHOTO_PET_DETECTOR_MODEL_URL="https://example.invalid/yolox_nano.onnx"
```

Production never executes Torch Hub. It loads a packaged DINOv2 TorchScript model,
or downloads the fixed HTTPS artifact declared by SHA-256 and exact byte size in
`src/iPhoto/pets/model_manifest.json`. Release engineering may regenerate the
artifact from the pinned source revision with
`tools/convert_dinov2_torchscript.py`; that tool also checks eager/TorchScript
numeric equivalence before publishing.
When DINOv2 is missing, iPhotron downloads the fixed `pet-models-v1` TorchScript
Release declared by HTTPS URL, SHA-256, exact byte size, cache schema, and
producer version in `src/iPhoto/pets/model_manifest.json`. It validates CPU
loading and output shape before publishing the model and metadata atomically.
Production never downloads or executes DINOv2 source and never traces a model on
the user's machine. `tools/convert_dinov2_torchscript.py` is release-only: it
loads a local checkout of the pinned source commit and verified official
checkpoint, then checks eager/TorchScript numeric equivalence.

For offline or packaged validation, disable first-use downloads with:

Expand Down
6 changes: 3 additions & 3 deletions docs/misc/BUILD_DEB.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ runtime depends on the helper binary plus the shared libraries under

Builds that ship People/Pets recognition must also preserve the selected AI
runtimes from the standalone bundle. People needs `insightface` and
`onnxruntime`; Pets needs `onnxruntime`, `torch`, `torchvision`, `usearch`, and
`onnxruntime`; Pets needs `onnxruntime`, `torch==2.12.1`, `usearch`, and
`certifi`. Offline builds also retain the shared `extension/models` cache.
These are added at the Nuitka stage described in
[`BUILD_EXE.md`](BUILD_EXE.md); the `.deb` stage must not strip them from
Expand Down Expand Up @@ -126,7 +126,7 @@ Description: Folder-native local photo album manager
```bash
find "$APP_ROOT" -path '*insightface*' -o -path '*onnxruntime*'
find "$APP_ROOT/extension/models" -name 'det_500m.onnx' -o -name 'w600k_mbf.onnx'
find "$APP_ROOT" -path '*torch*' -o -path '*torchvision*' -o -path '*usearch*'
find "$APP_ROOT" -path '*torch*' -o -path '*usearch*'
find "$APP_ROOT/extension/models/pets" -name 'yolox_nano_coco.onnx' -o -name 'dinov2_vits14.pt'
```

Expand Down Expand Up @@ -194,4 +194,4 @@ sudo apt remove iPhotron
| Native maps fail with GLX/XCB startup errors | The runtime was installed correctly, but the desktop session lacks XWayland/XCB GL integration | Install/enable XWayland and rerun, or set `IPHOTO_PREFER_OSMAND_NATIVE_WIDGET=0` to force the helper-backed Python OBF path |
| People scan is unavailable in the installed app | The standalone build was produced without the optional face runtime | Rebuild the standalone app with `insightface`, `onnxruntime`, and `src/extension/models` included before staging the `.deb` |
| People scan starts but never creates clusters | The model cache or an InsightFace submodel/dependency is missing from `/opt/iPhotron/` | Verify `extension/models`, exclude unused `albumentations`/`pydantic` packages at the Nuitka stage, and keep InsightFace limited to detection and recognition |
| Pets scan is unavailable in the installed app | The standalone build omitted `pets-ai` packages or `extension/models/pets` | Rebuild the standalone app with `onnxruntime`, `torch`, `torchvision`, `usearch`, `certifi`, and both Pets model files before staging the `.deb` |
| Pets scan is unavailable in the installed app | The standalone build omitted `pets-ai` packages or `extension/models/pets` | Rebuild the standalone app with `onnxruntime`, `torch==2.12.1`, `usearch`, `certifi`, and both Pets model files before staging the `.deb` |
8 changes: 4 additions & 4 deletions docs/misc/BUILD_EXE.md
Original file line number Diff line number Diff line change
Expand Up @@ -176,8 +176,9 @@ not enable it. Build environments that promise Pets support must install:
python -m pip install -e ".[pets-ai]"
```

The standalone bundle must retain `onnxruntime`, `torch`, `torchvision`,
`usearch`, and `certifi`. An offline-ready build must also include:
The standalone bundle must retain `onnxruntime`, `torch==2.12.1`, `usearch`, and
`certifi`. `torchvision` is a model-build dependency, not a production runtime
dependency. An offline-ready build must also include:

```text
extension/models/pets/
Expand All @@ -192,14 +193,13 @@ that enable Pets should also include the optional runtime explicitly:
```bash
--include-package=onnxruntime
--include-package=torch
--include-package=torchvision
--include-package=usearch
--include-package=certifi
--include-data-dir=src/extension/models=extension/models
```

The current platform build scripts explicitly include the People runtime but do
not yet add `torch`, `torchvision`, or `usearch` flags. Therefore a stock script
not yet add `torch` or `usearch` flags. Therefore a stock script
build must not be advertised as Pets-enabled merely because the model directory
was copied; add the flags above (or update the script) and perform the Pets smoke
test before release.
Expand Down
61 changes: 37 additions & 24 deletions docs/misc/PETS_RECOGNITION_RUNTIME.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,29 +38,44 @@ Install the optional runtime with:
pip install -e ".[pets-ai]"
```

The extra provides `onnxruntime`, `torch`, `torchvision`, `usearch`, and
`certifi`. Bundled models are read-only fallbacks; downloads are written to the
The extra provides `onnxruntime`, `torch==2.12.1`, `usearch`, and `certifi`.
`torchvision==0.27.1` is confined to the controlled model build workflow.
Bundled models are read-only fallbacks; downloads are written to the
platform user cache:

```text
src/extension/models/pets/
├── detector/yolox_nano_coco.onnx
└── embedding/dinov2_vits14/dinov2_vits14.pt
└── embedding/dinov2_vits14/
├── dinov2_vits14.pt
└── dinov2_vits14.pt.metadata.json
```

`IPHOTO_PET_MODEL_DIR` overrides that root. Missing models may be populated on
first use unless `IPHOTO_PET_MODEL_AUTO_DOWNLOAD=0`. The detector URL defaults
to the upstream YOLOX release and can be overridden with
`IPHOTO_PET_DETECTOR_MODEL_URL`. Production does not execute Torch Hub. DINOv2
must be supplied as the hash- and size-verified TorchScript artifact declared
in `iPhoto/pets/model_manifest.json`; Torch Hub is restricted to the release
conversion tool. `IPHOTO_PET_SCAN_DISABLED=1` disables the worker without
disabling the rest of the application.
Lookup uses the bundled extension first and then the platform user cache. A
writable development extension directory is preferred for installation; signed
macOS app bundles install directly into the user cache without a writability
probe. `IPHOTO_PET_MODEL_DIR` is authoritative: when set, both lookup and
installation use only that root.

Missing models may be populated lazily on the first non-empty scan unless
`IPHOTO_PET_MODEL_AUTO_DOWNLOAD=0`. The detector URL defaults to the upstream
YOLOX release and can be overridden with `IPHOTO_PET_DETECTOR_MODEL_URL`.
DINOv2 acquisition downloads only the fixed `pet-models-v1` TorchScript Release,
validates its SHA-256, exact size, producer/cache schema, CPU load, and output
shape, then publishes metadata first and the model as the visibility point.
Production never invokes Torch Hub, downloads source, imports xFormers, or traces
TorchScript. Legacy derived caches are replaced under the acquisition lock;
bundled artifacts are never deleted. Only local `EACCES`, `EPERM`, or `EROFS`
storage failures fall back to the user cache; network, TLS, disk-full, I/O, hash,
or compatibility failures do not masquerade as storage fallback.
`IPHOTO_PET_SCAN_DISABLED=1` disables the worker without disabling the rest of
the application.

Packaged/offline builds that promise Pets support must include the Python AI
runtime and the two model files under `extension/models/pets`. A build that
omits them must preserve graceful degradation: core browsing, People, Maps,
editing, and library state remain usable.
runtime, the YOLOX detector, the DINOv2 TorchScript artifact, and its
`dinov2_vits14.pt.metadata.json` sidecar under `extension/models/pets`. A build
that omits them must preserve graceful degradation: core browsing, People,
Maps, editing, and library state remain usable.

## Detection And Clustering Contract

Expand Down Expand Up @@ -125,16 +140,14 @@ starts before the previous drain finishes.
| `done` | Detection completed, including valid images with no pets. |
| `skipped` | Video, non-primary Live Photo component, or another ineligible asset. |

Interactive scans start Face and Pet workers alongside metadata scanning and
enqueue rows only after their asset batches commit. When a saved library needs
a startup metadata scan, startup first warms the gallery, runs that scan, then
starts both AI workers with closed input so they drain persisted
`pending`/`retry` rows. This avoids model initialization and competing AI work
on the first-frame path. If the metadata scan scope is already complete, startup
still starts the Pet backfill worker whenever persisted `pending` or `retry` rows
need draining. With no metadata scan and no queued AI work, startup does not
launch scan workers; an explicit rescan is only needed to reset or rediscover
otherwise completed/failed assets.
Interactive rescans start Face and Pet workers alongside metadata scanning and
enqueue rows only after their asset batches commit. Desktop startup keeps the
first-frame and metadata-scan paths AI-free. After the startup scan succeeds—or
immediately when its scope was already complete—a 1500 ms interaction-idle gate
starts both closed-input workers at `LowestPriority`. Click, wheel, key, drag,
touch, and gesture input restart the gate. Missing People/Pets models may then
download automatically; switching libraries, shutdown, cancellation, or a new
generation cancels pending activation.

The Pet worker uses small batches and queue top-up from the asset repository.
Missing dependencies/models are runtime-availability failures: pending rows are
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,8 +19,9 @@
因此之前记录的 Pets 自动化完成结论已经撤回。对 Pets 模型契约、大图库
复杂度、跨库一致性、升级回填、打包模型目录和身份语义的复审已完成本机自动化
核心修复。完整问题、裁决和验收要求见 `PETS_REVIEW_REMEDIATION_LEDGER.md`。项目固定
`pet-models-v1` TorchScript Release 尚未发布;远端 CI 已在
`e0001ee646e95fadc33659ebe277eb067e79a084` 上 9/9 成功。只读 packaged 安装、
[`pet-models-v1`](https://github.com/OliverZhaohaibin/iPhotron-LocalPhotoAlbumManager/releases/tag/pet-models-v1)
TorchScript Release 已发布,并由 [run 32807111211](https://github.com/OliverZhaohaibin/iPhotron-LocalPhotoAlbumManager/actions/runs/32807111211)
在 Ubuntu、macOS、Windows 验证同一资产。只读 packaged 安装、
真实升级库、网络失败和跨平台 50k 报告仍为
`manual_validation_pending`,因此不得恢复 `engineering_complete`。

Expand All @@ -45,7 +46,7 @@ Windows 实机复测表明前次 WIC fresh-decode 修复会被已持久化的错
- `DesktopCoordinatorRuntime` 是唯一桌面组合根,Recognition、Location/Info、Edit 与地图能力均延迟到首次使用;People dashboard 快照只在 People feature 首次创建时预热。
- settings/shell 同步初始化和 Windows/Linux pre-show Detail 异常也进入唯一 terminal 协议;pre-show Detail 使用可重试降级窗口。
- 模块预载使用 generation-aware owner 和完成信号,不再依赖持续轮询 timer;退出时等待预载线程收口。
- People/Pets 的模型扫描不再在 startup completed 后自动启动;首次进入识别功能才构造服务与 worker,消除快速关窗的 QThread 竞争
- People/Pets 在主 metadata scan 成功后通过 1500 ms 交互空闲门控自动启动;worker 使用最低优先级,切库、取消和 shutdown generation 阻止迟到启动

### Probe、数据库与慢存储

Expand Down
Loading
Loading