Skip to content

docs: add licensing note, container run, response samples, CI checks #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
saugataroyarghya merged 1 commit into from
Oct 11, 2026
Merged

docs: add licensing note, container run, response samples, CI checks #1

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
127 changes: 120 additions & 7 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,13 @@ An HTTP API that estimates IELTS rubric scores from Writing Task 2 essays and
spoken responses. Built with BentoML and PyTorch, it exposes trained scoring
models through authenticated endpoints.

> **Licensing:** The writing model's training data includes ELLIPSE
> (CC BY-NC-SA 4.0), and commercial-use rights for its weights have not been
> cleared. The speaking scorer carries its own base-model and training-data terms.
> This repository's code has no license yet, so no reuse rights are granted.
> Read the [third-party notices](THIRD_PARTY_NOTICES.md) before deploying beyond
> personal evaluation.

## What it solves

A trained model still needs input validation, preprocessing, request handling
Expand Down Expand Up @@ -40,8 +47,9 @@ export IELTS_API_KEY="$(uv run python -c 'import secrets; print(secrets.token_ur
uv run bentoml serve service:IELTSService --host 127.0.0.1
```

The first run downloads model weights and warms the models during startup.
`IELTS_DEVICE` selects `auto` (CUDA → MPS → CPU), `cuda`, `mps` or `cpu`.
`IELTS_API_KEY` must be at least 32 characters; the server refuses to start
without it. The first run downloads model weights and warms the models during
startup. `IELTS_DEVICE` selects `auto` (CUDA → MPS → CPU), `cuda`, `mps` or `cpu`.
Docker on macOS cannot use MPS; Apple GPU inference requires native serving.

**Writing request**
Expand All @@ -67,6 +75,69 @@ repetitions and disfluencies in transcripts. For automatic transcription, run
`uv run python -m scripts.fetch_models --asr`, start with `IELTS_ENABLE_ASR=1`,
and omit `transcript` where needed.

**Writing response**

```json
{
"scores": {"TR": 6.2318, "CC": 6.0874, "LR": 6.3351, "GRA": 5.9462},
"task2_mean_estimate": 6.150125,
"track": "ielts",
"max_tokens": 1024,
"truncated": false,
"release": {
"application_version": "0.1.0",
"model_id": "saugataroy/ielts-task2-modernbert-base-moe",
"model_revision": "ad150d954cc9af7fa848fedeafeef6c28c71e2c6",
"device": "cpu",
"precision": "float32"
}
}
```

`TR` is Task Response, `CC` Coherence and Cohesion, `LR` Lexical Resource and
`GRA` Grammatical Range and Accuracy. Scores are raw model estimates, neither
clipped nor rounded. `truncated` is `true` when the question and essay together
exceed `max_tokens`, so the end of the essay was cut.

**Speaking response**

```json
{
"scores": {
"fluency_and_coherence": 6.5,
"lexical_resource": 6.5,
"grammatical_range_and_accuracy": 6.0,
"pronunciation": 6.5
},
"speaking_band": 6.5,
"raw_scores": {
"fluency_and_coherence": 6.43,
"lexical_resource": 6.61,
"grammatical_range_and_accuracy": 6.07,
"pronunciation": 6.38
},
"score_status": "experimental_weakly_supervised",
"text_repair_validation_gate_passed": true,
"human_criterion_accuracy_verified": false,
"device": "cpu",
"assessment_scope": "provided_responses",
"request_id": "9f2c41d7a0b34e5c8d1e6f7a2b3c4d5e",
"model_id": "saugataroy/ielts-speaking-scorer",
"model_revision": "2c9e4cae199ce353299145231638085a83afa5b9",
"audio_seconds": 7.4,
"transcript_sources": ["caller_supplied"],
"original_session_timeline_available": false,
"disfluency_preservation_verified": false
}
```

`raw_scores` are the model's unrounded estimates. `scores` clip those to 1–9 and
round each to the nearest half band, and `speaking_band` is the mean of `scores`
rounded to the nearest half band. `transcript_sources` has one entry per
response: `caller_supplied` or `faster_whisper_unedited`. The status and
`*_verified` fields repeat the experimental status described above in every
response. All values in both samples are illustrative.

The authenticated API schema is at `/docs.json`. `/livez` and `/readyz` are
unauthenticated health endpoints.

Expand All @@ -93,10 +164,26 @@ measured capacity claim.

## Development and verification

CI runs these source checks on every push and pull request:

```bash
uv run pytest -q # Tests without loading model weights
uv sync --locked
uv run pytest -q # Tests without loading model weights
uv run ruff check .
uv run python -m scripts.verify_http # Against a running API
uv run ruff format --check .
uv run python -m scripts.export_requirements --check # requirements.txt matches uv.lock
```

`requirements.txt` is generated from `uv.lock` and is what the Bento image
installs. After changing dependencies, run `uv lock`, then
`uv run python -m scripts.export_requirements`, and commit `pyproject.toml`,
`uv.lock` and `requirements.txt` together.
`scripts.build_bento` refuses to build while `requirements.txt` is out of date.

Against a running API:

```bash
uv run python -m scripts.verify_http # Writing regression
uv run python -m scripts.benchmark --task writing --requests 20 --concurrency 4
```

Expand All @@ -117,16 +204,42 @@ uv run python -m scripts.benchmark --task speaking \
Reports include successful requests per second, latency percentiles and response
status counts.

Build a versioned Bento from a clean commit with
`uv run python -m scripts.build_bento`, then containerize its tag with
`uv run bentoml containerize <tag>`.
## Build and run a container

Build a versioned Bento from a clean commit, containerize its tag, and run it.
`scripts.build_bento` prints the tag, which is `ielts-inference-api:` followed by
the first 12 characters of the commit hash.

```bash
uv run python -m scripts.build_bento
uv run bentoml containerize ielts-inference-api:<commit>
docker run --rm -p 127.0.0.1:3000:3000 \
-e IELTS_API_KEY="$IELTS_API_KEY" \
ielts-inference-api:<commit>
```

The container serves the same endpoints on port 3000 and requires the same
`IELTS_API_KEY`, which it refuses to start without. Add
`-e IELTS_ENABLE_ASR=1` for automatic transcription.

- `bentoml containerize` embeds the pinned model weights in the image, fetching
them first if the build host has not cached them. The image sets
`HF_HUB_OFFLINE=1`, so a running container needs no network access and no
mounted cache.
- The Linux image includes PyTorch's CUDA libraries. It runs on CPU unless the
container is given an NVIDIA GPU (`docker run --gpus all` with the NVIDIA
Container Toolkit installed). Set `IELTS_DEVICE=cpu` to force CPU. Docker on
macOS always runs on CPU. CUDA execution in the container has not been verified.

## Code layout

```text
service.py API endpoints, worker configuration and batching
ielts_api/ Model inference, audio preparation, schemas and middleware
models.lock.json Model revisions and integrity checks
pyproject.toml Dependencies; uv.lock pins the resolved environment
requirements.txt Generated from uv.lock; installed by the Bento image
release.json Application version reported in responses
scripts/ Model download, build, parity checks and load testing
tests/ API contracts, preprocessing and regression fixtures
```
Expand Down
Loading