Skip to content

SOURCE.md

Latest commit

 

History

History
156 lines (132 loc) · 8.82 KB

SOURCE.md

File metadata and controls

156 lines (132 loc) · 8.82 KB
Raw

Source material

Demo clip (public/sample/storyteller.mp4)

Wikimedia Commons, under Creative Commons Attribution 3.0 Unported.

CC BY 3.0 requires credit, a link to the licence, and an indication that changes were made. All three are given above and in THIRD_PARTY_NOTICES.md. It carries no ShareAlike condition, so nothing in this repository inherits a copyleft obligation from it.

What it is. 67.4 seconds, 608x1080, 59.92 fps, H.264 High profile, AAC stereo, 11.8 MB. A single static, indoor, close-up shot of one speaker talking to camera against a plain wall, steady lighting throughout, no camera movement and no cuts. A small "clideo.com" watermark is burned into the bottom-right corner by the tool used to crop it. It was left in place rather than removed, because this project captions video and does not edit it, and because painting it out would be an undisclosed alteration of a CC BY source.

Aspect ratio. 608x1080 is 0.5630 versus an exact 9:16 of 0.5625, a difference of 0.00046. Comfortably inside the app's own 0.02 tolerance (ASPECT_TOLERANCE in src/upload/limits.ts), so the clip is captioned as ordinary portrait video with no special case; see the aspect-ratio note below.

Why this source, and what makes captioning it non-trivial. The choice was not incidental. Wikitongues records speakers of specific languages and dialects, so a Wikitongues clip is deliberately the opposite of the clean studio English the brief warns about.

What makes captioning it non-trivial, honestly. The camera work is easy: one locked-off shot, one face, consistent light, nothing for a caption template to fight. What is genuinely hard is the speech itself. The speaker narrates a short passage of Celtic legend and then, without a pause or a scene change, pivots into discussing Doric/Scots dialect words from north-east Scotland ("fit" for "what", and a phrase Whisper heard as "Farigarn" that is almost certainly "far ye gaan", i.e. "where are you going") and pronounces one himself as an example. That is a genuine ASR stress case: proper nouns and dialect vocabulary a general-purpose speech model has not seen, mixed into otherwise ordinary English in the same breath. It is not a hard clip by the usual visual measures (movement, low light, multiple speakers), and it would be inaccurate to claim otherwise; the difficulty here is entirely in the audio content.

Transcript. public/sample/storyteller-words.json was produced by running the app's real transcription path (transcribeFile / WhisperTranscriber, model base) against this file in a live browser session, the same pipeline a user's upload goes through, then committed as the fixture so the landing page loads captioned instantly with no transcription wait. 140 words, splitting to 24 segments and 47 lines at the default three-words-a-line templates (docs/proof/motion.md), one word timing nudged back into order, one short untranscribed stretch of trailing silence after the last word. The committed timings also carry the 110 ms lead correction derived in docs/proof/word-timing.md and applied in src/captions/asrBias.ts, because Whisper's word starts ran measurably late against the real audio. That correction is in the transcription path, so it covers real uploads too, not just this fixture. Read back against the video, it is correct and coherent: for example "He said, not fast enough. Then he said to culture, the great hero of the Celtic band of brotherhood..." and later "...these are two modes of speech ... the northeast is much affected by the connection they have to the lands of Norway and Sweden, and for instance the word what would be fit." Two words are clearly wrong rather than merely imperfect: "culture" is almost certainly a mishearing of a Gaelic hero's name (the surrounding sentence names "the great hero of the Celtic band of brotherhood"), and "Farigarn" is almost certainly the Doric phrase above. Both are exactly the kind of miss the dialect and proper-noun content predicts, not a sign of a broken pipeline.

Caption style reference (assets/reference/kapwing-pop-art-reference.webp)

The brief requires a caption style reference: "a screenshot from any short-form video whose captions you like. Include it in your submission." It is graded: "fidelity to your chosen reference, we compare side by side."

The reference is Kapwing's "Pop Art" subtitle preset, captured from Kapwing's own product page. The image shows three presets offered side by side, Pop Art selected among them, and its captions burned into a short-form video reading "CUSTOMIZE, AND EDIT".

The reference operates at two levels, and both are deliberate.

Kapwing is the product reference. It is why Captionizer exists in the shape it does: a video you can caption, a set of styles offered as a rail, a live preview, and an export. The screenshot literally shows a template picker above a captioned video, which is the surface this project builds. Choosing a tool rather than a single clip was a decision about what to make, not an accident.

Its Pop Art preset is the caption style reference, and the Pop Art template here is adapted from it. Fidelity is claimed at the level of the caption vocabulary rather than as a pixel match: heavy uppercase, no plate, a hard black stroke, a layered drop shadow, and a colour rule where words already spoken turn lime while the word being spoken and the words still to come stay white. That vocabulary is matched deliberately. What was not attempted is a reproduction of one preset, because the third deliverable asks for a system rather than a style, and fifteen templates driven entirely by style.json is the answer to that.

Measured values, taken from the reference and from the founder's Figma reconstruction of its text layer rather than judged by eye:

  1. Lime #78ed5a; the remaining words white.
  2. Tracking -0.76px at 38px, which is -0.02em.
  3. Two layered shadows, in order: 0 4px 3px rgba(0,0,0,0.35) then 0 4px 14px rgba(0,0,0,0.25).
  4. Stroke 2px at 38px, which is 0.053em.
  5. Typeface Outfit Black, already bundled here as a variable font covering weight 900.

Offsets are held as ratios of type size rather than pixels, so they survive a change of caption size.

Type size is the one value not transferred. The reference is a 16:9 landscape video; this project outputs 608x1080 portrait. Three words occupy a very different share of a portrait frame, so copying the reference's ratio would produce type that reads far too small. The size was set by matching visual weight against the reference and holding the line discipline the brief grades: around three words per line, no overflow, nothing in the bottom twelve percent.

The measured result, so a side-by-side comparison is not misread as a miss: the caption block here occupies about 90% of frame width, where the reference occupies about 50%. That is deliberate. A landscape frame gives three words room to sit centred with air either side; a 608 px wide portrait frame does not, and type small enough to occupy half of it would be unreadable at the size these videos are actually watched. 4.0cqh is the measured ceiling rather than a preference: at 4.2 the word "brotherhood" overflows the frame twice in this transcript. Across all 24 segments the set averages 2.98 words per line, widest line 550 px of 608, lowest ink at y=730 against the 950 boundary that marks the bottom twelve percent.

One honest caveat. This is a caption style rendered by a competing product, captured from that product's marketing page, rather than a screenshot of a video found in the wild. It is a real short-form caption treatment and it is the one that was liked, but it is worth stating plainly rather than leaving a reader to notice.

Attribution. Kapwing is a commercial product, not an openly licensed template. THIRD_PARTY_NOTICES.md records this as design attribution, naming what was adapted, and claims no licence.

Development fixture, superseded

An earlier synthetic clip generated locally with ffmpeg (gradient background, sine-tone audio, no speech, hand-authored word timings) was used to exercise the caption path before real footage existed. It has been replaced by the founder's clip above and is no longer in the tree. The command that produced it and the reasoning behind it are preserved in git history and in docs/build/07-transcription-findings.md, which also documents the separate speech fixture built with macOS say for measuring transcription accuracy before this clip was available.