Yankovinator

Copyright (C) 2025, Shyamal Suhana Chandra

Invented by Shyamal Chandra

Contact ssc56@duck.com to license code for commercial and non-commercial purposes.

Yankovinator is a Swift package that converts songs into parodies using Apple's NaturalLanguage framework and a local Ollama LLM (llama3.2:3b by default). It preserves syllable structure, rhyme scheme, capitalization, and punctuation while steering lyrics with theme keywords.

The yankovinator CLI (v1.06.0+) runs in batch mode only: put songs in --input-dir, one theme file per entry in --themes-dir, and read parodies from --output-dir as <theme>/<song>.parody.txt. Use benchmark for single-file timing, or the Swift library for one-off generation in code.

Website: https://shyamalschandra.github.io/Yankovinator/

Features

• Syllable-accurate parody generation (word-by-word matching)
• Automatic rhyme detection and enforcement
• Semantic coherence across lines (context-aware generation)
• Theme advancement (develop themes, not only mention keywords)
• Capitalization and punctuation matching
• Theme-based keyword integration
• Automatic keyword generation from subjects via Ollama
• Oxford English Dictionary (1913 / Webster) word suggestions for richer substitutions
• Unsupervised NLP helpers: embedding lexical substitution, rhyme clustering, next-line coherence critic
• NaturalLanguage framework integration
• Local or cloud Ollama integration (llama3.2:3b by default; any --ollama-url)
• Word-by-word part-of-speech matching and OED-filtered substitutions (v1.04.9+)
• ParodyFitScorer global ranking; optional --fit-optimize batch hill-climbing
• Parallel workers for batch jobs (--workers up to 10 / --jobs) with a producer–consumer queue (license max 10 concurrent consumers; optional --consumers cap; :cloud soft default 4)
• Batch TUI progress on stderr: Rust yankovinator-tui (ratatui/crossterm) with color boxes + emoji progress bars per worker thread; Swift ANSI fallback otherwise (--no-progress for plain logs; YANKOVINATOR_RUST_TUI=0 to force fallback)
• Batch-only yankovinator CLI: required --input-dir, --themes-dir, --output-dir (no single-file lyrics arg or --keywords)
• Combinatorial batch: every song × every theme via --input-dir + --themes-dir
• Multi-candidate ranking: --candidates up to 64 generates and scores variants per song×theme
• CLI tools: yankovinator, keyword-generator, benchmark
• XCTest suite (unit + Ollama integration tests)
• LaTeX/Beamer docs and GitHub Pages site

Requirements

• Swift 5.10 or later
• macOS 13.0+ (iOS 16.0+ for library use)
• Ollama installed and running
• llama3.2:3b downloaded in Ollama (ollama pull llama3.2:3b)
• For swift test: full Xcode app (Command Line Tools alone do not provide XCTest)
• Homebrew (optional, for Homebrew install)

Installation

Pre-built binaries (recommended)

Download from GitHub Releases (current: v1.06.14). No Swift toolchain required.

curl -L -o yankovinator-universal.tar.gz \
  https://github.com/shyamalschandra/Yankovinator/releases/download/v1.06.14/yankovinator-universal.tar.gz

tar -xzf yankovinator-universal.tar.gz
sudo mv yankovinator yankovinator-tui keyword-generator /usr/local/bin/
# Optional if present in the archive:
# sudo mv benchmark /usr/local/bin/

yankovinator --help
keyword-generator --help

See docs/RELEASES.md for architecture-specific downloads and troubleshooting.

Homebrew

brew tap shyamalschandra/yankovinator
brew install yankovinator

yankovinator --version          # → 1.06.14+
yankovinator --help
keyword-generator --help

Installs pre-built binaries from GitHub Releases (yankovinator, yankovinator-tui, keyword-generator, benchmark). Tap: homebrew-yankovinator.

brew update && brew upgrade yankovinator

Build from source

git clone https://github.com/shyamalschandra/Yankovinator.git
cd Yankovinator
swift build

Then install and start Ollama (see Ollama Installation).

Usage

Yankovinator ships three CLI tools:

Tool	Purpose
yankovinator	Batch parodies: all songs in --input-dir × all themes in --themes-dir
keyword-generator	Generate keyword: definition pairs from subjects
benchmark	Measure generation performance (single lyrics + keywords file)

Keyword generator

swift run keyword-generator <subject1> [subject2] ... [options]

Options:

• --count, -c <number>: Number of keyword pairs (default: 10)
• --ollama-url, -u <url>: Ollama API base URL (local or cloud; default: http://localhost:11434)
• --model, -m <name>: Ollama model (default: llama3.2:3b)
• --output, -o <file>: Output path (default: stdout)
• --workers, --jobs <n>: Max parallel subject jobs (1–10; license max)
• --verbose, -v: Verbose output

swift run keyword-generator "artificial intelligence" --output ai_keywords.txt
swift run keyword-generator "space exploration" "NASA" --count 15 --output space_keywords.txt
swift run keyword-generator "ai" "space" "music" --workers 10 \
  --ollama-url https://ollama.example.com --output keywords.txt

Parody generator (batch)

Every run is songs × themes. Each file in --themes-dir is a theme (keyword: definition lines). Outputs land under --output-dir/<theme-stem>/<song-stem>.parody.txt.

Swift (development):

swift run yankovinator --input-dir <songs-dir> --themes-dir <themes-dir> --output-dir <out-dir> [options]

Wrapper script (trims stray whitespace from arguments):

./yankovinator.sh --input-dir <songs-dir> --themes-dir <themes-dir> --output-dir <out-dir> [options]

Required:

• --input-dir <dir>: Directory of .txt lyrics files
• --themes-dir <dir>: Directory of theme keyword .txt files (keyword: definition per line)
• --output-dir <dir>: Output root (<theme>/<song>.parody.txt)

Optional:

• --ollama-url, -u <url>: Ollama API base URL (local or cloud)
• --model, -m <name>: Ollama model (default: llama3.2:3b)
• --workers, --jobs <n>: Parallel worker count (1–10; license max). Values above 10 are clamped with a stderr warning. Consumer pool = min(workers, 10, OLLAMA_NUM_PARALLEL on localhost) unless --consumers is set.
• --consumers <n>: Cap in-flight consumer tasks (1–10; license max; default follows --workers). Values above 10 are clamped.
• --ollama-num-parallel <n> (alias --ollama-num-workers): Ollama server OLLAMA_NUM_PARALLEL (or set env before ollama serve; see Ollama FAQ). Consumer effect is also capped at 10 (license).
• --candidates <n>: Generate N ranked variants per song×theme (1–64)
• --fit-optimize: Extra Ollama passes to hill-climb syllable/POS/coherence fit in batch (slower, higher scores)
• --keep-candidates: Also write ranked variants under <song>.candidates/
• --force: Allow songs×themes×candidates totals larger than 100 generations
• --fresh-batch: Ignore/delete checkpoint in --output-dir/.yankovinator and regenerate everything
• --ollama-timeout <sec>: Per-request Ollama HTTP timeout (30–900; heavy :cloud models default to 600s)
• --no-progress: Disable stderr progress bar for batch / multi-candidate runs
• --midi-progress: Lightweight MIDI cues per worker bar (macOS, interactive terminal only)
• --no-cloud-prescription: Disable auto tuning for :cloud models (worker cap ≤4, timeout, fast batch coherence)
• --analyze, -a: Show syllable analysis
• --verbose, -v: Verbose output

Quick start (local Ollama):

mkdir -p songs themes out
cp data/example_lyrics.txt songs/song.txt
cp data/example_keywords.txt themes/space.txt

swift run yankovinator --input-dir ./songs --themes-dir ./themes --output-dir ./out --verbose
# → out/space/song.parody.txt

Rank multiple candidates per song×theme:

swift run yankovinator --input-dir ./songs --themes-dir ./themes --output-dir ./out \
  --candidates 10 --keep-candidates --verbose
# Best: out/<theme>/<song>.parody.txt
# All ranked: out/<theme>/<song>.candidates/
# If songs×themes×candidates > 100, add --force

Stop and resume: Each finished song×theme×candidate is paged to disk under out/.yankovinator/ as it completes (append-only JSONL log + candidate files, fsynced). Re-run the same command after Ctrl+C to skip completed work — resume loads the completion/score index from JSONL and lazy-loads candidate text only when ranking or exporting winners (bounded LRU; not an all-or-nothing in-memory cache). Use --fresh-batch to wipe the checkpoint and start over. If you change songs, themes, --candidates, or --model, the fingerprint will not match—use --fresh-batch or a new --output-dir.

Batch TUI (fix garbled Unicode / segfaults): Release tarballs include yankovinator-tui (Rust + ratatui): ncurses-like color boxes and emoji progress bars for each threaded worker, plus an overall batch gauge and status feed. Install it next to yankovinator (same directory). The CLI auto-spawns it for multi-worker runs. From source: cd tui && cargo build --release (binary at tui/target/release/yankovinator-tui). Override path with YANKOVINATOR_TUI_PATH; disable with YANKOVINATOR_RUST_TUI=0. Use --no-progress for plain logs only.

Cloud Ollama (rate-limit safe):

Cloud models (*:cloud, including via local localhost:11434 proxy) get automatic retries on 429 / 502 / 503 (backoff + Retry-After), extra retries/backoff for DNS/connectivity to ollama.com, a default consumer cap of 4, and a cloud preflight probe before heavy batches. Prefer:

yankovinator --input-dir ./songs --themes-dir ./themes --output-dir ./out \
  --model gemma4:31b-cloud --workers 4 --candidates 20 --force --verbose

Tip: If you see lookup ollama.com: no such host / i/o timeout, check VPN/DNS/firewall (the host running Ollama must resolve ollama.com), retry later, or switch to a local model (--model llama3.2:3b). Single candidate failures no longer abort the whole batch when checkpoints exist — re-run the same command to resume.

Raising --workers / --ollama-num-workers above ~4 against cloud often triggers 429 or ephemeral-port exhaustion (can't assign requested address). Use --no-cloud-prescription only if you accept that risk (still hard-capped at 10 by license terms). If a failed run left a bad checkpoint, add --fresh-batch.

Ollama server parallelism (local ollama serve):

Match Ollama’s OLLAMA_NUM_PARALLEL to your --workers / consumer pool (e.g. 10; license max):

export OLLAMA_NUM_PARALLEL=10   # set before starting the server; restart required
ollama serve

The CLI reads $OLLAMA_NUM_PARALLEL on localhost or you can pass --ollama-num-parallel 10. Local (non-:cloud) models are not capped by cloud prescription, but concurrent consumers never exceed 10 (license).

Heavy cloud batch (qwen3.5:397b-cloud, etc.): stderr prescription caps parallel HTTP workers at 4, may set 600s timeout if unset, and uses batch fast path with checkpoints. Use --no-cloud-prescription for a higher --workers count (up to the license max of 10) at your own rate-limit risk.

Higher cloud concurrency (opt-in, ≤10):

yankovinator --input-dir ./songs --themes-dir ./themes --output-dir ./out \
  --workers 8 --no-cloud-prescription \
  --candidates 20 --force --verbose --model gemma4:31b-cloud

Benchmark

swift run benchmark \
  --lyrics data/example_lyrics.txt \
  --keywords data/example_keywords.txt \
  --iterations 5

swift run benchmark \
  --lyrics data/example_lyrics.txt \
  --keywords data/example_keywords.txt \
  --iterations 10 \
  --workers 10 \
  --ollama-url https://ollama.example.com

Programmatic usage

import Yankovinator

let lyrics = [
    "Twinkle twinkle little star",
    "How I wonder what you are"
]

let keywords = [
    "space": "the physical universe beyond Earth",
    "stars": "luminous celestial bodies"
]

let parody = try await Yankovinator.generateParody(
    originalLyrics: lyrics,
    keywords: keywords,
    ollamaURL: "http://localhost:11434",
    ollamaModel: "llama3.2:3b"
)

for line in parody {
    print(line)
}

Input format

Batch layout

songs/           # --input-dir: one .txt file per song (stem = job id)
  twinkle.txt
  verse2.txt
themes/          # --themes-dir: one .txt file per theme
  space.txt
  science.txt
out/             # --output-dir
  .yankovinator/   # resume checkpoint (manifest.json + completed.jsonl + candidates/)
  space/
    twinkle.parody.txt
  science/
    twinkle.parody.txt

Lyrics file (one song)

One line per verse (empty lines are preserved):

Twinkle twinkle little star
How I wonder what you are
Up above the world so high
Like a diamond in the sky

Sample: data/example_lyrics.txt

Theme / keywords file (one theme)

Format: keyword: definition

science: the study of natural phenomena
space: the physical universe beyond Earth
exploration: the action of traveling to discover

Sample: data/example_keywords.txt

Testing

XCTest needs the Xcode developer directory:

export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
swift test

The suite includes:

• Unit tests for syllable counting
• Keyword extraction tests
• Integration tests for Ollama connection and parody generation

Integration tests require Ollama with llama3.2:3b. If Ollama is unavailable, those tests skip gracefully.

Suggested local smoke checks:

swift build
swift run yankovinator --help
swift run keyword-generator --help
swift run benchmark --help
npm run build   # GitHub Pages TypeScript

Full release gate (unit, regression, UX, A/B, blackbox, Ollama E2E, fit-optimize):

export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
./scripts/certification-battery.sh

See docs/CERTIFICATION.md. Latest run summary: docs/certification-latest.txt.

Documentation

Resource	Description
QUICK_START.md	Fast path from clone to first parody
RELEASE_NOTES_v1.06.14.md	Latest (cloud DNS resilience + batch isolation)
RELEASE_NOTES_v1.06.13.md	License max 10 concurrent consumers
RELEASE_NOTES_v1.06.12.md	Disk-paged batch checkpoints
RELEASE_NOTES_v1.06.10.md	Per-worker elapsed/remain on progress bars
RELEASE_NOTES_v1.06.9.md	TUI color boxes + emoji bars per worker
RELEASE_NOTES_v1.06.8.md	Expanded certification battery + docs/Pages
RELEASE_NOTES_v1.06.7.md	Fit-optimize parallel NL segfault fix
RELEASE_NOTES_v1.06.6.md	PATH TUI resolve + no Swift multiline segfault
RELEASE_NOTES_v1.06.5.md	Prior (MIDI prewarm / resume / Rust TUI)
docs/CERTIFICATION.md	Certification battery (unit, regression, UX, A/B, blackbox, E2E)
RELEASE_NOTES_v1.06.4.md	Resume checkpoint + Rust TUI
RELEASE_NOTES_v1.06.3.md	Prior release (ANSI alt-screen TUI fix)
RELEASE_NOTES_v1.06.2.md	Superseded ncurses experiment
RELEASE_NOTES_v1.06.1.md	Parallel batch crash fix
RELEASE_NOTES_v1.06.0.md	Batch-only CLI
docs/README.md	GitHub Pages site source
docs/RELEASES.md	Binary release install guide
docs/DEPLOYMENT.md	Pages deployment status
docs/yankovinator.tex	Technical paper (LaTeX)
docs/presentation.tex	Beamer slides
docs/reference.tex	API reference manual

Generate PDFs:

cd docs
pdflatex yankovinator.tex
pdflatex presentation.tex
pdflatex reference.tex

Architecture

Core components

1. SyllableCounter — syllable structure via NaturalLanguage + heuristics
2. RhymeSchemeAnalyzer — baseline rhyme groups / schemes
3. UnsupervisedRhymeClustering — phonetic + embedding rhyme discovery (unlabeled)
4. LexicalSubstitutionEngine — syllable-matched NLEmbedding neighbors (MLM-style)
5. CoherenceCritic — next-line surprise / coherence scoring (embedding + optional Ollama)
6. OEDDictionary — dictionary-backed word suggestions
7. OllamaClient — Ollama HTTP API (AsyncHTTPClient)
8. ParodyGenerator — generation + refinement pipeline
9. BenchmarkRunner — timing harness for CLI benchmarking
10. Yankovinator — public library facade

Technology stack

• Swift 5.10+ / SwiftPM
• NaturalLanguage
• Ollama (llama3.2:3b default)
• AsyncHTTPClient
• ArgumentParser
• XCTest

Algorithm (high level)

1. Analyze syllable structure (line totals and per-word patterns)
2. Detect rhyme scheme from the original lyrics
3. For each non-empty line:
   • Apply syllable, rhyme, keyword, and context constraints
   • Request a candidate from Ollama
   • Refine word-by-word syllables and semantic coherence
   • Match capitalization and punctuation to the original
4. Preserve empty-line structure in the output

Ollama installation

Method 1: Ollama GUI (recommended)

1. Download from https://ollama.ai/download or brew install --cask ollama-app
2. Launch Ollama from Applications
3. Pull llama3.2:3b in the UI or via CLI

Method 2: Homebrew CLI

brew install ollama
ollama serve
ollama pull llama3.2:3b
ollama list

Verify

curl http://localhost:11434/api/tags
ollama run llama3.2:3b "Hello, how are you?"

Troubleshooting

Ollama not running

curl http://localhost:11434/api/tags
# GUI: open Ollama from Applications
# CLI: ollama serve

Model not found

ollama list
ollama pull llama3.2:3b
ollama show llama3.2:3b

swift test fails with no such module 'XCTest'

Point at Xcode, not Command Line Tools:

sudo xcode-select -s /Applications/Xcode.app/Contents/Developer
# or for a single shell:
export DEVELOPER_DIR=/Applications/Xcode.app/Contents/Developer
swift test

Connection refused

1. Confirm Ollama is running
2. Check the port: lsof -i :11434
3. Override URL if needed: --ollama-url http://localhost:11434

Syllable count mismatch

Syllable counting uses heuristics and may differ on unusual words; that is expected.

License

Copyright (C) 2025, Shyamal Suhana Chandra

Invented by Shyamal Chandra

Contact ssc56@duck.com to license code for commercial and non-commercial purposes.

References

• Apple NaturalLanguage Framework
• Ollama Documentation
• Ollama Download
• Swift Package Manager
• Yankovinator on GitHub Pages