Repository navigation
Conversation
… sparse attention - Engine: FastVideo's FastH3 Omni pack runs its trained 8 steps: 32 stacked output heads fused per step from the config's pdd_* contract on the 0.999 clock; other step counts, turbo and reference-less requests are named 400s. - VSA-H3 sparse attention (src/h3_vsa.zig): FastVideo's tile geometry, routing and gate branch; a NAX block-sparse kernel, the SIMD-group kernel elsewhere, gather + masked SDPA on CPU; pooling and assembly fused into kernels. - Converter --partition omni, oracle fixtures, tests/test_fasth3_omni.sh. - Fast recipe: a step-cache skip advances the audio on the forward's own clock, and a forward after a skip recomputes attention; the two reuse levers had compounded into stale attention that garbled speech on the full model. - App: FastH3 Omni download (fixed 8 steps, no turbo/chain/fast toggle, its own step cost in the estimate); Turbo stays for FL2VA. Enhance sends the reference pictures to a vision chat model. Console: Omni's references and fixed steps. - Docs, NOTICE (FastVideo, Apache-2.0), CHANGELOG.
…decode and a banded sliding prefill - Model: RecursiveSI Flint (model_type `flint`, 39.8B-A2.5B) on laguna's attention arm with learned sinks (`self_attn.softmax_offset`, cast to the activation dtype at load), 14 NoPE full + 35 sliding(513) layers, sigmoid top-k MoE with an ungated shared expert. The release's uint8-packed 4-bit weights are viewed as MLX's affine layout. Oracle against Recursive's rec_mlx model: tests/dump_flint_fixtures.py + the `flint fixture` tests. - Template and tokenizer: jinja.cpp lexes `\xHH` (Python's code point, UTF-8) so Flint's template renders natively; a `.cased` pre-tokenizer style with isolated runs under either grammar matches HF on Flint (tokenizer parity pack added). - Decode: concurrent requests batch-decode (per-slot sink attention); the ungated shared expert rides the affine-4 MoE kernels; NoPE layers take the fused QK-norm kernel (rd 0, two cached configs); the redundant sliding decode mask is gone for laguna/kolibri1/flint (the decode view is the window). - Prefill: the NAX attention kernel takes q/k 128 as well as 192 and serves Flint's sliding layers with their sinks, reading only the band; other GPUs keep the masked SDPA. - App: Flint is chat-pickable. tests/test_flint.sh covers the live surfaces, incl. two concurrent requests through batched decode. The Flint docs (AGENTS rows/rules, CHANGELOG, gotchas, tests/CLAUDE.md) landed in 7072dce.
…in the memory bill, Sushi and SSD-streaming catalog copy - #912: OpenAI's frequency_penalty was read as repeat_penalty 1 + x; Crush's frequency_penalty 1 halved every logit already generated and Qwen3.8-27B wrote its tool calls as prose. It is now its own field, subtracted once per occurrence (applyPenalties); repeat_penalty stays the explicit multiplier. - #932: the free-RAM term of every memory bill counts purgeable pages again (dropped by #776), in status.zig and the app meter's twin. - #843: the Sushi 3bpw pick is gone; the 2bpw pack is the 64 GB Mac pick. - #913: the SSD weight streaming explainer names the GGUF and says MLX packs load fully into memory.
Owner
Author
|
Pushed straight to main instead (94670d9). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three commits, squashed:
frequency_penaltywas read asrepeat_penalty1 + x, so Crush'sfrequency_penalty: 1halved every logit already generated and Qwen3.8-27B wrote its tool calls as prose. It is now its own additive per-occurrence penalty. On the reporter's replayed request, v26.10.2 made 0 good calls in 3 runs; this build makes 4/4 (plus 1/1 streamed).Checks: full
zig build test(4803 passed, 588 skipped),RecommendedModelsTests+SystemMetricsMemoryTests(48 passed), andtests/test_repeat_penalty.shon Qwen3.5-0.8B (all PASS).