Auto device fallback on provider failure and tokenizer_options passthrough in Text2TextGenerationPipeline - #1670

Closed

Suh0161 wants to merge 5 commits into

huggingface:mainfrom

Suh0161:feat/cuda-fallback-tokenizer-options

Commits on May 1, 2026

Add start/end offsets to token classification and return_offsets_mapping to tokenizer
Suh0161
committed
53cb31b
Fix num_logits_to_keep default and add get_available_devices()
Suh0161
committed
a90fc32
Auto device fallback on provider failure and tokenizer_options passthrough
Suh0161
committed
9f498ef
Refactor #1642 and #1096 fixes with proper separation of concerns
Suh0161
committed
57c4267
fix(tokenizer): align GPT-2 offset_mapping with ByteLevel space prefix
Suh0161
committed
72d4c78