Repository navigation
NLP explainability: standalone NlpExplainer stack - #775
Merged
Merged
Conversation
after the user picks a token (e.g. great, terrible), color all points by that token's SHAP contribution
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## develop #775 +/- ##
===========================================
+ Coverage 80.16% 86.08% +5.92%
===========================================
Files 59 115 +56
Lines 7410 11825 +4415
Branches 1557 2158 +601
===========================================
+ Hits 5940 10180 +4240
- Misses 901 1009 +108
- Partials 569 636 +67
🚀 New features to boost your workflow:
|
…ce SHAP - NlpExplanation carries output_space, is_signed and baseline_token; all three are saved to meta.json and output_space/baseline_token enter the cache key - NlpShapBackend(output_space="logit") explains raw logits via the new SupportsLogits capability - LIG uses [MASK] as its baseline token (like SHAP) and defaults to n_steps=100 - serve_nlp.py gains --output-space; the webapp info box shows the space
- NlpLimeBackend accepts a TextModel, like the SHAP and LIG backends: it scores through `predict` and takes `label_names` from the model unless you pass your own. - NlpLimeBackend without `label_names` now asks the model for its output width and explains every class. Before, LIME explained class 1 only and the contributions were silently zero. - NlpExplainer._predict accepts a bare scoring callable that returns an (n_texts, n_classes) array (e.g. sklearn's predict_proba), with clear errors for a non-2-D array or mismatched label_names. - explain(cache_dir=...) refuses a bare callable: it has no identity, so two different callables would share cache entries. HFPipelineModel.model_id now includes the pipeline's checkpoint for the same reason. - The webapp honours `is_additive`: the Waterfall tab is not mounted and the "Base + Σ = Total" summary is dropped (webapp and `.plot.sentence`) for non-additive explanations such as LIME. - Rewrite the NlpLimeBackend and NlpExplainer docstrings with runnable examples (TextModel, sklearn predict_proba, HF pipeline). Tests: merge test_captum_backends.py into test_nlp_backends.py, with one section per backend (SHAP, LIG, LIME) plus the cross-backend checks. Only the LIME tests are gated on the `lime` extra. Move the AblationFlip flip test to test_nlp_whatif.py and drop the pipeline auto-bind test, which that file already covered. Add unit tests for the fixes above.
- default num_features to "all", sized per text from LIME's own split - cast vocabulary tokens to plain str - add show_progress, sharing _progress_iter on NlpBackend with LIG - fix random_state=0 in the docstring examples; document passing it as an int
- add explanation.plot.compare(others, kind="heatmap"|"bars"|"highlight") - align each backend's word units to the reference by character span (NaN where a backend attributes nothing, e.g. punctuation for LIME) - normalise per backend (max_abs by default, sum_abs or raw) - show Spearman and top-k overlap against the reference in the subtitle - warn when the explanations are in different output spaces
the two tests used assertWarns, which loops over every module in sys.modules and reads __warningregistry__. Once other tests have loaded transformers, that read triggers its lazy imports, and one of them pulls in torchvision which is not installed
- corpus_agreement() scores every text × backend pair; plot.compare_corpus() shows the distribution - add pearson, cosine and per-backend top-word share to the agreement scores - rename compute/token_alignment.py to compute/backend_comparison.py; add explainer/nlp_comparison.py
Each NLP backend now records, for every unit it explains, the (start, end) character spans of the source text that unit covers. These spans are a coordinate all backends can share, even though they cut a text into units differently and have no common token axis. - New shapash.compute.spans: build spans from tokenizer offsets, trim whitespace, partition a text into groups every unit fits in, aggregate unit values onto those groups, and locate spans from strings for older artifacts. - NlpContributions and NlpExplanation carry token_spans. The .xpl file stores them in token_spans.parquet. Older files load without spans. - SHAP reads the masker tokenizer's offsets. Words are taken from the source text, so characters shared by two tokens are no longer duplicated, and tokens that share a character merge into one word. - LIG reads offsets through the new SupportsCaptumIG.token_offsets(). Words are shown as written (case, accents, emoji), and units covering only whitespace are folded into the baseline, so attributions still sum to the total. - LIME records every occurrence of each word. - The cache key now includes a unit-layout version, so cached explanations are recomputed once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Backend comparison now matches units by the characters they cover
instead of searching for their strings in the text. String search failed
silently on every tokenizer except DistilBERT's: words were dropped
(stripped accents, glued punctuation, emoji) or given another word's
value (naive vs naïve).
- align_units merges all backends' units into the finest groups every
unit fits in, then sums each backend's values per group. A group with
no unit is NaN. Where one backend's unit covers several of another's
("didn't" against "didn" "'" "t"), the finer units are summed into it,
so no attribution is lost. LIME's single weight per word counts at
each occurrence.
- align_row reads all backends on these shared units instead of the
reference backend's words.
- align_token_values and locate_tokens are removed.
- Explanations saved without spans fall back to string search, and
check_comparable warns about it.
- backend_agreement adds a coverage score: the share of groups both
backends attribute.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Runs the real SHAP, LIG and LIME backends on six tokenizer families: WordPiece (uncased), byte-level BPE, ModernBERT BPE, and SentencePiece (XLM-R unigram, uncased ALBERT, DeBERTa-v3). The input texts are chosen to break string matching: accents (precomposed and decomposed), emoji, curly quotes, full-width characters, double spaces and newlines, contractions, and CJK. The tests check that every unit is placed and reads as written, that SHAP and LIG cover the same characters, that SHAP stays additive, that alignment keeps each backend's total, and that truncation shows up as partial coverage. The models are tiny and randomly initialised from each checkpoint's config, so only the config and tokenizer are downloaded. SHAP's Text masker rebuilds its input from segments. That can collapse whitespace and repeat characters shared by two tokens, so the additivity test checks against the string the masker actually builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Some docstrings and comments pointed to design notes that are not part of the repository. Those references are removed; the explanations stay in the code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
shap.maskers.Text re-tokenizes masked strings, so the model can see another text ("était" -> "éétait", collapsed whitespace), and hiding a token changes its neighbours ("Ġlove" -> "love").
- NlpShapBackend masks the model's own encoding when the model exposes token ids with a fast tokenizer: hidden tokens take LIG's [MASK] reference ids, and each word is one branch of the Partition hierarchy. Other models, an explicit masker= or explainer_args keep the string path; backend.masking reports which path ran. NlpExplainer's default backend takes the new path.
- A shared WordLayout (compute/word_layout.py) builds words, spans and baseline folding for both SHAP and LIG, so they report identical units.
- The cache key uses shapash.__version__ instead of a units layout version.
- Tests: exact token-id SHAP values on an additive fake model, word layout unit tests, and integration checks across six tokenizer families (additivity against the text, SHAP/LIG unit identity, SHAP vs LIME in probability space, LIG fallbacks without offsets).
guillaume-vignal
approved these changes
Oct 13, 2026
milton-minervino
marked this pull request as ready for review
October 13, 2026 14:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a parallel, standalone stack for explaining text classifiers —
NlpExplainer/NlpExplanation/NlpWebApp— built alongsideSmartExplainerrather than through it. It deliberately bypassesSmartExplainer.compile()instead of adding a half-finishedmodality="text"branch that would need to be written and then deleted once tabular and text are unified behind one API. Full design rationale, architecture diagram, and API reference:docs/wiki/NLP-Explainability.mdNothing about the existing public API changed —
shapash/__init__.pyandshapash/explainer/smart_explainer.pyare untouched.SmartExplainerusers are unaffected; thenlpextra is opt-in and adds no dependencies to the default install.What's in this PR
1. The NLP stack (new code)
shapash/model/— capability-basedTextModellayer (ABC +SupportsTokenization/Embeddings/Gradients/CaptumIGmixins) with HF pipeline, HF checkpoint, torch encoder+head, and sentence-transformers adaptersshapash/backend/nlp_{shap,lime,captum_lig}_backend.py— three interchangeable attribution backends, self-registered by.nameshapash/compute/generators/—HotFlipGenerator/AblationFlipGeneratorcounterfactuals over a shared minimal-perturbation searchshapash/compute/diagnostics/— in-house confident-learning label-noise detection + an independentLabelProbeto distinguish label errors from model errorsshapash/compute/{embeddings,embedding_store,retrieval}— embedding cache + nearest-neighbour similar-example retrievalshapash/explainer/{nlp_explainer,nlp_explanation,nlp_plotter,interactive}.py—NlpExplainer(fit/explain, memoized),NlpExplanation(frozen, model-free artifact, save/load as zip of parquet — no pickle)shapash/webapp/nlp_app.py+nlp_components/— a second, capability-gated Dash app (10 panels: word importance, scatter, waterfall, error analysis, what-if lab, etc.)shapash/plots/— 8 new pure plot functions reused by the abovetutorial/nlp/overview.ipynb,demo/serve_nlp.py(+ Dockerfile/compose) — runnable end-to-end examples2. Small, deliberate touches to shared/tabular code
shapash/backend/{__init__,base_backend,shap_backend}.py— generalizedBackendbase class soget_backend_cls_from_namecovers both tabular and text backends;BaseBackendnow extends it (tabular backends are unaffected)shapash/webapp/smart_app.py— fixes an index-selection click-relay bug in the tabular webapp (n_submit/n_clickswere treated as booleans instead of counters, which could drop triggers); own test intest_smart_app_click_relay.pyshapash/plots/plot_contribution.py,plot_evaluation_metrics.py,style/colors.json,style/style_utils.py,webapp/assets/style.css— shared styling/utility additions consumed by the new NLP plots, reused as-is by tabular codedevelopmerges (plotly/scikit-learn upgrade compatibility)3. Packaging & CI
nlpextra (torch, transformers, datasets, sentence-transformers, sentencepiece, protobuf, captum, pacmap) — opt-in,uv sync --extra nlporpip install ".[nlp]"[dependency-groups]; the oldallextra is removedpandas_compatjob explicitly asserts the NLP stack stays absent to guard the classical (non-NLP) install pathCONTRIBUTING.mdrewrite, PR template nudge to targetdevelop, new.dockerignoreWhy a separate stack, not a
SmartExplainerextensionText classification needs a materially different pipeline (tokenization, gradient/embedding access, counterfactual search, label-noise diagnostics) that doesn't map onto
compile()'s tabular assumptions. Building it as a capability-basedModel/Interpreter/Generatorsplit (similar to Google PAIR's LIT) alongsideSmartExplainerlets it ship and be used now, while keeping the pieces (TextModelcapabilities,Backend,CounterfactualGenerator,WebappComponent) positioned to generalize into a unified API later rather than being a throwaway prototype. See "Why a separate stack" in the linked doc for the full reasoning.Type of change
shapash/webapp/smart_app.py)docs/wiki/NLP-Explainability.md)How has this been tested?
tests/unit_tests/{model,compute,backend,explainer,webapp,plots}/...), including dedicated coverage for each of the 10 new webapp panels and all 8 new plot functionstests/integration_tests/exercise the stack against a real (tiny) HF model; NLP test modules are markedpytest.mark.nlpand skip cleanly when thenlpextra isn't installedpandas_compatjob confirming the classical install has no NLP dependenciesdemo/serve_nlp.pyagainstbhadresh-savani/distilbert-base-uncased-emotion/dair-ai/emotion(cached run indemo/nlp_cache/) andtutorial/nlp/overview.ipynbTest configuration:
feature/nlp-refacto)Checklist
shapash/)docs/wiki/NLP-Explainability.md)