NLP explainability: standalone NlpExplainer stack

.dockerignore

@@ -0,0 +1,47 @@
+# Build context for demo/Dockerfile (context = repo root — see demo/README.md).
+#
+# The image only needs pyproject.toml, README.md, MANIFEST.in, shapash/, demo/serve_nlp.py
+# and demo/nlp_cache/. Everything below is either huge or irrelevant to the build; without
+# these excludes the context is several GB (.venv alone is multi-GB), which is minutes of
+# upload on a remote Docker host.
+
+# Virtualenvs and VCS
+.git
+.gitignore
+.venv
+venv
+
+# Build / packaging / coverage artifacts
+dist/
+build/
+htmlcov/
+.coverage
+*.egg-info/
+
+# Not used by the image
+docs/
+tutorial/
+tests/
+data/
+dev/
+catboost_info/
+demo/private_model/
+
+# Caches
+**/__pycache__/
+*.py[cod]
+.mypy_cache/
+.pytest_cache/
+.ruff_cache/
+.ipynb_checkpoints/
+
+# Editor / agent config
+.idea/
+.vscode/
+.claude/
+
+# Optional: ship only the model cache actually being served (saves ~50 MB, and keeps
+# unrelated models' contributions out of the image). Uncomment both lines together —
+# the second re-includes the one directory the default ServeConfig needs.
+# demo/nlp_cache/*/
+# !demo/nlp_cache/bhadresh-savani__distilbert-base-uncased-emotion/

.github/workflows/main.yml

@@ -33,7 +33,7 @@ jobs:
       # mypy needs `test` optional dependencies that ship types
       run: |
         python -m pip install --upgrade pip
-        python -m pip install '.[dev,test]'
+        python -m pip install . --group dev --group test
     - name: Check linting
       run: make lint
     - name: Check formatting
@@ -42,14 +42,18 @@ jobs:
       run: make typecheck
 
   test:
-    name: Build & Test
+    name: Test (Python ${{ matrix.python-version }})
     needs: code_quality
     runs-on: ubuntu-latest
     timeout-minutes: 60
     strategy:
       fail-fast: false
       matrix:
         python-version: ["3.11", "3.12", "3.13", "3.14"]
+        include:
+          # One job measures coverage and uploads it: the others run the same suite.
+          - python-version: "3.12"
+            coverage: true
     steps:
     - uses: actions/checkout@v6
       with:
@@ -60,24 +64,45 @@ jobs:
         python-version: ${{ matrix.python-version }}
         cache: pip
         cache-dependency-path: pyproject.toml
+    # Full suite, NLP included. CPU-only torch first: the PyPI Linux wheel drags in several GB of
+    # CUDA libraries the tests never use. The `nlp` extra's `torch>=` bound is then already
+    # satisfied and pip keeps it.
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        python -m pip install '.[all]'
-    - name: Build python package
-      run: |
-        python -m pip install --upgrade build
-        python -m build
+        python -m pip install torch --index-url https://download.pytorch.org/whl/cpu
+        python -m pip install '.[nlp]' --group test
+    # NLP test modules skip (not fail) when their imports are missing, so fail loudly here instead.
+    - name: Check NLP stack is importable
+      run: python -c "import torch, transformers, sentence_transformers, captum; print(torch.__version__)"
+    # Models the NLP tests download from the Hugging Face Hub: reuse them instead of hitting the Hub
+    # from every job. The key follows the tests, so a newly referenced model gets saved too.
+    - name: Cache Hugging Face models
+      uses: actions/cache@v6
+      with:
+        path: ~/.cache/huggingface
+        key: hf-models-${{ hashFiles('tests/**/*.py') }}
+        restore-keys: hf-models-
+    - name: Test
+      if: ${{ !matrix.coverage }}
+      run: make test
     - name: Test and coverage
+      if: ${{ matrix.coverage }}
       run: make coverage-report
     - name: Upload coverage to Codecov
-      if: matrix.python-version == '3.13'
+      if: ${{ matrix.coverage }}
       uses: codecov/codecov-action@v6
       with:
         token: ${{ secrets.CODECOV_TOKEN }}
         files: ./coverage.xml
         fail_ci_if_error: false
         disable_search: true
+    # Pure-Python package: building it once is enough.
+    - name: Build python package
+      if: ${{ matrix.coverage }}
+      run: |
+        python -m pip install --upgrade build
+        python -m build
 
   pandas_compat:
     name: Pandas Compat (${{ matrix.pandas }})
@@ -87,7 +112,8 @@ jobs:
     strategy:
       fail-fast: false
       matrix:
-        pandas: ["2.2.2", "2.3.*", "3.*"]
+        # Lower bound + last 2.x. Latest pandas 3.x is already what the `test` jobs resolve to.
+        pandas: ["2.2.2", "2.3.*"]
     steps:
     - uses: actions/checkout@v6
       with:
@@ -98,11 +124,13 @@ jobs:
         python-version: "3.12"
         cache: pip
         cache-dependency-path: pyproject.toml
+    # Classical-install jobs: deliberately without the `nlp` extra.
     - name: Install dependencies
       run: |
         python -m pip install --upgrade pip
-        python -m pip install '.[all]'
-        python -m pip install "pandas==${{ matrix.pandas }}"
+        python -m pip install . --group test "pandas==${{ matrix.pandas }}"
         python -c "import pandas; print(pandas.__version__)"
-    - name: Test and coverage
-      run: make coverage
+    - name: Check NLP stack is absent
+      run: python -c "import importlib.util as u; assert u.find_spec('torch') is None, 'torch installed'"
+    - name: Test
+      run: make test

.pre-commit-config.yaml

@@ -49,6 +49,7 @@ repos:
     rev: 1.20.0
     hooks:
     -   id: blacken-docs
+        args: [--line-length=120]
         additional_dependencies: [black==26.5.1]
 -   repo: https://github.com/compilerla/conventional-pre-commit
     rev: v3.3.0

CONTRIBUTING.md

@@ -45,16 +45,24 @@ git pull upstream develop
 
 Python `>=3.11, <3.15` is required.
 
-After creating a virtual environment, run:
+We recommend `uv`. A plain sync installs shapash with the `dev` and `test` dependency groups:
 ```
-pip install -e ".[dev,test]"
+uv sync
 ```
 
-Alternatively, we recommend using `uv`:
+Alternatively, with pip (>= 25.1), after creating a virtual environment, run:
 ```
-uv sync --extra dev --extra test
+pip install -e . --group dev --group test
 ```
 
+This covers classical (tabular) shapash. The NLP modules and their tests need the heavier `nlp`
+extra (torch, transformers, ...); without it, those tests are skipped. To work on NLP:
+```
+uv sync --extra nlp            # or: pip install -e ".[nlp]" --group dev --group test
+```
+A later plain `uv sync` removes the NLP packages again (it syncs exactly). `uv run` and
+`uv sync --inexact` leave them in place.
+
 See `pyproject.toml` for other dependency groups.
 
 ## Set up pre-commit hooks

README.md

@@ -69,6 +69,7 @@ Shapash is suitable for Regression, Binary Classification and Multiclass problem
 
 | Version       | New Feature                                                                           | Description                                                                                                                            | Tutorial |
 |:-------------:|:-------------------------------------------------------------------------------------:|:--------------------------------------------------------------------------------------------------------------------------------------:|:--------:|
+| Preview       |  NLP explainability  | Explain text classifiers at the token level, with a webapp, a What-if Lab (counterfactuals) and label-noise detection. Opt-in: `pip install shapash[nlp]` |  [notebook](https://github.com/MAIF/shapash/blob/feature/nlp-refacto/tutorial/nlp/overview.ipynb) |
 | 2.3.x         |  Additional dataset columns <br> [New demo](https://shapash-demo.ossbymaif.fr/) <br> [Article](https://pub.towardsai.net/shapash-2-3-0-comprehensive-model-interpretation-40b50157c2fb)                                                                | In Webapp: Target and error columns added to dataset and possibility to add features outside the model for more filtering options            |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/add_column_icon.png" width="50" title="add_column">](https://github.com/MAIF/shapash/blob/master/tutorial/generate_webapp/tuto-webapp01-additional-data.ipynb)
 | 2.3.x         |  Identity card <br> [New demo](https://shapash-demo.ossbymaif.fr/) <br> [Article](https://pub.towardsai.net/shapash-2-3-0-comprehensive-model-interpretation-40b50157c2fb)                                                                  | In Webapp: New identity card to summarize the information of the selected sample                  |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/identity_card.png" width="50" title="identity">](https://github.com/MAIF/shapash/blob/master/tutorial/generate_webapp/tuto-webapp01-additional-data.ipynb)
 | 2.2.x         |  Picking samples <br> [Article](https://www.kdnuggets.com/2022/11/picking-examples-understand-machine-learning-model.html)                                                                | New tab in the webapp for picking samples. The graph represents the "True Values Vs Predicted Values"            |  [<img src="https://raw.githubusercontent.com/MAIF/shapash/master/docs/_static/picking.png" width="50" title="picking">](https://github.com/MAIF/shapash/blob/master/tutorial/plots_and_charts/tuto-plot06-prediction_plot.ipynb)
@@ -144,6 +145,39 @@ Shapash can use category-encoders object, sklearn ColumnTransformer or simply fe
 - Category_encoder: *OneHotEncoder*, *OrdinalEncoder*, *BaseNEncoder*, *BinaryEncoder*, *TargetEncoder*
 - Sklearn ColumnTransformer: *OneHotEncoder*, *OrdinalEncoder*, *StandardScaler*, *QuantileTransformer*, *PowerTransformer*
 
+## 🔤 Text / NLP Explainability (preview)
+
+Shapash also makes **text classifiers** explainable: see which words drove a prediction, why the model chose one label over another, and how the prediction changes when the text does.
+The NLP stack (`NlpExplainer`, `NlpExplanation`, `NlpWebApp`) is a standalone, opt-in module that lives alongside `SmartExplainer`:
+the tabular API is unchanged and the default install pulls no NLP dependency.
+
+- **Token-level explanations** for a text classifier: which words pushed the prediction towards which label. Choose between SHAP, LIME and Captum (Integrated Gradients) backends
+- **Local and global views**: sentence highlights and waterfall plots for one text; word importance, word profiles and a confusion matrix with per-error word analysis for the whole corpus
+- **What-if Lab**: edit a text and see the live prediction, and get auto-generated minimal word changes (counterfactuals) that flip it
+- **Dataset exploration**: embeddings scatter plot, similar examples, and label-noise detection that tells label errors from model errors
+- **Portable results**: `explain()` returns a model-free artifact that can be saved and reloaded on its own (no pickle), and explored in a dedicated Dash webapp
+
+Works with encoder-based single-label classifiers (BERT, DistilBERT, RoBERTa, DeBERTa, CamemBERT, XLM-R...) from Hugging Face,
+sentence-transformers or plain PyTorch.
+
+```python
+from shapash.model.hf import HFClassifierModel
+from shapash.explainer.nlp_explainer import NlpExplainer
+
+labels = ["sadness", "joy", "love", "anger", "fear", "surprise"]
+model = HFClassifierModel.from_pretrained("bhadresh-savani/distilbert-base-uncased-emotion", label_names=labels)
+
+xpl = NlpExplainer(model, label_names=labels)
+explanations = xpl.explain(["i am so happy today", "i feel terrified"])
+
+explanations.plot.word_importance()  # corpus-level word importance
+xpl.run_app(explanations)  # interactive webapp
+```
+
+Install it with `pip install "shapash[nlp]"`. Learn more in the [overview notebook](https://github.com/MAIF/shapash/blob/feature/nlp-refacto/tutorial/nlp/overview.ipynb),
+the [technical documentation](https://github.com/MAIF/shapash/blob/feature/nlp-refacto/docs/wiki/NLP-Explainability.md)
+and the [demo app](https://github.com/MAIF/shapash/blob/feature/nlp-refacto/demo/README.md).
+
 ## 🛠 Installation
 
 Shapash is intended to work with Python versions 3.11 to 3.14. Installation can be done with pip:
@@ -158,6 +192,11 @@ You can install these using the following command :
 pip install shapash[report]
 ```
 
+To explain text models, install the NLP extra (torch, transformers, captum...):
+```bash
+pip install shapash[nlp]
+```
+
 If you encounter **compatibility issues** you may check the corresponding section in the Shapash documentation [here](https://shapash.readthedocs.io/en/latest/installation-instructions/index.html).
 
 ## 🕐 Quickstart

demo/Dockerfile

@@ -0,0 +1,52 @@
+# NLP Explainer image (CPU-only, live model — the What-if Lab needs it in memory).
+#
+# This is a convenience wrapper around `demo/serve_nlp.py`, the actual serving script —
+# run that once to generate `demo/nlp_cache/` before building. The expensive part — SHAP +
+# the projection — is computed ONCE outside Docker and copied in as a cache (see demo/README.md);
+# the build never runs SHAP. The model + dataset are downloaded in a build layer (fast, ~270 MB,
+# cached like any pip layer), so runtime is fully offline: no downloads, no SHAP, no GPU at run time.
+#
+# Build from the repo root (fails fast if nlp_cache is missing — do Step 1 first):
+#   docker build -f demo/Dockerfile -t shapash-nlp .
+#   docker run --rm -p 8050:8050 shapash-nlp
+# Then open http://localhost:8050
+
+FROM python:3.11-slim
+
+WORKDIR /app
+
+# CPU-only torch first (from PyTorch's CPU index — smaller, no CUDA), so the [nlp] extra
+# sees torch already satisfied and doesn't pull the CUDA build from PyPI.
+RUN pip install --no-cache-dir --index-url https://download.pytorch.org/whl/cpu "torch>=2.12.1"
+
+COPY pyproject.toml README.md MANIFEST.in ./
+COPY shapash/ ./shapash/
+# setuptools-scm can't find .git in the build context; any placeholder works since the
+# running app doesn't use the installed package version at runtime.
+ENV SETUPTOOLS_SCM_PRETEND_VERSION_FOR_SHAPASH=0.0.0.dev
+RUN pip install --no-cache-dir ".[nlp]" "pyarrow>=17.0.0"
+
+# Download the model + dataset into the image (a plain download, not SHAP compute).
+ENV HF_HOME=/app/hf_cache
+RUN python -c "\
+from transformers import AutoTokenizer, AutoModelForSequenceClassification; \
+import datasets; \
+m = 'bhadresh-savani/distilbert-base-uncased-emotion'; \
+AutoTokenizer.from_pretrained(m, use_fast=True); \
+AutoModelForSequenceClassification.from_pretrained(m); \
+datasets.load_dataset('dair-ai/emotion', split='test')"
+
+COPY demo/serve_nlp.py ./serve_nlp.py
+# Pre-computed SHAP + projection cache (built by running serve_nlp.py once — see README).
+COPY demo/nlp_cache/ ./nlp_cache/
+
+# Serve fully offline: model from the baked HF cache, SHAP from nlp_cache, no network.
+# PYTHONUNBUFFERED streams logs; TOKENIZERS_PARALLELISM avoids a fork deadlock.
+ENV HF_HUB_OFFLINE=1 \
+    TRANSFORMERS_OFFLINE=1 \
+    TOKENIZERS_PARALLELISM=false \
+    PYTHONUNBUFFERED=1
+
+EXPOSE 8050
+
+CMD ["python", "serve_nlp.py", "--n", "1000", "--n-reference", "5000"]

demo/README.md

@@ -0,0 +1,82 @@
+# Shapash NLP Explainer — Demo
+
+Interactive webapp that explains a DistilBERT emotion classifier at the token level using SHAP
+values, with a live **What-if Lab**: edit a sample's text for a live prediction + token-contribution
+highlight, and get auto-generated minimal token flips (counterfactuals) that change the prediction.
+
+**Webapp layout**, three panels:
+- Left: control panel with **Dataset** (Dash AG Grid to explore the dataset and click on texts),
+  **Embeddings** (scatter plot, used to lasso/box select, highlight word importance), and **Data Editor**
+- Upper Right: **Word Importance** (global explainability) tab and **Counterfactuals** tab.
+- Lower Right: **Local contributions** divided as sentence highlights tab and waterfall plot tab.
+
+**`demo/serve_nlp.py` is the serving script** — everything below runs it with its defaults (the
+emotion model/dataset). `Dockerfile`/`docker-compose.yml` are just a convenience wrapper around it:
+they never run SHAP themselves, they only copy in a cache built by running the script once (Step 1
+below). Model, dataset, split, columns, sample count, and a ground-truth label-renaming map are all
+configurable via `ServeConfig` in that file — see its module docstring for
+`--model-name`/`--dataset-name`/`--dataset-split`/`--text-column`/`--label-column`/`--label-map`/`--n`
+and worked examples (e.g. swapping in an IMDB sentiment model/dataset). **Nothing has to come from the
+HuggingFace hub**: `--model-name /path/to/checkpoint` serves a local `save_pretrained` directory (its
+tokenizer is resolved from the checkpoint, falling back to the base model in its `config.json`; pass
+`--tokenizer-name` for a customised one, and `--label-names` when the config has no class names), and
+`--dataset-path` serves local `<split>.parquet` / `<split>.pkl` dataframes instead of `--dataset-name`.
+A private model + private data gets the full webapp — highlights, What-if Lab, counterfactuals,
+similar examples — like any hub model. `Dockerfile` itself stays
+hardcoded to the emotion example on purpose (it's meant to run out of the box for anyone cloning the
+repo) — its build-time warm-up (`RUN python -c "..."`) downloads exactly that model/dataset/split, so
+it always matches `ServeConfig`'s defaults. Serving a different model/dataset combo in Docker means
+editing that warm-up line to match, then redoing Step 1 below with the same flags before rebuilding.
+
+---
+
+## Prerequisites
+
+- [Docker](https://docs.docker.com/get-docker/) installed and running (for the Docker path), or the
+  `[nlp]` extra installed locally (`pip install "shapash[nlp]"`)
+
+## Step 1 — Pre-compute the SHAP cache (one-time, GPU optional)
+
+Needs the `[nlp]` extra (torch, transformers, datasets, pacmap). Run the script once — it computes SHAP
++ the projection (a few minutes on CPU, faster on GPU), writes the cache, then starts serving. Once it
+prints the local URL, the cache is on disk — press `Ctrl+C`:
+
+```bash
+pip install "shapash[nlp]"
+python demo/serve_nlp.py   # Ctrl+C once it starts serving
+```
+
+This writes `demo/nlp_cache/` — SHAP contributions, predictions, and the 2-D projection, keyed by a
+hash of the input texts. The Docker build copies it in, so this expensive step never runs in Docker.
+
+> **Already have `demo/nlp_cache/`?** Skip to Step 2.
+
+## Step 2 — Build and run
+
+Run from the **repo root**:
+
+```bash
+docker compose -f demo/docker-compose.yml up --build shapash-nlp
+```
+
+The build downloads the model + dataset into the image and copies in the SHAP cache; at **run** time it
+serves fully offline — no downloads, no SHAP, no GPU. (The build fails fast if `demo/nlp_cache/` is
+missing — do Step 1 first.)
+
+Then open **http://localhost:8050** in your browser. Press `Ctrl+C` to stop.
+
+## Run without Docker
+
+Just run the same script — it serves on http://127.0.0.1:8050 and reuses the `demo/nlp_cache/`
+from Step 1 (recomputing only if the input texts change, or if you pass `--recompute`):
+
+```bash
+python demo/serve_nlp.py             # ServeConfig defaults (500 samples, dair-ai/emotion test split)
+python demo/serve_nlp.py --n 1000    # more samples (recomputes + re-caches)
+```
+
+It uses the GPU if present, otherwise CPU.
+
+> **Iterating on the webapp code?** Don't restart the process — run `serve_nlp.py`'s body in a REPL
+> once, then re-call `xpl.run_app(...)` after reloading the changed webapp modules. The model stays
+> resident, so you never pay the reload.