Repository navigation

NLP explainability: standalone `NlpExplainer` stack - #775

Merged

milton-minervino merged 147 commits into

developfrom

feature/nlp-refacto

Oct 14, 2026

Merged

NLP explainability: standalone `NlpExplainer` stack#775
milton-minervino merged 147 commits into
developfrom
feature/nlp-refacto

Commits on Sep 4, 2026
- build: add nlp dependencies
  milton-minervino
  committed
  914646a
- feat: create nlp backend
  milton-minervino
  committed
  7704927
- feat: add nlp shap backend
  milton-minervino
  committed
  3feabf0
- feat: add nlp lime backend
  milton-minervino
  committed
  d323ae0
- docs: update docstring in shap backend
  milton-minervino
  committed
  5812343
- feat: create nlp explainer object
  milton-minervino
  committed
  fd87bd0
- feat: add new plot functions for nlp web app
  milton-minervino
  committed
  14177ec
- feat: create new dash webapp for nlp
  milton-minervino
  committed
  320961c
- chore: add new objects in init
  milton-minervino
  committed
  ce1912b
- test: add test for the nlp part
  milton-minervino
  committed
  a252d90
- chore: add script to launch and test nlp webapp
  milton-minervino
  committed
  42da306
- fix: fix version for datasets
  milton-minervino
  committed
  3bc694d
- feat: add class probability to text samples table
  milton-minervino
  committed
  a057f68
- build: add dash-ag-grid dependency
  milton-minervino
  committed
  a010aea
- feat: replace dash table with ag grid
  milton-minervino
  committed
  29ae2a2
- feat: allow filtering text samples when clicking on a word in global importance
  milton-minervino
  committed
  8a07dbf
- feat: change webapp layout
  milton-minervino
  committed
  e15c533
- test: update nlp tests
  milton-minervino
  committed
  f646f07
- chore: move nlp tests into unit tests folder
  milton-minervino
  committed
  8fd3a68
- chore: move nlp prototype to demo
  milton-minervino
  committed
  8bd6ee0
- build: build nlp webapp with docker compose
  milton-minervino
  committed
  63b5cd2
- build: add serve nlp script
  milton-minervino
  committed
  7f8f83b
- refactor: adapt the webapp
  milton-minervino
  committed
  3a51516
- fix: fix dockerfile with setuptools-scm
  milton-minervino
  committed
  62aab07
- chore: change to dev version since any placeholder works
  milton-minervino
  committed
  44d4cbd
- feat: add filter to display misclassification errors
  milton-minervino
  committed
  9db5e85
- docs: simplify run steps in readme
  milton-minervino
  committed
  f860b08
- feat: add contributions of a selected word in scatter plot
  milton-minervino
  committed
  f3c114b
- style: modify buttons
  milton-minervino
  committed
  3bf7dbb
- feat: add counterfactual generators modules
  milton-minervino
  committed
  d92c342
- feat: adapt explainer with interactive model
  milton-minervino
  committed
  9fc2aa2
- feat: add new webapp components
  milton-minervino
  committed
  0e64f2b
- feat: add readonly nlp view
  milton-minervino
  committed
  7dc007a
- feat: add model abstraction layer
  milton-minervino
  committed
  e5c1140
- feat: add serving for the extended webapp
  milton-minervino
  committed
  06973aa
- build: create docker for serving the new webapp
  milton-minervino
  committed
  d7d8ce7
- feat: create new version of the webapp
  milton-minervino
  committed
  1096228
- test: add tests for new functionalities
  milton-minervino
  committed
  ce9d26e
- feat: move model error switch and highlight errors in embeddings scatterplot
  milton-minervino
  committed
  bf425ff
- refactor: add serve config to make script more customizable
  milton-minervino
  committed
  67ea30f
- chore: put device and batch size inside model pipeline
  milton-minervino
  committed
  3a30f90
- feat: add plot confusion matrix
  milton-minervino
  committed
  2e19416
- feat: add error analysis tab and review filtering layers
  milton-minervino
  committed
  95717ed
- test: add tests for recent changes
  milton-minervino
  committed
  242fe27
- chore: add nlp cache for tests
  milton-minervino
  committed
  ae6e53b
- chore: minor update on serving
  milton-minervino
  committed
  6465ab7
- build: add captum
  milton-minervino
  committed
  fa08b5d
- feat: add captum lig backend
  milton-minervino
  committed
  5bae8b9
- feat: add captum ablation flip cf method
  milton-minervino
  committed
  f360377
- refactor: add support gradients for tokens view
  milton-minervino
  committed
  11ef5df
- feat: add captum capabilities to model
  milton-minervino
  committed
  7bcbdaa
- refactor: add ablation to explainer
  milton-minervino
  committed
  8859b06
- test: add tests of new functionalities
  milton-minervino
  committed
  43ae576
- refactor: update serving script with new captum methods
  milton-minervino
  committed
  295adbb
- refactor: allow both counterfactual methods in webapp
  milton-minervino
  committed
  66e613a
- test: update tests
  milton-minervino
  committed
  42f01bc
- refactor: update main serving script
  milton-minervino
  committed
  c24bee3
- feat: integrate two different global and local class selector
  milton-minervino
  committed
  6d79196
- test: update tests
  milton-minervino
  committed
  8b44159
- fix: aggregate token contributions to word contribution in shap backend
  milton-minervino
  committed
  46ee78b
- test: add test to shap backend
  milton-minervino
  committed
  8e16a9f
- fix: validate label_names against pipeline output in HFPipelineModel
  milton-minervino
  committed
  f2db5eb
- test: add unit test for hf model
  milton-minervino
  committed
  041711c
- refactor(backend): extract modality-agnostic Backend ABC
  milton-minervino
  committed
  ab3f45a
- refactor(explainer): restructure results with dataclasses
  milton-minervino
  committed
  b11d697
- chore: clean artifacts
  milton-minervino
  committed
  6e38bd4
- refactor(webapp): route NlpWebApp reads through NlpView, drop raw explainer handle
  milton-minervino
  committed
  307dd4e
- refactor(webapp): extract Sentence Highlight + Waterfall as WebappComponents
  milton-minervino
  committed
  2c69312
- feat: add similar examples
  milton-minervino
  committed
  a817f5c
- feat: add compatibility for new models with different tokenizer and special tokens
  milton-minervino
  committed
  92410e2
- refactor: update main serving script
  milton-minervino
  committed
  f0de217
- build: add sentencepiece in nlp dependencies
  milton-minervino
  committed
  35174f8
- test: fix test whatif with new changes
  milton-minervino
  committed
  63a47ed
- feat: upgrade TextModel protocol
  milton-minervino
  committed
  250b97b
- feat(model): extend to encoder models with classifier heads
  milton-minervino
  committed
  a097f98
- feat(explainer): add embedding store and compute projection method
  milton-minervino
  committed
  59a64ab
- chore: add masker in shap backend
  milton-minervino
  committed
  038dbff
- test: add tests covering new functionalities
  milton-minervino
  committed
  9000fbb
- refactor: update main serving script
  milton-minervino
  committed
  e66f7f2
- fix(model): raise a named error when a hooked embedding space never runs
  milton-minervino
  committed
  d99980d
- fix(model): pin fused encoder backbones to inference mode
  milton-minervino
  committed
  46c67d9
- fix(generators): detect the tokenizer's word-start marker
  milton-minervino
  committed
  af6405d
- fix(explainer): include model and backend in the compile cache key
  milton-minervino
  committed
  a554f8b
- test: add tests to cover recent fixes
  milton-minervino
  committed
  5bef295
- perf(generators): batch counterfactual search, drop captum from ablation
  milton-minervino
  committed
  09811ae
- chore(demo): consolidate NLP demo into a single serve_nlp.py flow
  milton-minervino
  committed
  20b2bb0
- build: add protobuf dependency
  milton-minervino
  committed
  c451840
- fix(demo): add --lig-batch-size to avoid CUDA OOM on LIG attribution
  milton-minervino
  committed
  540d453
- fix(model): guard against a truncation no-op aborting the CUDA context
  milton-minervino
  committed
  7e3f407
- feat(model): serve local checkpoints via HFClassifierModel.from_pretrained
  milton-minervino
  committed
  f8081b6
- fix(types): resolve mypy errors
  milton-minervino
  committed
  835527f
- fix(nlp): segment text contributions at word boundaries, not whitespace
  milton-minervino
  committed
  dbaaa8f
- feat(diagnostics): detect label noise by confident learning
  
  milton-minervino
  and
  claude
  committed
  b6ee040
- feat(retrieval): add batched query_many with exclude_self
  
  milton-minervino
  and
  claude
  committed
  a29938c
- fix: add batch size to HF pipeline
  milton-minervino
  committed
  23e651a
- refactor: replace compile with fit/explain
  milton-minervino
  committed
  4042e17
- test: update tests after refactoring
  milton-minervino
  committed
  6de2ba4
- feat(webapp): mount the NLP webapp under a URL prefix
  milton-minervino
  committed
  86cbb73
- feat(app): add word profile tab and more filtering on word importance
  milton-minervino
  committed
  7e88494
Commits on Sep 8, 2026
- refactor(webapp): rewrite nlp app layout
  milton-minervino
  committed
  2be411a
Commits on Sep 9, 2026
- refactor: change representation method and documentation
  milton-minervino
  committed
  c2bd8c5
- refactor: add kill app
  milton-minervino
  committed
  153dfc9
- refactor: modify sentence plot and confusion matrix
  milton-minervino
  committed
  2ee4ee4
- test: add nlp webapp run test
  milton-minervino
  committed
  9c9edc7
- refactor: extract scatter plot
  milton-minervino
  committed
  8e081d5
Commits on Sep 10, 2026
Commits on Sep 11, 2026
- doc(notebook): fix save and load, change projection
  milton-minervino
  committed
  2c51801
Commits on Sep 14, 2026
- refactor(nlp): split projection and embeddings into a dedicated Embedding object
  milton-minervino
  committed
  45c6328
- doc(nlp): add technical readme for the nlp part
  milton-minervino
  committed
  62ed161
Commits on Sep 16, 2026
Commits on Sep 17, 2026
- refactor: move tests
  milton-minervino
  committed
  3bb0af8
Commits on Sep 18, 2026
- refactor: change confusion matrix color palette
  milton-minervino
  committed
  00ebfcb
Commits on Sep 21, 2026
Commits on Sep 24, 2026
- docs: update readme with nlp preview
  milton-minervino
  committed
  2d6528b
- chore: align precommit blackendocs with ruff (120 line length)
  milton-minervino
  committed
  1d6c73f
Commits on Sep 30, 2026
Commits on Oct 1, 2026
Commits on Oct 5, 2026
- feat(nlp): add corpus-level backend agreement
  milton-minervino
  committed
  74c4754
- docs: add compare backends notebook
  milton-minervino
  committed
  90dcf0b
Commits on Oct 7, 2026
- feat(nlp): record character spans for every explained unit
  
  milton-minervino
  and
  claude
  committed
  41ae4db
- refactor(nlp): align backend comparison on character spans
  
  milton-minervino
  and
  claude
  committed
  f901185
- test(nlp): check unit alignment across tokenizer families
  
  milton-minervino
  and
  claude
  committed
  4780191
- docs(nlp): drop references to internal design docs from code
  
  milton-minervino
  and
  claude
  committed
  092751c
- feat(nlp): mask token ids instead of strings in SHAP
  milton-minervino
  committed
  7e2d5ca
- docs(demo): describe token-id SHAP masking in serve_nlp
  milton-minervino
  committed
  e868f42
- fix(nlp): restore the SHAP progress bar on the token-id path
  milton-minervino
  committed
  455a224
Commits on Oct 12, 2026
- Merge branch 'develop' into feature/nlp-refacto
  milton-minervino
  authored
  27e8269
- fix: fix mypy errors
  milton-minervino
  committed
  f3b1e20

Commits

Commits on Sep 4, 2026

Commits on Sep 8, 2026

Commits on Sep 9, 2026

Commits on Sep 10, 2026

Commits on Sep 11, 2026

Commits on Sep 14, 2026

Commits on Sep 16, 2026

Commits on Sep 17, 2026

Commits on Sep 18, 2026

Commits on Sep 21, 2026

Commits on Sep 24, 2026

Commits on Sep 30, 2026