Skip to content

refactor(nlp): give NLP backends their model at fit() instead of construction - #834

Open
milton-minervino wants to merge 5 commits into
MAIF:developfrom
milton-minervino:refactor/nlp-backend-fit
Open

milton-minervino wants to merge 5 commits into
MAIF:developfrom
milton-minervino:refactor/nlp-backend-fit

Conversation

@milton-minervino

Copy link
Copy Markdown
Collaborator

Summary

NLP backends now hold settings only and receive the model at fit(), so you no longer pass the model twice. NlpExplainer owns the model and fits the backend.

Before

backend = NlpShapBackend(model, label_names=labels, output_space="logit")
xpl = NlpExplainer(model, label_names=labels, backend=backend)

After

xpl = NlpExplainer(model, label_names=labels, backend=NlpShapBackend(output_space="logit"))

Changes

  • NlpShapBackend, NlpLimeBackend and NlpCaptumLigBackend are scikit-learn BaseEstimators. Constructors store their arguments unchanged; fit(model, label_names=None) builds the explainer and returns self.
  • Attributes set by fit end in _ (model_, label_names_, explainer_, masker_, masking_, baseline_token_, …). Settings keep plain names.
  • NlpBackend.run_explainer checks that the backend is fitted (NotFittedError otherwise), then calls the subclass's _run_explainer.
  • NlpExplainer fits a fresh copy, type(backend)(**backend.get_params(deep=False)). It doesn't use sklearn.clone, which would deep-copy maskers and tokenizers.
  • The explanation cache key is the backend name plus get_params(), with nested dicts sorted. Before, the key listed a few attributes by hand, so a setting outside that list (for example NlpLimeBackend(mask_string=...)) didn't change it and could reload a stale cached explanation. Display-only settings (display_params, currently show_progress) are left out of the key, so turning on a progress bar doesn't invalidate the cache.
  • NlpLimeBackend adds num_features="all" in fit and leaves the caller's explainer_compute_args dict untouched.
  • Tutorial (compare_backends.ipynb), demo (serve_nlp.py) and wiki (NLP-Explainability.md) are updated to the new constructors.

Breaking changes

The NLP path hasn't been released yet, so this only affects code written against develop.

  • Backend constructors no longer take model, label_names or preprocessing (it was ignored). NlpCaptumLigBackend also loses its unused explainer_args.
  • NlpExplainer no longer takes explainer_args / explainer_compute_args. Pass them to the backend instead: NlpShapBackend(explainer_args=...).
  • get_local_contributions is gone from NlpBackend and from the shared Backend contract. The tabular BaseBackend keeps its own, and SmartExplainer is unaffected.
  • xpl.backend is now a fitted copy, not the instance you passed in. The instance you pass in is never modified, so you can reuse one settings object across several explainers. Code that inspected its own backend object after construction should read xpl.backend instead.

Testing

  • Unit and integration tests pass (2070), including the real-model integration tests run offline against the local Hugging Face cache.
  • New tests cover: the caller's instance stays unfitted, every setting changes the cache key, nested settings are order-insensitive in the key, display settings don't change the key, and run_explainer before fit raises.
  • ruff check is clean. Coverage on the changed modules is 92–99%.
  • The compare_backends notebook was edited and re-run.

…nd contract

The text backends never use it, and the tabular BaseBackend keeps its own implementation. The shared contract is now run_explainer alone.
…truction

A backend now holds settings only. NlpShapBackend, NlpLimeBackend and NlpCaptumLigBackend are scikit-learn estimators: the constructor stores its arguments unchanged, and fit(model, label_names=None) builds the explainer. NlpExplainer owns the model and binds a fresh copy of the backend it is given, built from get_params() rather than sklearn's clone, which would deep-copy maskers and tokenizers. The model is no longer passed twice, and the caller's instance is never modified and can be reused.

- Values fit() derives end in "_" (model_, explainer_, masking_, baseline_token_). Reading one before fit raises.
- NlpBackend.run_explainer checks the backend is fitted, then calls the  subclass's _run_explainer.
- The cache key is the backend name plus get_params(), with nested dicts sorted. Display-only settings (display_params: show_progress) are left out.
- Removed: the ignored preprocessing parameter, label_names from the constructors (it now arrives through fit), the unused explainer_args of NlpCaptumLigBackend, NlpBackend.get_local_contributions, and the explainer_args / explainer_compute_args parameters of NlpExplainer.
- NlpLimeBackend applies num_features="all" in fit, so the caller's explainer_compute_args stay as given.

xpl.backend is now a fitted copy, not the instance passed in.
@milton-minervino milton-minervino changed the title Refactor/nlp backend fit refactor(nlp): give NLP backends their model at fit() instead of construction Nov 10, 2026
@codecov

codecov Bot commented Nov 10, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 95.14563% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.40%. Comparing base (cda4d07) to head (9fae56d).

Files with missing lines Patch % Lines
shapash/backend/nlp_lime_backend.py 88.88% 2 Missing and 1 partial ⚠️
shapash/backend/nlp_shap_backend.py 94.59% 1 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop     #834      +/-   ##
===========================================
+ Coverage    86.37%   86.40%   +0.03%     
===========================================
  Files          115      115              
  Lines        11958    11978      +20     
  Branches      2176     2175       -1     
===========================================
+ Hits         10329    10350      +21     
+ Misses         996      995       -1     
  Partials       633      633              
Files with missing lines Coverage Δ
shapash/backend/backend.py 87.50% <ø> (+5.68%) ⬆️
shapash/backend/nlp_backend.py 96.00% <100.00%> (+0.25%) ⬆️
shapash/backend/nlp_captum_lig_backend.py 95.32% <100.00%> (+0.13%) ⬆️
shapash/explainer/nlp_explainer.py 89.20% <100.00%> (+0.07%) ⬆️
shapash/backend/nlp_shap_backend.py 98.92% <94.59%> (+0.04%) ⬆️
shapash/backend/nlp_lime_backend.py 89.47% <88.88%> (+0.46%) ⬆️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@milton-minervino
milton-minervino marked this pull request as ready for review November 16, 2026 10:12

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant