Skip to content

feat(typing): Introduce PluginName NewType so plugin backends can type check #3794

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
FBruzzesi merged 4 commits into from
Jul 17, 2026
Merged

feat(typing): Introduce PluginName NewType so plugin backends can type check #3794

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
cc139e7
feat(typing): introduce `PluginName` NewType so plugin backends can t…
FBruzzesi Jul 14, 2026
d142211
keeps using generic
FBruzzesi Jul 15, 2026
a16cbb2
merge main
FBruzzesi Jul 16, 2026
782ca87
Feedback adjustments
FBruzzesi Jul 16, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 20 additions & 2 deletions src/narwhals/_typing.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
from __future__ import annotations

from types import ModuleType
from typing import TYPE_CHECKING, Literal
from typing import TYPE_CHECKING, Literal, NewType

from narwhals._typing_compat import TypeVar
from narwhals._utils import Implementation, _NoDefault
Expand Down Expand Up @@ -91,7 +91,25 @@
- An Implementation, such as: `Implementation.DASK`, `Implementation.PYSPARK`, ...
"""

BackendT = TypeVar("BackendT", bound=Backend)
PluginName = NewType("PluginName", str)
"""Name of a plugin backend's [entry point](https://packaging.python.org/en/latest/specifications/entry-points/).
Comment thread
dangotbanned marked this conversation as resolved.
Outdated

Plugin backends are discovered at runtime, so their names cannot join the
`Literal` unions that describe the built-in backends. `PluginName` is a
[`NewType`](https://typing.python.org/en/latest/spec/aliases.html#newtype):
type checkers treat it as a distinct subtype of `str`, meaning
Comment thread
FBruzzesi marked this conversation as resolved.
Outdated

- an arbitrary `str` is still rejected where a `backend` is expected, and
- a value explicitly wrapped as `PluginName("...")` is accepted.

The contract is that a wrapped string **must** name an installed plugin's
entry point in the `narwhals.plugins` group.
Comment thread
dangotbanned marked this conversation as resolved.
Outdated

Comment thread
dangotbanned marked this conversation as resolved.
Signatures that dispatch to plugins include it in the `IntoBackend`
parameter, e.g. `IntoBackend[EagerAllowed | PluginName]`.
"""

BackendT = TypeVar("BackendT", bound=Backend | PluginName)

@dangotbanned dangotbanned Jul 16, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for this!

Much better idea than I had, while getting to the same goal 🥳

Edit: woops that was meant to start the review

IntoBackend: TypeAlias = BackendT | ModuleType
"""Anything that can be converted into a [`narwhals.Implementation`][].

Expand Down
5 changes: 2 additions & 3 deletions src/narwhals/_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,7 @@
from narwhals._typing import (
Backend,
IntoBackend,
PluginName,
_ArrowImpl,
_CuDFImpl,
_DaskImpl,
Expand Down Expand Up @@ -140,8 +141,6 @@
_SliceNone,
)

UnknownBackendName: TypeAlias = str
Comment thread
FBruzzesi marked this conversation as resolved.

FrameOrSeriesT = TypeVar(
"FrameOrSeriesT", bound=LazyFrame[Any] | DataFrame[Any] | Series[Any]
)
Expand Down Expand Up @@ -395,7 +394,7 @@ def from_string(cls: type[Self], backend_name: str) -> Implementation:

@classmethod
def from_backend(
cls: type[Self], backend: IntoBackend[Backend] | UnknownBackendName
cls: type[Self], backend: IntoBackend[Backend | PluginName]
) -> Implementation:
"""Instantiate from native namespace module, string, or Implementation.

Expand Down
19 changes: 14 additions & 5 deletions src/narwhals/dataframe.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -72,7 +72,13 @@
from narwhals._compliant.typing import CompliantExprAny
from narwhals._expression_parsing import ExprMetadata
from narwhals._translate import IntoArrowTable
from narwhals._typing import EagerAllowed, IntoBackend, LazyAllowed, Polars
from narwhals._typing import (
EagerAllowed,
IntoBackend,
LazyAllowed,
PluginName,
Polars,
)
from narwhals.group_by import GroupBy, LazyGroupBy
from narwhals.typing import (
AsofJoinStrategy,
Expand Down Expand Up @@ -502,7 +508,10 @@ def __init__(self, df: Any, *, level: Literal["full", "lazy", "interchange"]) ->

@classmethod
def from_arrow(
cls, native_frame: IntoArrowTable, *, backend: IntoBackend[EagerAllowed]
cls,
native_frame: IntoArrowTable,
*,
backend: IntoBackend[EagerAllowed | PluginName],
) -> DataFrame[Any]:
"""Construct a DataFrame from an object which supports the PyCapsule Interface.

Expand Down Expand Up @@ -559,7 +568,7 @@ def from_dict(
data: Mapping[str, Any],
schema: IntoSchema | Mapping[str, IntoDType | None] | None = None,
*,
backend: IntoBackend[EagerAllowed] | None = None,
backend: IntoBackend[EagerAllowed | PluginName] | None = None,
) -> DataFrame[Any]:
"""Instantiate DataFrame from dictionary.

Expand Down Expand Up @@ -622,7 +631,7 @@ def from_dicts(
data: Sequence[Mapping[str, Any]],
schema: IntoSchema | Mapping[str, IntoDType | None] | None = None,
*,
backend: IntoBackend[EagerAllowed],
backend: IntoBackend[EagerAllowed | PluginName],
) -> DataFrame[Any]:
"""Instantiate DataFrame from a sequence of dictionaries representing rows.

Expand Down Expand Up @@ -696,7 +705,7 @@ def from_numpy(
data: _2DArray,
schema: IntoSchema | Sequence[str] | None = None,
*,
backend: IntoBackend[EagerAllowed],
backend: IntoBackend[EagerAllowed | PluginName],
) -> DataFrame[Any]:
"""Construct a DataFrame from a NumPy ndarray.

Expand Down
22 changes: 11 additions & 11 deletions src/narwhals/functions.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@

from narwhals._native import NativeDataFrame, NativeLazyFrame, NativeSeries
from narwhals._translate import IntoArrowTable
from narwhals._typing import Backend, EagerAllowed, IntoBackend
from narwhals._typing import Backend, EagerAllowed, IntoBackend, PluginName
from narwhals.dataframe import DataFrame, LazyFrame
from narwhals.series import Series
from narwhals.typing import (
Expand Down Expand Up @@ -169,7 +169,7 @@ def new_series(
values: Any,
dtype: IntoDType | None = None,
*,
backend: IntoBackend[EagerAllowed],
backend: IntoBackend[EagerAllowed | PluginName],
) -> Series[Any]:
"""Instantiate Narwhals Series from iterable (e.g. list or array).

Expand Down Expand Up @@ -211,7 +211,7 @@ def _new_series_impl(
values: Any,
dtype: IntoDType | None = None,
*,
backend: IntoBackend[EagerAllowed],
backend: IntoBackend[EagerAllowed | PluginName],
) -> Series[Any]:
implementation = Implementation.from_backend(backend)
if is_eager_allowed(implementation):
Expand Down Expand Up @@ -241,7 +241,7 @@ def from_dict(
data: Mapping[str, Any],
schema: IntoSchema | Mapping[str, IntoDType | None] | None = None,
*,
backend: IntoBackend[EagerAllowed] | None = None,
backend: IntoBackend[EagerAllowed | PluginName] | None = None,
native_namespace: ModuleType | None = None, # noqa: ARG001
) -> DataFrame[Any]:
"""Instantiate DataFrame from dictionary.
Expand Down Expand Up @@ -332,7 +332,7 @@ def from_dicts(
data: Sequence[Mapping[str, Any]],
schema: IntoSchema | Mapping[str, IntoDType | None] | None = None,
*,
backend: IntoBackend[EagerAllowed],
backend: IntoBackend[EagerAllowed | PluginName],
) -> DataFrame[Any]:
"""Instantiate DataFrame from a sequence of dictionaries representing rows.

Expand Down Expand Up @@ -393,7 +393,7 @@ def from_numpy(
data: _2DArray,
schema: IntoSchema | Sequence[str] | None = None,
*,
backend: IntoBackend[EagerAllowed],
backend: IntoBackend[EagerAllowed | PluginName],
) -> DataFrame[Any]:
"""Construct a DataFrame from a NumPy ndarray.

Expand Down Expand Up @@ -482,7 +482,7 @@ def _is_into_schema(obj: Any) -> TypeIs[_IntoSchema]:


def from_arrow(
native_frame: IntoArrowTable, *, backend: IntoBackend[EagerAllowed]
native_frame: IntoArrowTable, *, backend: IntoBackend[EagerAllowed | PluginName]
) -> DataFrame[Any]: # pragma: no cover
"""Construct a DataFrame from an object which supports the PyCapsule Interface.

Expand Down Expand Up @@ -652,7 +652,7 @@ def _validate_separator_pyarrow(separator: str, **kwargs: Any) -> Any:
def read_csv(
source: FileSource,
*,
backend: IntoBackend[EagerAllowed],
backend: IntoBackend[EagerAllowed | PluginName],
separator: str = ",",
**kwargs: Any,
) -> DataFrame[Any]:
Expand Down Expand Up @@ -727,7 +727,7 @@ def read_csv(
def scan_csv(
source: FileSource,
*,
backend: IntoBackend[Backend],
backend: IntoBackend[Backend | PluginName],
separator: str = ",",
**kwargs: Any,
) -> LazyFrame[Any]:
Expand Down Expand Up @@ -813,7 +813,7 @@ def scan_csv(


def read_parquet(
source: FileSource, *, backend: IntoBackend[EagerAllowed], **kwargs: Any
source: FileSource, *, backend: IntoBackend[EagerAllowed | PluginName], **kwargs: Any
) -> DataFrame[Any]:
"""Read into a DataFrame from a parquet file.

Expand Down Expand Up @@ -886,7 +886,7 @@ def read_parquet(


def scan_parquet(
source: FileSource, *, backend: IntoBackend[Backend], **kwargs: Any
source: FileSource, *, backend: IntoBackend[Backend | PluginName], **kwargs: Any
) -> LazyFrame[Any]:
"""Lazily read from a parquet file.

Expand Down
6 changes: 3 additions & 3 deletions src/narwhals/series.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -45,7 +45,7 @@
from typing_extensions import Self

from narwhals._compliant import CompliantSeries
from narwhals._typing import EagerAllowed, IntoBackend, NoDefault
from narwhals._typing import EagerAllowed, IntoBackend, NoDefault, PluginName
from narwhals.dataframe import DataFrame, MultiIndexSelector
from narwhals.dtypes import DType
from narwhals.typing import (
Expand Down Expand Up @@ -121,7 +121,7 @@ def from_numpy(
values: _1DArray,
dtype: IntoDType | None = None,
*,
backend: IntoBackend[EagerAllowed],
backend: IntoBackend[EagerAllowed | PluginName],
) -> Series[Any]:
"""Construct a Series from a NumPy ndarray.

Expand Down Expand Up @@ -186,7 +186,7 @@ def from_iterable(
values: Iterable[Any],
dtype: IntoDType | None = None,
*,
backend: IntoBackend[EagerAllowed],
backend: IntoBackend[EagerAllowed | PluginName],

@dangotbanned dangotbanned Jul 16, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note

Not a review, just trying to add context from our conversation on discord

If you are downstream from Narwhals and make use of our typing - we have been suggesting (#3149 (comment)) you write things like:

backend: IntoBackend[EagerAllowed]

Following this PR, that type describes exactly the same thing.
It does not indicate plugin support, meaning that there should be no expectation that downstream tests against plugins.

If downstream wants to opt-in to plugins, the update to their typing small:

- backend: IntoBackend[EagerAllowed]
+ backend: IntoBackend[EagerAllowed | PluginName]

But it gives a very clear signal (to their users) that plugins should work.
I hope that downstream (authors) will be more likely to update their tests, to account for plugins, when these concepts are visible in each signature with a backend 🙂

@FBruzzesi FBruzzesi Jul 16, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this should go somewhere! but where exactly? 🧐

@FBruzzesi FBruzzesi Jul 16, 2026

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also made me realize that one should import from two modules:

from narwhals.typing import IntoBackend, EagerAllowed
from narwhals.plugins import PluginName

...

@dangotbanned dangotbanned Jul 16, 2026

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All this should go somewhere! but where exactly? 🧐

We could do either the PR description or the release notes until we find somewhere else?
For release notes I mean if we had a short highlight message in this style (https://github.com/vega/altair/releases#release-v5.5.0)

Both can work too 😅

) -> Series[Any]:
"""Construct a Series from an iterable.

Expand Down
Loading
Loading