feat: Add empty_as_null and keep_nulls options to explode - #3709
feat: Add empty_as_null and keep_nulls options to explode#3709FBruzzesi wants to merge 11 commits into
empty_as_null and keep_nulls options to explode#3709Conversation
|
CI failures are related to a change in polars error message for |
|
Regarding the pyspark-connect, I guess we can lower coverage to 90%? (it's 94 now, target is 95) |
MarcoGorelli
left a comment
There was a problem hiding this comment.
thanks @FBruzzesi !
Regarding the pyspark-connect, I guess we can lower coverage to 90%? (it's 94 now, target is 95)
yup sure
|
I ported a bunch of the tests from |
|
I've marked this as high priority (#3709 (comment)) because of the CI fix. I obviously agree with adding the flags 😅 (#3347). I haven't reviewed the changes in each backend, but if @MarcoGorelli has and is happy - then maybe we just merge? |
|
that's a good point thanks! sure, let's do that, that'll make this easier to revert if necessary |
Description
In light of polars deprecation warnings showing up in our CI, this PR expands the
explodesignature to match polars' one. Adapted to all backends and backported for polars (safe for 1 column explode, let polars do polars internal on multiple columns).Test cases were Claude assisted, trying to cover all four flag combos for single and multi columns including null-vs-empty same-index case, and shape-mismatch errors.
Merging this PR would make a lot of the jobs in #3711 happy
What type of PR is this? (check all applicable)