Skip to content

[Repo Assist] fix(dummy_outcome_refuter): correct preprocess_data_by_treatment bugs - #1720

Draft
github-actions[bot] wants to merge 1 commit into
mainfrom
repo-assist/fix-dummy-outcome-refuter-preprocess-20260729-6ab404be70f5c8ea
Draft

[Repo Assist] fix(dummy_outcome_refuter): correct preprocess_data_by_treatment bugs#1720
github-actions[bot] wants to merge 1 commit into
mainfrom
repo-assist/fix-dummy-outcome-refuter-preprocess-20260729-6ab404be70f5c8ea

Conversation

@github-actions

Copy link
Copy Markdown
Contributor

🤖 This PR was created by Repo Assist, an automated AI assistant.

Related to #1665 (pandas 3.x + scikit-learn compatibility audit).

Problem

preprocess_data_by_treatment() in dowhy/causal_refuters/dummy_outcome_refuter.py contained five bugs, some of which had been present since the function's introduction.

Bug 1 — assert instead of ValueError

assert len(treatment_name) == 1, "At present, DoWhy supports a simgle treatment variable"

Python's assert can be disabled with -O (optimised mode). Production code should use an explicit ValueError.

Bug 2 — num_bins computed from a per-column Series, not a scalar

num_bins = (data.max() - data.min()) / (bucket_size_scale_factor * std_dev)

data.max() - data.min() returns a Series (the range of all columns), not the scalar range of the treatment column. Passing a Series to pd.cut(data[t], num_bins) works coincidentally only if the column ranges happen to be monotonically increasing; otherwise pd.cut raises ValueError: bins must increase monotonically.

Fix: compute the scalar range from the treatment column only:

t_range = data[treatment_variable_name].max() - data[treatment_variable_name].min()
num_bins = max(1, int(t_range / (bucket_size_scale_factor * std_dev)))

Bug 3 — Input DataFrame mutated

The function called data["bins"] = pd.cut(...) and data.drop("bins", ..., inplace=True) on the caller's DataFrame. This is surprising behaviour and is incompatible with pandas 3 Copy-on-Write semantics, which returns a read-only view for sliced DataFrames.

Fix: call data = data.copy() at the start of the function.

Bug 4 — Dead no-op line

data = data  # no-op

Removed.

Bug 5 — Unreachable categorical branch + immediate overwrite

elif "categorical" in variable_type.name:   # BUG: "categorical" not in "category"
    groups = data.groupby(treatment_variable_name)
    groups = data.groupby("bins")            # BUG: overwrites; "bins" doesn't exist
    return groups

pandas names its categorical dtype "category", not "categorical". Since "categorical" in "category" is False, this entire branch was dead code and could never execute.

The second line also immediately overwrote the first, and the "bins" column was never created at that point.

Fix: change the guard to "categor" in variable_type.name (a substring of both "category" and "categorical"), remove the overwrite.

Additional — observed=True missing on groupby calls

All three groupby calls now include observed=True to suppress the pandas FutureWarning about the default changing.


Changes

Only two files changed:

File Change
dowhy/causal_refuters/dummy_outcome_refuter.py Fix all five bugs in preprocess_data_by_treatment
tests/causal_refuters/test_dummy_outcome_refuter.py Add 3 regression tests

Test Status

3 new regression tests — all pass:

Test Verifies
test_preprocess_data_by_treatment_multiple_treatments_raises Multiple treatment names raises ValueError (not AssertionError)
test_preprocess_data_by_treatment_continuous_treatment Continuous treatment bins correctly; input DataFrame is not mutated
test_preprocess_data_by_treatment_categorical_treatment Categorical treatment (pd.Categorical) now reachable and returns correct groups

black --check, flake8 --select=E9,F63,F7,F82: no errors.

Generated by 🤖 Repo Assist, see workflow run.

Generated by 🌈 Repo Assist, see workflow run. Learn more.

To install this agentic workflow, run

gh aw add githubnext/agentics/workflows/repo-assist.md@11c9a2c442e519ff2b427bf58679f5a525353f76

Five bugs fixed in preprocess_data_by_treatment():

1. assert -> ValueError: assert can be disabled with -O; use an
   explicit ValueError('single treatment variable') instead.

2. num_bins scalar bug: data.max() - data.min() returns a per-column
   Series, not a scalar, breaking pd.cut(). Fix: compute the range
   of the treatment column only and cast to int with a minimum of 1.

3. input mutation: data["bins"] = ... and data.drop(..., inplace=True)
   mutate the caller's DataFrame, which is surprising and incompatible
   with pandas 3 Copy-on-Write semantics. Fix: work on data.copy().

4. dead-code overwrite: in the continuous branch 'data = data' was a
   no-op; removed.

5. unreachable categorical branch: pandas names its categorical dtype
   'category', not 'categorical', so 'categorical' in 'category' is
   always False. The categorical branch was dead code and also contained
   a second bug (groups immediately overwritten by groupby('bins') which
   doesn't exist). Fix: change guard to 'categor' in variable_type.name,
   remove overwrite, add observed=True for pandas 3.x compatibility.

Also adds observed=True to the bool and categorical groupby calls to
silence the pandas FutureWarning about observed= defaulting to True.

Three new unit tests:
- test_preprocess_data_by_treatment_multiple_treatments_raises
- test_preprocess_data_by_treatment_continuous_treatment
- test_preprocess_data_by_treatment_categorical_treatment

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automation bug Something isn't working repo-assist

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants