ProbableOdyssey

TIL: How to Address Stray Pandas `SettingWithCopy` Warnings

If you’ve used Pandas, you’ve probably seen:

SettingWithCopyWarning:
A value is trying to be set on a copy of a slice from a DataFrame

It means Pandas isn’t sure if you’re editing the original DataFrame or a temporary view. Sometimes the change sticks, sometimes it doesn’t — which is a recipe for silent bugs.

To fix it:

  1. Find the warnings

Run scripts/tests and force warnings to be raised as errors:

# If using pytest, run your tests
pytest -W error::pandas.errors.SettingWithCopyWarning
# Otherwise, run your script and make all warnings raise errors:
python -W always your_script.py 2>&1 | grep SettingWithCopy

If you’re in a notebook environment, we can force all these warning to be errors, to find where they are and address them. At the top of the notebook:

import warnings
import pandas as pd
warnings.simplefilter("error", category=pd.errors.SettingWithCopyWarning)
  1. Inspect each case

Look for slices like df[df["age"] > 30] or df.loc[df["y"] == "a"] followed by mutations.

Decide whether you want a copy or intend to mutate the original.

  1. Be explicit

Always make your intent clear with .copy() or .loc. The warning goes away, and your code becomes predictable.

Long term fix: Polars instead of Pandas

polars sidesteps this entire issue. Every operation is explicit: you either create a new DataFrame or you don’t. No ambiguous views, no surprise warnings, and you also get fast performance out of the box.

Here’s the same operation in Polars:

import polars as pl

df = pl.DataFrame({"age": [25, 35, 50]})
df2 = df.filter(pl.col("age") > 30).with_columns(
    pl.lit("senior").alias("category")
)
print(df2)

No warnings, no ambiguity — just the damn result you asked for.

Reply to this post by email ↪