API Reference¶

@df_in¶

Validates DataFrame parameters passed to a function.

@df_in(
    name: str | None = None,
    columns: list[str] | dict[str, str | dict] | None = None,
    strict: bool | None = None,
    lazy: bool | None = None,
    composite_unique: list[list[str]] | None = None,
    row_validator: type[BaseModel] | None = None,
    min_rows: int | None = None,
    max_rows: int | None = None,
    exact_rows: int | None = None,
    allow_empty: bool | None = None
)

Parameters:

Examples:

# Column list
@df_in(["a", "b", "c"])

# With dtypes
@df_in({"a": "int64", "b": "object"})

# With constraints
@df_in({"price": {"dtype": "float64", "nullable": False, "checks": {"gt": 0}}})

# Multiple DataFrames — use name to target specific parameters
@df_in(name="orders", columns=["id", "total"])
@df_in(name="customers", columns=["id", "name"])

# Row validation
@df_in(row_validator=OrderModel)

# Shape/lazy controls
@df_in({"email": {"nullable": False}}, lazy=True, min_rows=1, allow_empty=False)

@df_out¶

Validates the DataFrame returned by a function.

@df_out(
    columns: list[str] | dict[str, str | dict] | None = None,
    strict: bool | None = None,
    lazy: bool | None = None,
    composite_unique: list[list[str]] | None = None,
    row_validator: type[BaseModel] | None = None,
    min_rows: int | None = None,
    max_rows: int | None = None,
    exact_rows: int | None = None,
    allow_empty: bool | None = None
)

Parameters:

Examples:

@df_out(["result", "score"])

@df_out({"score": "float64"}, strict=True)

@df_out(row_validator=ResultModel)

@df_out(min_rows=10, max_rows=100, lazy=True)

@df_log¶

Logs DataFrame structure when entering and exiting a function.

@df_log(
    level: int = logging.DEBUG,
    include_dtypes: bool = False
)

Parameters:

Examples:

@df_log(level=logging.INFO)
def process(df):
    return df
# Logs: Function process parameters contained a DataFrame: columns: ['a', 'b']
# Logs: Function process returned a DataFrame: columns: ['a', 'b']

@df_log(include_dtypes=True)
def process(df):
    return df
# Logs: ... columns: ['a', 'b'] with dtypes [Int64, String]

Column Specifications¶

Columns can be specified in several formats:

List Format¶

Simple list of required column names:

columns=["a", "b", "c"]

Dict with Dtypes¶

Map column names to expected dtypes:

columns={"a": "int64", "b": "object", "c": "float64"}

Parameterised dtypes can be declared by base name, which matches whatever parameters the column carries. Spell the parameters out when you want to constrain them:

columns={"created_at": "datetime"}                                  # any time unit or time zone
columns={"created_at": "datetime(time_unit='ns', time_zone=none)"}  # exactly this one
columns={"tags": "list", "address": "struct", "grade": "enum"}

Note that "object" and "str" both mean "string" — a Pandas object column is reported as a string regardless of what it holds, so {"a": "str"} will not catch an object column containing numbers or dicts.

Rich Column Spec¶

Full control over column validation:

columns={
    "column_name": {
        "dtype": "float64",      # Expected dtype (optional)
        "nullable": False,       # Allow null values? Default: True
        "unique": True,          # Require unique values? Default: False
        "required": True,        # Is column required? Default: True
        "checks": {              # Value checks (optional)
            "gt": 0,
            "lt": 100
        }
    }
}

Those five are the only valid constraint names. A misspelling such as {"nullabel": False} used to be accepted and then do nothing; it now raises ValueError when the decorator is applied. nullable, unique and required must be True or False — "False" is a string, and is rejected rather than ignored.

For editor and type-checker support, annotate specs with ColumnConstraints, which catches the same typos statically:

from daffy import ColumnConstraints

SPEC: dict[str, ColumnConstraints] = {"price": {"nullable": False, "checks": {"gt": 0}}}

@df_in(SPEC)
def process_data(df):
    ...

Regex Patterns¶

Match multiple columns with regex:

columns=["id", "r/feature_\\d+/"]  # Matches feature_1, feature_2, etc.

columns={"r/score_\\d+/": {"dtype": "float64", "checks": {"between": (0, 100)}}}

Value Checks¶

Available built-in checks for the checks parameter:

Custom checks are also supported by passing a callable as the check value. A callable may use any name; a non-callable value must name one of the built-in checks above, and a name that matches neither is rejected when the decorator is applied rather than on the first call.

A check is also rejected if it cannot mean anything on the column's dtype — an ordering check (gt, ge, lt, le, between) on a string column, or a str_* check on a non-string column. Backends disagreed about these: some raised their own error while Polars compared strings lexicographically and reported a violation that looked real.

str_regex applies your pattern as written: it matches anywhere in the value unless you anchor it yourself. Use r"^\d+" to require a match at the start and r"^\d+$" to require a full match.

The pattern is handed to the DataFrame library's own regex engine, so advanced syntax is not portable. Lookarounds and backreferences work on Pandas (Python re) but raise ComputeError on Polars and ArrowInvalid on PyArrow. Stick to basic syntax if your code runs on more than one backend.

Checks constrain values, and a null is not a value. Daffy does not rewrite null comparison results, so null handling follows your backend: on Polars, PyArrow and Pandas nullable dtypes a comparison against null is "unknown" and is not reported, while Pandas float NaN compares as False and therefore does fail. Use nullable=False or the notnull check to constrain nulls explicitly — those work identically everywhere.

Built-in string checks ignore null values on every backend, including Pandas object columns.

Examples:

columns={
    "price": {"checks": {"gt": 0, "lt": 10000}},
    "score": {"checks": {"between": (0, 100)}},
    "status": {"checks": {"isin": ["active", "pending"]}},
    "email": {"checks": {"str_regex": r"^[^@]+@[^@]+\.[^@]+$"}}
}

Configuration¶

Configure Daffy in pyproject.toml:

[tool.daffy]
strict = false                 # Reject unexpected columns (default: false)
lazy = false                   # Collect all errors before raising (default: false)
allow_empty = true             # Allow DataFrames with no rows (default: true)
strict_specs = false           # Raise TypeError on invalid column keys or spec types (default: false)
row_validation_max_errors = 5  # Max failed rows shown in row validation (default: 5)
checks_max_samples = 5         # Max sample values in check errors (default: 5)

Configuration is read from the pyproject.toml in the current working directory or any parent directory. Discovery starts at the process working directory, not at your module, so running the same code from a different directory can pick up a different configuration — worth knowing in a monorepo, or when a service runs with a WORKDIR that differs from your development checkout.