band_drift
from compileml.monitor import band_drift
Observed bad rate against the emitted PD, per logged band.
Rows are grouped by the band each decision logged, never by a band rebuilt from scores, so a row on a boundary is counted where production put it. Every band in the artifact is listed, including empty ones.
outside_interval is whether the band's mean emitted PD falls outside
the Wilson interval on its observed bad rate — None below min_n,
where a flag would mean little. At 95% intervals a perfectly calibrated
artifact still flags about one band in twenty; with ten bands, expect one
flag every two periods on average.
Outcomes must have matured: last month's decisions have not had time to default, and will look better calibrated than they are.
Nothing is recalibrated. The report is evidence for an explicit call to
recalibrate_artifact.
Parameters#
| Name | Type | Default | Kind |
|---|---|---|---|
| artifact | — | required | positional |
| decisions | — | required | positional |
| y | — | required | positional |
| min_n | int | 30 | keyword-only |
| z | float | 1.96 | keyword-only |
Returns#
dict
Raises#
- ValueError
Read from the function body and the private helpers it calls, not inferred. A function that raises nothing has no section here.