Examples · Validation & governance

recalibrate_artifact

The zero-churn demonstration: refresh probabilities on new outcomes without moving a single applicant to a different band.

Code#

recalibrate_artifact.py

from _setup import X_test, artifact, whitebox, y_test
 
from compileml.artifact import recalibrate_artifact
 
# A later observation window: the same model, scored on newer accounts whose
# outcomes are now known. This is the case recalibration exists for.
latent_new = whitebox.predict(X_test).clip(0, 1)
refreshed = recalibrate_artifact(artifact, latent_new, y_test)
 
print("model unchanged      :", refreshed["model"] == artifact["model"])
print("band edges unchanged :", refreshed["bands"]["edges_int"] == artifact["bands"]["edges_int"])
print("calibration changed  :", refreshed["calibration"] != artifact["calibration"])
print()
print("artifact hash before :", artifact["artifact_hash"][:24])
print("artifact hash after  :", refreshed["artifact_hash"][:24])
 
print()
print("A recalibration is a new artifact — the hash moves, as it must — but no")
print("account can change band, because nothing band assignment depends on moved.")

Output#

Captured from an actual run against compileml 0.9.0 and the UCI credit panel. If this script stops working, the build fails.

captured in CIrecalibrate_artifact.py

model unchanged      : True
band edges unchanged : True
calibration changed  : True
 
artifact hash before : a31afc9bf476ce94714946a1
artifact hash after  : 0cddc3dfe3c2b8c0a2e20d6e
 
A recalibration is a new artifact — the hash moves, as it must — but no
account can change band, because nothing band assignment depends on moved.

Notes#

Only the calibration table and provenance metadata change, so the new hash documents a recalibration rather than a new model.

API reference: recalibrate_artifact →

Related notebook