Examples · Tuning

sweep_bands

Sweeping band counts and builders against monotonicity and churn at once.

Code#

sweep_bands.py

from _setup import latent, y_train
 
from compileml.tune import sweep_bands
 
rows = sweep_bands(latent, y_train, k_grid=(4, 6, 8, 10, 12, 16))
 
print(f"{"bands":>6}{"band gini":>12}{"retained":>10}{"gap":>9}{"worst AUC":>11}  verdict")
for row in rows:
    print(
        f"{row["n_bands"]:>6}{row["band_ordinal_gini"]:>12.4f}"
        f"{row["gini_retention_pct"]:>9.1f}%{row["gini_gap"]:>9.4f}"
        f"{row["worst_within_band_auc"]:>11.3f}  {row["worst_band_verdict"]}"
    )
 
print()
print("Each extra band recovers some of the gap the ladder introduces.")
print("The verdict says whether the worst band still has ordering left in it.")

Output#

Captured from an actual run against compileml 0.9.0 and the UCI credit panel. If this script stops working, the build fails.

captured in CIsweep_bands.py

 bands   band gini  retained      gap  worst AUC  verdict
     4      0.5098     90.7%   0.0523      0.705  refinable
     6      0.5403     96.1%   0.0217      0.661  refinable
     8      0.5518     98.2%   0.0103      0.613  refinable
    10      0.5535     98.5%   0.0085      0.589  refinable
    12      0.5572     99.2%   0.0048      0.603  refinable
    16      0.5582     99.3%   0.0038      0.588  refinable
 
Each extra band recovers some of the gap the ladder introduces.
The verdict says whether the worst band still has ordering left in it.

API reference: sweep_bands →

Related notebook