Examples · Validation & governance

validate_artifact

All eleven checks against a holdout, then a deliberately corrupted artifact so you can see a failure report.

Code#

validate_artifact.py

from _setup import ARTIFACT_PATH, X_test, X_train, latent, y_test, y_train
 
from compileml.reference import fit_reference
from compileml.validate import validate_artifact
 
reference = fit_reference(X_train, y_train)
 
report = validate_artifact(
    ARTIFACT_PATH,
    X_test,
    y_test,
    latent_train=latent,
    reference=reference,
)
 
print("all checks pass:", report["all_pass"])
print()
for name, check in report["checks"].items():
    state = "skip" if check.get("skipped") else ("pass" if check["pass"] else "FAIL")
    print(f"  {state:<5} {name}")

Output#

Captured from an actual run against compileml 0.9.0 and the UCI credit panel. If this script stops working, the build fails.

captured in CIvalidate_artifact.py

all checks pass: True
 
  pass  1_integrity
  pass  2_reconciliation
  skip  3_fidelity
  pass  4_band_properties
  pass  5_semantic_monotonicity
  pass  6_churn_baseline
  pass  7_explainability_stability
  pass  8_reason_coverage
  skip  9_monotone_constraints
  pass  10_reference_floor
  skip  11_selection_hygiene

Notes#

Checks that lack the inputs they need report skipped: true rather than passing silently.

API reference: validate_artifact →

Related notebook