Expected Calibration Error (ECE)
Core Metrics DS practice problem on Onlearn.
Difficulty: medium.
Topics: Understanding Model Calibration, Binning Strategy, Confidence Score, Accuracy-Confidence Gap, Weighted Average Calculation, Vectorized Probability Processing, Probability Theory, Statistics, Machine Learning Evaluation, Model Interpretability, Uncertainty Estimation, Calibration Curves, Reliability Diagrams, Frequentist Statistics, Overconfidence Bias, Classification Metrics.
Implement a function to calculate the Expected Calibration Error (ECE) for a multi class classification problem. Given a list of predicted probabilities and the true labels, divide the probabilities into 10 equal bins (0.0 to 1.0) and calculate the weighted average of the absolute difference between accuracy and confidence for each bin.