Open question · OPEN-CLASSIFIER-CALIBRATION-FI
How does calibration error become information loss?
Openpublication: open
Meaning
A general propagation bound from classifier calibration error to retained Fisher information remains open.
Scientific context
There is no quantitative propagation from classifier calibration error to retained-information loss. This is missing rather than merely unproved: no theorem of this kind exists yet.
Statement
The audited CLASSIFIER-CALIBRATION-SCORE-LIPSCHITZ bridge supplies an upper score-error budget from excess Brier error. Reliability and resolution gap split that posterior error; a reverse score-error bound needs an injective chart and explicit constants. Calibration error does not certify retention distortion, and AUC does not determine proxy reporting error. Remaining: certify the resolution gap without truth scores (OPEN-REPRESENTATION-LOSS-ESTIMATION), sharper constants, and D_s efficiency.
Relationships
Related settled results
- Posterior error to score error for mixture-fraction scores: one Lipschitz inequality, and the excess Brier score as the classifier requirementCLASSIFIER-CALIBRATION-SCORE-LIPSCHITZ
- Calibrated classifier posterior odds recover component density ratiosCLASSIFIER-RATIO-ORACLE
- True retained Fisher information under a score proxyPROXY-TRUE-RETAINED-FI
The full chain beneath it
- Classifier posteriors map explicitly to mixture score coordinatesCLASSIFIER-MIXTURE-SCORE-FORMULA
- Reported versus true retention of a frozen label map under a score error: an exact two-term expansion in the Fisher-whitened score errorSCORE-ERROR-RETENTION-BUDGET
- Quantized Fisher-information identityFI-QUANT-IDENTITY
- Linear-mixture component ratios suffice for score constructionMIXTURE-RATIO-SCORE
- Representation loss and hard-quantization loss separateREPRESENTATION-QUANTIZATION-LOSS
- Normalized retained-information spectrumINFO-RETENTION-SPECTRUM
- D-efficiency relative to unbinned inferenceINFO-D-EFFICIENCY
- D-optimal partition invariance under invertible reparameterizationD-REPARAM-INVARIANCE
- Local score can be obtained from a local density ratioRATIO-LOCAL-SCORE
- Exact Fisher-information loss decompositionFI-LOSS-DECOMPOSITION
Where it comes from
Raised by
- Posterior error to score error for mixture-fraction scores: one Lipschitz inequality, and the excess Brier score as the classifier requirementCLASSIFIER-CALIBRATION-SCORE-LIPSCHITZ
- Calibrated classifier posterior odds recover component density ratiosCLASSIFIER-RATIO-ORACLE
- True retained Fisher information under a score proxyPROXY-TRUE-RETAINED-FI
Local graph
The question in full
Classifier calibration error → Fisher loss
Relate posterior/ratio calibration error to density-ratio error, score error, pre-quantization
representation loss, and final D/ efficiency. The target is a classifier-quality
requirement beyond AUC. Half settled (8 Sep 2026, O8.4, CLASSIFIER-CALIBRATION-SCORE-LIPSCHITZ):
the mixture score is Lipschitz in the posterior, so the whitened score error is bounded by the
excess Brier score, which is reliability plus
the resolution gap (needs truth). The audit requires an injective chart and explicit
constants for a reverse score-error bound; calibration error alone does not imply
retention distortion. Reliability is a population quantity estimated from labelled data,
not an exact finite-data certificate. AUC does not determine reporting error (CE-AUC-INVARIANT-PROXY-RETENTION-001).
Remaining: certifying the resolution gap without truth scores (with OPEN-REPRESENTATION-LOSS-ESTIMATION). Target claim:
OPEN-CLASSIFIER-CALIBRATION-FI.
From OPEN_PROBLEMS.md