ScoreQuant

Result · REPRESENTATION-QUANTIZATION-LOSS

Information loss splits into representation and binning parts

Derived from the literaturepublication: derived from published

Read the proof and evidence →

Meaning

Conditional projection decomposes information lost in the representation and in its subsequent hard labels.

Statement

For representation R(X), I_R=Var(E[s|R]) and I_q<=I_R<=I_full, so I_full-I_R is representation/oracle loss and I_R-I_q is hard-quantization loss when truth scores are available.

Relationships

Rests on

The full chain beneath it

Enables

Raises

Open next door

Local graph
Local map of Representation loss and hard-quantization loss separate1 prerequisites to the left, 3 results it enables or questions it raises to the right, 0 boundary cases below.Exact Fisher-information loss decompositionExact Fisher-informat…Estimate representation versus quantization lossEstimate representati…Perturbation theory for estimated scoresPerturbation theory f…Reported versus true retention of a frozen label map under a score error: an exact two-term expansion in the Fisher-whitened score errorReported versus true…Representation loss and hard-quantization loss separateRepresentation loss a…
Prerequisites to the left, what it enables or raises to the right, boundary cases below. Open in the map.

Proof

O5. Representation loss and quantization loss separate

For a representation R(X)R(X),

IR=Var⁡(E[s∣R]),Iq⪯IR⪯Ifull.I_R=\operatorname{Var}(E[s\mid R]), \qquad I_q\preceq I_R\preceq I_{\rm full}.

This separates oracle/representation loss from hard-quantization loss whenever truth scores are available for validation.

From Score, ratio and classifier access · chapter contents