ScoreQuant

Result · FI-LOSS-DECOMPOSITION

Binning loss is within-cell score scatter

Derived from the literaturepublication: derived from published

Read the proof and evidence →

Meaning

The difference between full and retained Fisher information equals the within-cell second-moment loss.

Scientific context

The difference between the full information and the binned information is the expected within-cell covariance of the score, which is positive semidefinite: binning can never add information. So the whole problem here is a minimization of an unavoidable loss, never a search for a lucky gain.

Statement

I_full-I_q=E[Cov(S|Z)] is positive semidefinite; hard quantization cannot increase Fisher information.

unbinned-vs-binned information accounting

Closest prior work

Nearest prior work

Relationships

Rests on

Enables

Local graph
Local map of Exact Fisher-information loss decomposition1 prerequisites to the left, 3 results it enables or questions it raises to the right, 0 boundary cases below.Quantized Fisher-information identityQuantized Fisher-info…Local Fisher-losslessness iff the score is bin-measurableLocal Fisher-lossless…Normalized retained-information spectrumNormalized retained-i…Representation loss and hard-quantization loss separateRepresentation loss a…Exact Fisher-information loss decompositionExact Fisher-informat…
Prerequisites to the left, what it enables or raises to the right, boundary cases below. Open in the map.

Proof

U2. Exact information-loss decomposition
Ifull−Iq=E[Cov⁡(S∣Z)]⪰0.\boxed{ I_{\rm full}-I_q=E[\operatorname{Cov}(S\mid Z)]\succeq0. }

Consequences:

  • hard quantization cannot increase Fisher information;
  • all retained-information criteria here depend on a quantizer only through (Wb,mb)(W_b,m_b).

From Universal information structure · chapter contents