Result · SOFT-RANDOMIZED-FIM

Soft assignments define a randomized quantizer

Proved herepublication: internal

Meaning

The soft information objective corresponds to an actual randomized labelling mechanism.

Scientific context

Replacing a hard assignment by an assignment probability is not a smoothing trick: the resulting soft assignment is the exact Fisher information of a genuinely randomized quantizer, provided the randomization does not itself depend on the parameter. This gives a gradient-based route to a rule that hard-boundary optimization cannot supply directly.

Statement

For probabilities r_ib, W_b=Σ_i w_i r_ib and m_b=Σ_i w_i r_ib s_i give I_soft=Σ_b m_bm_b^T/W_b, the Fisher information of the corresponding randomized categorical observation when the randomization rule is theta-independent.

Assumptions

nonnegative assignments
rows sum to one
theta-independent randomization rule

Closest prior work

Nearest prior work

de Castro, Dorigo (2019)INFERNO: Inference-Aware Neural Optimisation

Relationships

Enables

Raises

Atomic randomization gapOPEN-ATOMIC-RANDOMIZATION-GAP

Open next door

In the library

Carried by SoftVoronoiConfig. Reference (opens in a new tab).

Every object and refusal

Local graph

Prerequisites to the left, what it enables or raises to the right, boundary cases below. Open in the map.

Proof

S1. Soft assignments are an actual randomized quantizer

For $r_{ib}\ge0$ , $\sum_br_{ib}=1$ ,

W_b=\sum_iw_ir_{ib},\qquad m_b=\sum_iw_ir_{ib}s_i,

\boxed{ I_{\rm soft}=\sum_b\frac{m_bm_b^\top}{W_b}. }

This is exactly the Fisher information of the corresponding randomized categorical observation at the reference point, provided the randomization rule is held fixed with respect to $\theta$ .

From Randomized and soft quantizers · chapter contents