Result · OPEN-DS-DOMINATION-EQUALITY
Efficient-score domination is tight exactly when nuisance projections agree
Proved herepublication: internal
Meaning
The exact gap and equality conditions identify when the efficient-score upper bound is attained.
Statement
Var(E[S_hat|q]) - S_psi^+(I_q) = (B_full - B_q) I^q_ll (B_full - B_q)^T exactly; equality for fixed q iff (B_full - B_q) I^q_ll = 0 (the full-data nuisance projection is binned-optimal); along partition sequences refining to the Borel sigma-field the gap vanishes. At global finite Ds optima the gap is the DS15 projection tax and vanishes almost surely for conditionally centered laws with d_psi = d_lambda = 1 (DS15 conclusion (v), audited); beyond that class it is OPEN-DS-MARGINS-NONCENTERED.
Relationships
Rests on
Open next door
Local graph
Proof
DS11. Variational form of the profiled objective and $\Phi$-neutral splits
Audit (28 Aug 2026, AUDITS/AUDIT-DS-POPULATION-BRIDGE-001.md): the boxed
identity is classical — it is the extremal characterization of the generalized
Schur complement (Krein 1947; Anderson's shorted operator 1971; Li–Mathias,
SIAM Review 42(2), 2000, Thm 2.2, including the Loewner order, the
pseudo-inverse, and the exact attainment set). The statistical reading
additionally requires
centered scores (). The binned transfer and consequences (a)–(d)
are project-level.
For any partition with binned information blocks define the generalized profiled information (Moore–Penrose pseudo-inverse; the ordinary Schur complement when ). Then
a Loewner minimum over matrices , attained at every solution of (in particular ; solutions exist because for a PSD block matrix).
Proof. ; for any normal-equation solution , completion of squares gives , and . ∎
Consequences.
(a) Efficient-score domination, singular extension, and the exact equality condition (an earlier, since-closed question, fixed ). Evaluating at re-derives DS7 in one line, extends it to singular binned nuisance blocks, and gives the exact gap
Equality for a fixed holds iff
: the full-data nuisance
projection is already optimal for the binned centroids. The finite- gap is
therefore exactly the cost of estimating the nuisance projection from bins.
Along any refining sequence of partitions generating the Borel
-field, in (Lévy upward martingale
convergence), so ; provided
(implicit in defining ),
the binned nuisance blocks are eventually nonsingular, where
is continuous, so
and the gap vanishes — the part
of an earlier, since-closed question. (The pseudo-inverse map is discontinuous at rank drops — witness
, ,
— so the nonsingular-limit hypothesis is load-bearing; audit
§8.) The behavior at global optima was an earlier, since-closed question and is now settled by DS15 for
conditionally centered laws at : the gap is the
projection tax and vanishes a.s. along free global optima (DS15 conclusion 5);
beyond that class it is OPEN-DS-MARGINS-NONCENTERED.
(b) Refinement monotonicity with exact equality characterization (). Splitting cell into : for every slope , with ,
Hence never decreases under refinement, and the split is profiled-information-neutral iff some simultaneously minimizes and equalizes . (Audit: the identity is the exact between-group variance decomposition; the "iff" holds by evaluating at a minimizer of the split problem, which exists by the normal-equation argument. The same statement and proof hold for every with the rank-one PSD gap and Loewner sandwiching — the form DS14's merged variant consumes; audit §8.)
(c) Wasted cells and exact global ties. If the merged configuration is entirely nuisance-degenerate (all cells have , so is constant and every slope minimizes it), every split whose sub-cells have distinct nuisance means is exactly neutral, while a split with equal (hence zero) nuisance means and distinct POI means strictly increases but keeps the nuisance block singular — it leaves the in-bin formulation (DS9). (Audit hardening: the neutrality argument needs the minimizer set of to meet the equalizing slopes; full nuisance degeneracy supplies that, a partially degenerate configuration need not.) Exact witnesses:
COUNTEREXAMPLES/CE-DS-DEGENERATE-GLOBAL-TIE-001.json: a centered equal-weight sample whose exact global in-bin optimum is attained by 31 distinct labelings — exactly the feasible refinements of one reduced bipartition — every one of which has two exactly coincident projected centroids; the unique nuisance-mean-equal refinement is infeasible (singular nuisance block) with generalized value .COUNTEREXAMPLES/CE-DS-POP-WASTED-CELLS-001.json: the population/quadrature wasted-cell construction of DS12.
(d) Identifiability up to neutral splits. The profiled objective is invariant under -neutral splits, so a finite global optimum is in general identified only up to the reduced configuration of projected centroids ; deployable content lives at the reduced level.
From Profiled Ds: from a sample to the population · chapter contents