Phase 3A reference-availability audit — 2026-09-19
Scope
Bounded post-freeze stability readout over 1,171 deterministic identity-holdout molecules and 500 selected queries (100 per fold), using the 231,749-row PubChem structure index and the 2,539,608-row local spectral index. This is not a prospective untouched estimate.
The audit separates structure candidate coverage from local spectral-reference availability. Retrieval uses only adduct and precursor_mz. Fold-eligible local references exclude the held-out query identity and any references belonging to that fold's held-out identities.
Aggregate results
| Policy | Structure pool recall | Any local candidate reference | Any fold-eligible candidate reference | Target has local reference | Target has fold-eligible reference |
|---|---|---|---|---|---|
| P0 strict ±10 ppm | 94.11% | 95.58% | 58.11% | 100.00% | 0.00% |
| P1 Na ±30 ppm | 94.11% | 95.58% | 58.53% | 100.00% | 0.00% |
| P2 empty-pool fallback | 94.11% | 95.58% | 58.11% | 100.00% | 0.00% |
Candidate identity pool P50/P95 was 20/63 for all three policies. Fold-eligible candidate-reference identity pool P50/P95 was 1/9.
Interpretation
1. The PubChem structure pool is substantially broader than the eligible local spectral-reference pool.
2. Every selected target has a local reference in the full index because the queries come from the visible training release. This is useful for the library ceiling only.
3. Every selected target has zero fold-eligible references by design: the target identities are held out. Therefore, a pure local spectral-library matcher cannot produce a conventional target rank for this identity-held-out cohort.
4. Only about 58% of queries have at least one fold-eligible spectral-reference candidate anywhere in the structure mass pool. This is the meaningful reference-availability constraint for cross-identity decoy analysis.
5. P1's sodium widening changes candidate/reference availability only marginally in this small bounded audit; its main benefit remains target structure coverage in the earlier query-level audit.
Decision
Keep the library-ceiling rerank as an optimistic implementation control. Do not report its MRR as unseen-identity evidence. For the next baseline, use a structure-aware score for PubChem candidates and report three failure classes separately:
Artifact: reports/phase3a-reference-availability-20260919.json.
Script: scripts/audit_phase3a_reference_availability.py.
Maintained tests: 17 passed in 29.99 seconds.