Deterministic ranking diagnostic — 2026-09-19
Scope
This is a bounded diagnostic over the frozen 231,749-row PubChem structure index. It is not a learned ranker, an official CASMI score, a hidden-test estimate, or a submission. The deterministic ranking key is absolute neutral-mass deviation, followed by identity14 for ties. Retrieval uses only adduct and precursor_mz; target identity is used only for scoring the diagnostic.
The partition is the previously created post-freeze SHA-256 identity holdout: 1,171 identities and 56,150 valid queries. It is a stability readout, not a genuinely pre-registered untouched estimate.
Results
| Policy | Pool recall | Recall @25 | Recall @100 | Conditional MRR | Rank P95 |
|---|---|---|---|---|---|
| P0 strict ±10 ppm | 82.18% | 73.96% | 82.07% | 0.3578 | 38 |
| P1 Na ±30 ppm | 85.30% | 75.80% | 85.20% | 0.3487 | 40 |
| P2 empty-pool fallback | 84.04% | 75.35% | 83.93% | 0.3542 | 38 |
Readout
The mass-only ranking diagnostic confirms that candidate retrieval, rather than the deterministic mass ordering itself, is the first bottleneck. P1 improves coverage and @25 recall, but its conditional MRR is slightly worse than P0 because widening the sodium pool adds candidates without improving the mass-order position consistently. P2 retains nearly the P0 rank profile while recovering additional molecule-level coverage.
This is enough to define a useful non-learning baseline and a ranking evaluation contract. It is not enough to justify model training or competition claims. The next ranking comparison should use spectral similarity only within the frozen candidate pools, with the same policies, caps, identity deduplication, and reserved-partition caveat.
Verification
Maintained test suite: 17 passed in 29.03s.
Artifact: reports/phase3a-deterministic-mass-ranking-20260919.json
Script: scripts/diagnose_phase3a_mass_ranking.py