← Back
📝 Notes / reports / todo-review-20260918 / G1-OPTIMIZATION-DESIGN

📑 Contents

G1 evaluator optimization design — 2026-09-18StatusObserved bottleneckSafe optimization sequenceNon-negotiable semanticsStop/restart rule

G1 evaluator optimization design — 2026-09-18


Status

Design only; not launched while the current persistent g1 run is active.


Observed bottleneck

The evaluator scans the full training Parquet once per regime and, for each valid query spectrum, loops through every mass-window reference row and calls Python-level cosine similarity. Checkpoints show the dense interval dominates runtime: by batch 400, 69,211,929 candidate rows had been scored. The current np.searchsorted(masses[order], ...) expression also materializes/indexes the ordered mass view repeatedly.


Safe optimization sequence


1. Preserve a correctness oracle. Keep the current evaluator and its synthetic tests as the reference on a bounded fixture. Record candidate membership, identity deduplication, deterministic tie order and metrics before optimizing.

2. Precompute ordered mass arrays once. At evaluator startup, bind sorted_masses = masses[order] (or persist a validated memory-mapped sorted array). Prove identical lo:hi membership for fixture queries before using it.

3. Separate query extraction from candidate scoring. Stream training once per regime to create compact valid query records and checkpoint counts; score each query against a reusable candidate index. Do not load all peaks or all training rows into pandas.

4. Use an inverted mass-bin index. Bucket reference row positions by a conservative mass bin, then apply the exact ppm/floor filter inside the selected bins. Candidate membership must be byte-for-byte equivalent to the oracle on a fixture and a bounded production slice.

5. Batch/vectorize similarity. Convert candidate peaks to a compact representation and batch cosine calculations where memory permits. Keep a bounded candidate chunk size and record peak RSS.

6. Checkpoint at query and candidate-chunk boundaries. Checkpoints must include regime, query row/spectrum, candidate rows, elapsed time, RSS and deterministic input hashes. Atomic replacement is required.

7. Run paired validation. Compare optimized vs oracle outputs on a synthetic fixture, then a bounded real slice, then the full frozen cohorts. Promote only if candidate membership, ranks and aggregate metrics match.


Non-negotiable semantics


  • Regime-A query selection is the frozen row mask, not identity membership.
  • Regime-B reference exclusion is identity-wide and fold-specific.
  • Pool coverage is target presence in the uncapped full eligible pool.
  • Conditional MRR@25 divides by all pool-covered molecules, including covered targets ranked below 25.
  • Candidate identities are deduplicated before rank caps.
  • Resource and provenance evidence are part of the result, not an afterthought.

    • Stop/restart rule

    If the active run reaches a checkpoint with RSS approaching the host budget, exits unexpectedly, or fails to advance for 90 minutes, preserve the last checkpoint and stop it. Do not call the partial output a G1 result. Optimize in a separate implementation, prove oracle equivalence, then rerun.