Enveda CASMI 2026 Todo

Legend: [ ] pending · [-] in progress · [x] complete · [!] blocked

The priority order is deliberate: establish trustworthy evaluation and a strong retrieval floor before investing in larger models.

Phase 0 — Ground truth and infrastructure

[x] P0 Download the current train.parquet release from Kaggle.

[x] P0 Record file size, SHA-256, Kaggle metadata and download timestamp in manifests/.

[x] P0 Inspect the actual training schema, row groups, library counts and identity counts.

[x] P0 Add a low-RAM data access path using PyArrow row-group streaming and selected columns.

[x] P0 Create a pinned RDKit 2026.03.3 environment for scorer-compatible identity matching.

[x] P0 Implement tautomer-canonicalized InChIKey14 identity logic in src/identity.py.

[x] P0 Add scorer unit tests for tautomer equivalence, stereochemistry, invalid SMILES and rank cutoff.

[x] P0 Implement all ten advertised adduct mass conversions and round-trip tests.

[x] P0 Freeze molecule-grouped validation manifests for library, database-only and novelty-proxy regimes. Scorer-compatible five-fold manifest plus strict PubChem structure-only and scaffold-disjoint proxy manifests with frozen leakage rules are complete; operational exclusion checks belong to Phase 1 evaluation.

[x] P1 Add a reproducible experiment configuration format under configs/.

Resource constraint: the local development host has approximately 11 GiB RAM. Full-data operations must stream or use bounded artifacts; do not load the entire training table into pandas.

Gate G0: CLOSED under the documented bounded-external-audit scope. PubChem results are proxies, not claims about the full hidden-test novelty distribution.

Phase 1 — Spectral retrieval baseline

[x] P0 Build a streaming Parquet reader and compact peak-array representation.

[x] P0 Implement precursor/adduct to neutral-mass filtering.

[x] P0 Implement peak cleaning variants: intensity floor, precursor exclusion and top-N/windowed peaks.

[x] P0 Implement fragment cosine similarity.

[x] P0 Implement spectral entropy similarity.

[x] P0 Implement neutral-loss similarity.

[x] P0 Build an indexed reference search over training spectra. Disk-backed float32 arrays built from all 2,539,608 rows; 2,160,035 valid spectra indexed.

[x] P0 Aggregate evidence across all spectra for each molecule_id. The local writer scored 1,213 spectra and aggregates maximum identity evidence across 400 molecules.

[x] P0 Deduplicate candidates by scorer-compatible identity. Candidate pools are keyed by scorer identity14 before ranking.

[x] P0 Generate and validate a first local submission file. submissions/phase1-retrieval-molecule.csv passes the 400-ID sample contract with 1–25 candidates per row.

[ ] P1 Add source, adduct, polarity and collision-energy compatibility features.

[x] P1 Add timing, RAM and candidate-coverage measurements. Index build: 149.85 s; submission run: approximately 2 min; peak observed RSS: approximately 1.5 GiB; molecule coverage: 100%; 769,437 reference rows scored.

Gate G1: deterministic retrieval baseline with molecule-level MRR@25, Top-1, Recall@25 and runtime reports on frozen validation splits.

Phase 2 — Learned ranking and domain adaptation

[ ] P0 Generate out-of-fold candidate features without molecule leakage.

[ ] P0 Train a first molecule-grouped LightGBM/CatBoost ranker.

[ ] P0 Compare learned ranking against hand-tuned retrieval aggregation.

[ ] P1 Balance or reweight training sources to reduce synthetic-library dominance.

[ ] P1 Add reference-count bias correction.

[ ] P1 Add confidence margins and uncertainty diagnostics.

[ ] P1 Run ablations for fragment, entropy, neutral-loss and multi-spectrum evidence.

Gate G2: learned ranking improves grouped validation without severe regression in any intended novelty regime.

Phase 3 — Known structures without reference spectra

[ ] P0 Acquire and document an eligible COCONUT snapshot.

[ ] P1 Build a mass/formula index for competition structures and COCONUT.

[ ] P1 Add a broader PubChem-derived structure candidate source if storage and licensing permit.

[ ] P1 Implement spectrum-to-fingerprint neighbour transfer.

[ ] P1 Test pretrained spectrum embeddings such as DreaMS/MIST after license and offline checks.

[ ] P1 Train a spectrum-to-fingerprint retrieval model.

[ ] P1 Evaluate formula inference and candidate-pool recall.

[ ] P2 Add a forward-spectrum reranker for shortlisted structures.

Gate G3: measurable improvement in the structure-known/no-spectrum validation regime, reported separately as pool coverage and conditional MRR.

Phase 4 — Novel structures and de novo generation

[ ] P1 Establish a clean novelty-proxy split with structure and spectral exclusions.

[ ] P1 Build a formula-, adduct- and collision-energy-conditioned spectrum encoder.

[ ] P1 Implement a bounded SMILES decoder/generator.

[ ] P1 Add beam search and controlled sampling with valid-SMILES filtering.

[ ] P1 Add analogue editing from close spectral neighbours.

[ ] P1 Deduplicate generated outputs by scorer-compatible identity.

[ ] P2 Add forward-spectrum and formula consistency reranking.

[ ] P2 Measure unique candidate recall and actual MRR contribution, not validity alone.

Gate G4: de novo candidates add measurable clean-validation MRR or candidate coverage within the offline runtime budget.

Phase 5 — Unified ensemble

[ ] P0 Union retrieval, database and de novo candidate pools.

[ ] P0 Train a final OOF molecule-level ensemble ranker.

[ ] P0 Produce exactly one row per runtime molecule with at most 25 unique candidates.

[ ] P1 Calibrate route confidence and preserve exploration for uncertain molecules.

[ ] P1 Stress-test rare adducts, high masses, one-spectrum molecules and noisy spectra.

[ ] P1 Compare library-heavy, balanced and novelty-heavy validation mixtures.

[ ] P1 Perform paired-bootstrap comparisons and record promotion decisions.

Gate G5: robust integrated model beats the retrieval baseline across the selected validation scenarios.

Phase 6 — Kaggle packaging and final submission

[ ] P0 Package model weights, indexes, tokenizers and dependencies for offline notebook execution.

[ ] P0 Run with internet disabled.

[ ] P0 Test against dynamically loaded test IDs; do not rely on placeholder IDs.

[ ] P0 Verify submission.csv columns, nulls, duplicate IDs and maximum candidate count.

[ ] P0 Complete two clean-room notebook reruns within the 9-hour limit.

[ ] P1 Prepare two complementary final candidates only if validation supports both.

[ ] P1 Record final code, configuration, hashes, licenses and reproduction instructions.

[ ] P0 Select final submissions explicitly before the deadline.

Gate G6: two successful offline reruns, reproducible output and final compliance audit.

Active experiment log

Date	Experiment	Hypothesis	Result	Decision
2026-09-16	Competition review and workspace setup	Establish a complete execution baseline	Plan and project scaffold created; no model run	Start Phase 0
2026-09-16	Phase 0 data acquisition	Verify current Kaggle release and characterize it under low RAM	3,033,286,496-byte file; SHA-256 recorded; 2,539,608 rows, 21 row groups, 275,810 unique InChIKey14 identities; audit complete	Proceed to grouped split manifests
2026-09-16	Molecule-grouped split	Prevent spectral/structure leakage in local validation	Deterministic 5-fold manifest validated: 275,810 groups and 2,539,608 rows; fold assignment is group-exclusive	Use fold 0 as initial validation; keep novelty labels separate
2026-09-16	G0 data/infrastructure gate	Confirm reproducible Phase 0 inputs and assumptions	All eight gate checks passed; config and split hashes recorded in `manifests/phase0-gate.json`	Close data/infrastructure work; proceed to retrieval baseline
2026-09-16	Phase 0 audit	Verify that validation grouping uses scorer identity	Failed: 12/100 sampled normalized SMILES differed between stored and scorer-derived InChIKey14; full probe hit pathological tautomer enumeration	Reopen G0; rebuild identity-safe split before retrieval evaluation
2026-09-16	Phase 0 corrective audit	Verify corrected scorer-identity split and reproducibility gate	Passed: 8 tests; compilation passed; 2,539,608 rows and 274,288 scorer groups validated; cache/checkpoint agree; full G0 gate passed; 100-row identity probe had 0 invalid structures	Proceed to retrieval baseline; retain external novelty regimes as pending
2026-09-16	Phase 1 retrieval baseline	Build a low-memory, identity-safe spectral retrieval floor	Index built over all 2,539,608 rows; 2,160,035 valid spectra; fold-0 mask excludes all 503,842 held-out rows; molecule-level fixture submission has 400/400 covered molecules and passes schema/count checks	Keep baseline; run grouped MRR/Top-1/Recall evaluation before closing G1

Blockers and decisions

Current blocker: database-only and novelty-proxy validation labels require an external structure/scaffold audit; known-library regime is ready.

Do not start: large model training before G0 scorer, split and data-manifest gates pass.

Do not submit: until the first local output passes scorer-compatible validation and offline packaging checks.

📑 Contents