Kaggle Community Review Findings — Enveda CASMI 2026

Date: 2026-09-17

Competition: enveda-CASMI26-molecule-id-mass-spectra

Review scope: 77 public Kaggle notebooks and 14 public discussion threads

Collection method: Kaggle CLI 2.2.4

Status: Community evidence and planning guidance; not production validation

Purpose

This review examined all publicly shared competition notebooks and discussion threads to identify ideas worth incorporating into the current Enveda CASMI 2026 plan. The review intentionally inspected public code, markdown, metadata, and available outputs rather than selecting methods solely by leaderboard score.

Public claims and leaderboard figures are treated as hypotheses. They must not replace evidence from the authoritative rebuilt namespace:

artifacts/rebuild-20260917/

No community method should be promoted into production without leakage-safe validation on the current split and scorer contract.

Executive conclusion

The community review strongly supports the current architecture, especially:

It does not justify copying public leaderboard numbers, fixed ensemble weights, or moving directly to a large neural model. The main planning change is to make structure-only candidate retrieval and analog propagation the next serious track after qualified G1, rather than treating de novo generation as the first response to hidden novelty classes.

---

1. The downloadable test.parquet is a dummy file

Discussion 741404 includes a host clarification that the downloadable test file contains copied training examples. The public leaderboard is evaluated against a hidden natural-product test set, not against the contents of the downloadable demonstration file.

This explains the discussion titled “Data leak: every test spectrum appears verbatim in train.parquet” (741756). The visible file is intentionally a sample demonstrating the format and scale; this is not evidence of an accidental competition leak.

Planning implication

---

2. Library-level holdout of enveda-np-examples is invalid

Discussion 741597 provides the most important validation warning. The 250 natural-product structures associated with enveda-np-examples also occur in other libraries, including riken, gnps, mona, massbank, and pluskal_ms2.

The following validation design is therefore invalid:

Validation = rows where ingest_lib == enveda-np-examplesReference  = every other row

The same identities remain available in the reference data, allowing direct spectral retrieval and inflating the validation score.

Planning implication

The current Regime-B policy is correct: exclude every spectrum belonging to a held-out identity, not merely rows from the named library.

Keep the following as mandatory invariants:

Do not replace identity-wide exclusion with library-level exclusion.

---

3. Multi-spectrum aggregation is essential

Many strong public notebooks emphasize that a molecule_id can have multiple spectra, potentially differing by:

Recurring recommendations include aggregating evidence across all spectra, deduplicating by scorer-compatible InChIKey14, preserving complementary spectra, and avoiding arbitrary selection of a single spectrum.

Planning implication

The all-spectrum evaluator currently running is the appropriate next validation step. After it completes, compare controlled aggregation policies such as:

1. maximum candidate score across spectra;

2. mean or top-k score;

3. reciprocal-rank fusion;

4. collision-energy-aware aggregation; and

5. same-adduct and same-polarity evidence.

The first safe baseline should be max, followed by a controlled multi-spectrum ablation. A learned aggregator should not be introduced before the simple policies are independently measured.

---

4. Direct library retrieval is a Class-1 floor

Across public notebooks, the recurring strategic picture is:

Approximate public figures are commonly discussed around:

These figures are public leaderboard claims and are not directly comparable to local rebuilt-namespace validation.

Planning implication

Treat direct spectral retrieval as a clean, reproducible Class-1 floor. It is not expected to solve the majority of hidden natural-product cases by itself.

This supports the current sequence:

1. establish trustworthy leakage-safe retrieval;

2. complete qualified local G1;

3. add structure-only candidates and analog propagation; and

4. only then train learned rankers.

---

5. Mass-shifted analog propagation is the most promising next retrieval extension

The strongest recurring non-neural idea is a mass-shifted analog route:

spectral neighbor    → mass-shifted candidate neighborhood    → scorer-identity deduplication    → fingerprint / mass / spectral reranking

Typical components include:

Planning implication

Add analog propagation as a planned Phase 3 route, after G1. It should remain separate from the clean retrieval baseline and be evaluated with explicit provenance:

Do not import public analog weights or leaderboard-calibrated constants without local ablation evidence.

---

6. COCONUT/database retrieval is more credible than immediate de novo SMILES generation

The notebooks repeatedly highlight that exact InChIKey14 scoring is unforgiving. A chemically similar generated SMILES can still score zero if it does not canonicalize to the correct scorer identity.

Autoregressive de novo generation is reported as unstable and difficult to validate. For known structures without reference spectra, a more practical route is:

formula / neutral-mass gate    → COCONUT or PubChem candidates    → fingerprint or fragmentation-informed ranking    → exact identity deduplication

Planning implication

Promote COCONUT/PubChem from a general idea to a high-priority structure-only candidate route.

Requirements:

Regime-B evaluation must ensure that any structure-only candidate pool is not derived from held-out spectra. Structure availability alone is not spectral availability, and the two must not be conflated.

---

7. Full adduct handling matters

Several notebooks describe errors caused by incomplete or inconsistent adduct tables, including:

The local Phase 0 audit already identified and corrected a formate mass issue.

Planning implication

Promote adduct normalization to a permanent pre-evaluation gate. Add explicit alias and round-trip tests, including:

[M+FA-H]- ≡ [M+CH2O2-H]-

The full supported adduct table should be tested for:

---

8. Do not blindly copy mass windows or candidate caps

Public notebooks use materially different settings for:

Some authors report that restrictive candidate caps silently remove the correct Class-2 candidate. Others report that broad mass windows increase false positives.

Planning implication

Treat these as controlled local ablations, not fixed truths. Suggested grid:

Mass window:       8.5 / 10 / 20 / 30 ppmCandidate cap:     none / 100 / 1,000 / 6,000Intensity floor:   0.002 / 0.005 / 0.01

Record for every setting:

No public setting should be adopted without evidence on the rebuilt namespace.

---

9. Spectrum-to-fingerprint prediction is the most credible learned route

The more credible learned approaches predict a molecular fingerprint from the spectrum and use that representation to rank valid candidate structures.

This is preferable to direct SMILES generation because:

Public approaches include:

Planning implication

Prioritize the following Phase 2 architecture after qualified G1:

spectrum    → fingerprint logits / embedding    → candidate fingerprint similarity    → molecule-level learned reranking

Training and evaluation must use leakage-safe out-of-fold features under the rebuilt Regime-B masks. No learned result should be accepted based only on random row splits or library-level holdouts.

---

10. Static rank fusion is not yet justified

Some public notebooks combine multiple public submissions using fixed reciprocal-rank or hand-tuned weights. This may be useful for competition packaging, but it is not evidence of a scientific improvement unless:

Planning implication

Leave rank fusion for Phase 5. Do not introduce fixed weights copied from public notebooks into the current baseline.

---

Changes to the current plan

Keep unchanged

Strengthen

1. Add a permanent warning that test.parquet is a placeholder and cannot support leaderboard claims.

2. Add a regression test documenting why library-level holdout is invalid.

3. Add adduct alias and full-table round-trip tests.

4. Add controlled mass-window and candidate-cap ablations.

5. Add an explicit multi-spectrum aggregation benchmark.

Promote in priority

1. Phase 3A: pinned COCONUT/PubChem structure-only candidate pool.

2. Phase 3B: formula- and mass-gated candidate generation.

3. Phase 2: spectrum-to-fingerprint out-of-fold ranker.

4. Phase 3C: mass-shifted analog propagation.

5. Phase 5: rank fusion and ensemble calibration.

Keep deferred

---

Recommended execution sequence

1. Complete the current all-validation-spectrum G1 run.

2. Independently validate query counts, runtime, memory, and aggregation semantics.

3. Close qualified local G1 only if the run is internally consistent.

4. Add the visible-test placeholder warning and community leakage regression tests.

5. Run mass-window, candidate-cap, and multi-spectrum aggregation ablations.

6. Build a pinned structure-only COCONUT snapshot.

7. Begin Phase 2 fingerprint/ranking experiments with out-of-fold Regime-B features.

8. Add mass-shifted analog propagation as a separate candidate-generation track.

9. Keep official scorer parity and hidden-test claims explicitly unresolved until organizer scorer evidence becomes available.

Evidence boundaries and caveats

Relevant Kaggle sources

Discussion threads

Representative public notebooks

Bottom line

The community evidence confirms that the current leakage-first rebuild is the right foundation. The most valuable additions are not leaderboard tricks; they are better candidate coverage and better evidence aggregation:

1. finish and audit all-spectrum G1;

2. add pinned structure-only database candidates;

3. add mass-shifted analog propagation; and

4. train a leakage-safe spectrum-to-fingerprint reranker.

De novo generation and static ensemble fusion should remain later experiments, not the next default path.