Analog-propagation notebook: bounded technical review
Provenance and evidence standard
Reviewed public notebook prvsiyan/analog-propagation-casmi-2026-baseline. One supported kaggle kernels pull … -m retrieved source/metadata at 2026-09-22 11:04:12–14 UTC / 19:04:12–14 MYT. Source SHA-256:
3ce550ab18441573ee519a4e1df88ae03090e00e424c16671e02bc6b76214a9e
The 157,046-byte source is byte-identical to the preserved 15:15 snapshot. Version unknown: returned metadata contains kernel ID 134523031, not a version; narrative “v5/v16” labels do not establish the downloaded version.
Verified: JSON/text and static AST inspection; 37 cells, including 17 code cells, all execution counts null, zero saved outputs. Thus every performance/runtime/table figure below is an author report, not independently reproduced evidence. No notebook cells, training, inference, benchmarks, installation or submissions were executed.
Anchors use zero-based cells and one-based source lines: C007:L080. C001:TRAIN identifies the safely decoded string literal, not executed code. Evidence root: research/reviews/analog-propagation-20260922-1902/; see retrieval-manifest.json, inspection-summary.json, cells/, training-script-annotated.txt, and hashed repo-source/ copies.
Ranked design takeaways for the plans
1. Specify four evidence channels; distinguish actual analogue propagation
The active channels are exact-mass library entropy matching, mass-shifted analogue/fingerprint propagation, candidate bond-fragment explanation, and spectrum-to-fingerprint scoring—not four proven statistically independent predictors.
analog_sim searches ±CFG.ANALOG_WIN = 200 Da, retains CFG.N_ANALOG = 200, and maximizes evidence across query spectra (C007:L170–187). Its kernel chooses the better whole-spectrum direct or uniformly shifted entropy score, not simultaneous direct/shifted fragment assignment (C006:L095–114). Candidate propagation uses max[Tanimoto × similarity^P_SIM] (C012:L029–037).
The repository’s corrected analogue diagnostic instead uses 10/30-ppm reference windows, direct cosine, at most 25 reference spectra, and cosine×Tanimoto (scripts/diagnose_phase3a_analogue_transfer_corrected.py:18–23,75–105). Its results do not validate this wider mass-shifted route. Keep Phase 3C separately gated, with identity-wide spectral exclusions and candidate/reference provenance.
2. Separate reference-mass correction from query-mass inference
CFG.LIB_MASS_FORMULA indexes reference spectra by formula-derived neutral mass, falling back to precursor/adduct conversion (C007:L032–061,080–088). Author-reported Class-1 MRR is 0.9206→0.9253; this supports an audit hypothesis, not a promised gain (C020:L012–037).
Local index construction currently derives mass from precursor/adduct and does not read formula/instrument fields (src/phase1_baseline.py:160–189). Add a separate, provenance-tagged reference-index comparison, preserving raw precursor residuals. Do not use unavailable test formulas or silently repair spectra from labels. Validate full formula parsing, ionic/electron-mass constants, charge/multimer conventions, rounding floors, unsupported adducts and cross-adduct consensus. The notebook’s permissive formula regex is not a chemistry validator.
3. Make fingerprint targets and objective explicit
The notebook concatenates ECFP4(4096), ECFP6(4096), RDKitFP(2048, maxPath=6) and MACCS(167), retaining 6,930 frequency-selected bits (C009:L010–012; C010:L012–028). Compare this with the planned 2,048-bit Morgan control; rare-bit removal is not inherently beneficial for isomer discrimination.
Training is BCE + --lam × hard-negative softmax cross-entropy, default --lam=1, --K=63; scores are computed in fp32 and centered/scaled by √nbits (C001:TRAIN:L0122–130,0162–171). Negatives are sampled with replacement from ±10 ppm around the positive structure’s mass; singleton windows fall back to global random decoys (L0041–052). Audit duplicate/positive-equivalent negatives and realistic precursor-error windows.
Inference f·z equals independent-Bernoulli bit log-likelihood up to a candidate-independent constant (C016:L009–025). It is neither guaranteed calibrated correctness nor direct MRR@25 optimization. Preserve best-checkpoint selection and separate training/selection/report cohorts.
4. Compare acquisition views without corrupting ion metadata
Author reports favor a separately trained single-spectrum model plus a merged-input model; --merge_p=0.6 samples 2–4 same-structure acquisitions (C029:L028–044; C001:TRAIN:L0055–064). Treat this as a bounded future comparison, not justification for unrestricted peak concatenation.
Active merging combines all molecule spectra, then supplies median precursor, the first adduct/instrument, fixed 25-eV energy and averaged polarity (C018:L030–062). Training peers are likewise not adduct-grouped. Prefer compatible adduct/polarity/charge groups or metadata-preserving set aggregation, with matched train/inference views.
Borrow the concrete surviving-row fix: _logits_from keeps cleaned peaks and their own metadata together (C018:L065–095). Include empty-spectrum and mixed-adduct fixtures. Instrument preference currently uses the global modal test instrument, not per-query adaptation to a mixed hidden set (C027:L015–020).
5. Audit ranker features, pruning and calibration together
Useful features include cross-channel agreement, maxima/margins, candidate count and set-relative ranks/z-scores (C012:L019–072). Exclude unearned pool-provenance signals: the author reports src or missing np_likeness identifying training-library positives and inflating simulated performance to 0.94 (C022:L013–023). This does not ban legitimate source-quality metadata; it requires source/missingness ablations.
The notebook fits eight HistGradientBoostingClassifiers using W1_PRIORS=(0.30,0.60) and four seeds on attached rank_train.npz (C026:L006–030). The actual grouped-CV/feature-generation harness is absent. Require OOF upstream features, query-grouped selection/calibration, feature-schema hashes, and appropriate query weighting; predict_proba alone proves no calibration.
Measure answer retention before/after every cap. Pruning changes set-relative feature distributions, so cached uncapped training features may become incompatible (C021:L022–065). Recompute/refit or establish invariance when pools, channels or caps change.
6. Preserve uncertainty about class mixture, pool expansion and generation
0.151/0.93≈16% is an assumption-dependent public-mixture inference, not an observed class label/count; the remainder includes Classes 2 and 3 (C000:L067–083). Likewise COCONUT’s reported 99.6% is specific to 250 curated NP examples, not universal coverage (C035:L081–105).
The reported PubChem 0.335→0.205 loss cannot uniquely identify hidden recall: recall and ranking dilution are confounded. Keep optional USE_BIO_DB/PC_TOPK expansion behind measured coverage-versus-dilution gates, not blanket rejection (C034:L092–128).
Reported unchanged-config scores 0.335/0.328 motivate artifact/prediction hashes, controlled seeds and paired molecule uncertainty—not a transferable ±0.004 noise threshold (C025). The “family” mixes configurations; causes are unverified.
Do not inherit “generation is the wrong tool”: generators can recover in-pool and out-of-pool structures. Beam width does not guarantee 25 unique valid scoring identities or optimize MRR directly. Retain calibrated union/reranking without fixed retrieval/generation slot quotas.
7. Add an independent same-formula diagnostic before expensive cross-encoding
Mass cannot resolve constitutional isomers. Benchmark a predefined same-formula cohort, not only cases where the current ranker failed (C033:L010–044). Author-reported n=241 within-isomer MRR: analogue 0.5411, fingerprint 0.4904, fragmentation 0.2779, random 0.1688.
C032 reports four unsuccessful (z,f) interaction-head experiments: holdout −0.007; small held-out training set overfit; artificial-noise training −0.18; tiny regularized head approximately neutral/−0.008. These do not disprove learned reranking. A candidate–peak interaction model or learned fragmenter is a separate, deferred hypothesis requiring OOF trunk features. Neither that implementation nor the inferred private competitor architecture is supplied. The prose’s universal information-loss/Bayes-optimality conclusions are stronger than its evidence.
8. Keep the DreaMS result narrowly scoped
C031:L016–032 reports the same queries, subsampled analogue windows and propagation arithmetic, changing similarity only. At n=130: entropy 0.1555, DreaMS cosine 0.1129, entropy×DreaMS^0.5/0.25/1: 0.1428/0.1526/0.1409. At n=40, entropy×√DreaMS appeared better: 0.1848 vs 0.1700 entropy and 0.1604 cosine.
No experiment code, checkpoint identity, exact preprocessing/subsampling manifest or saved outputs are present. This is not full-library evaluation, fingerprint-head finetuning, or proof that DreaMS universally fails. The missing mass-shift invariance explanation remains a hypothesis. Any future representation experiment needs its own eligibility and bounded validation gate.
Active-code/prose contradictions not to copy
CFG.SIM_POWER is unused; P_SIM=3.0 controls features. CFG.W1 is unused; W1_PRIORS controls fitting (C002/C012/C026; static summary).CAND_CAP=500 actually truncates candidates for all downstream channels, using 100×library_sim + standardized f·z, or mass fallback—not model-only pruning (C027:L040–065).Rights and reproducibility blockers
Metadata lists six external datasets, but certifies neither their versions/licenses nor model-training exclusions. Notebook license is not exposed. COCONUT “CC-BY,” DreaMS “MIT,” host permission and timings are author statements only. Verify code, checkpoint, upstream-data and redistribution rights separately; do not redistribute competition-derived training structures. Required weights, bit ordering, rank_train.npz, preprocessing assets and the referenced fpmodel module need pinned provenance before reuse. Plans remain hypotheses; this review authorizes no model execution.