2026-09-22 — Enveda CASMI26: Phase 4 completion, copy bug diagnosis, kernel v4

Phase 4 v3 Results — COMPLETED (but broken)

Kernel v3 completed at ~10:47 AM MYT after 3 epochs (~7h on Kaggle T4).

Training metrics:

But all generation metrics were zero:

Root Cause — TWO BUGS

Bug 1: Loss not shifted (the copy bug)

The code that ran on Kaggle (v3) computed loss as:

F.cross_entropy(logits.view(-1, ...), structure_tokens.view(-1))

The tokenizer auto-adds BOS/EOS, so structure_tokens = [BOS, t1, ..., tN, EOS, PAD...].

The decoder input IS structure_tokens (teacher forcing).

Without shifting, the model learns to predict token at position i GIVEN token at position i — i.e., just copy its own input.

Confirmed with a direct test: fed the trained model [BOS, X] and it predicted X with score ~12-13 (vs next-best ~2). Every single token, 5/5 tests.

This is why train_loss collapsed to 0.0007 — the model perfectly learned the identity function, which is trivial. It's also why valid_smiles was 0.0 — during beam search there's no input token to copy at the first step, so it degenerates.

Bug 2: No causal mask during beam search

tgt_mask = Noneif structure_tokens is not None:    tgt_mask = nn.Transformer.generate_square_subsequent_mask(...)

The causal mask was only applied during training (when structure_tokens is passed). During beam search (no structure_tokens), tgt_mask=None — every position attends to every other position. Train/generate mismatch.

The Fix (v4, pushed 14:30)

Fix 1 — shifted loss:

out["loss"] = F.cross_entropy(    logits[:, :-1].reshape(-1, logits.size(-1)),    structure_tokens[:, 1:].reshape(-1),    ignore_index=self.pad_token_id,)

Fix 2 — causal mask always applied:

tgt_mask = nn.Transformer.generate_square_subsequent_mask(    tgt.shape[1], device=tgt.device)

Moved outside the if structure_tokens is not None block.

Fix 3 — rdkit auto-install:

Kaggle's GPU image doesn't ship rdkit, so Tanimoto/exact_match were null.

Added pip install rdkit with graceful fallback to the syntax-only checker.

All three verified: file compiles, diff confirmed against Kaggle's pulled source.

Kernel v4

Cron Health

Files

Lesson

The copy bug was invisible in the loss curve — train_loss 0.0007 looked like excellent convergence.

The tell was valid_smiles=0.0 with near-zero loss: if the model were genuinely learning, generation

would produce something. Near-zero loss + zero generation output = the model learned something

trivial (identity), not the task. Always sanity-check generation, not just loss.