Enveda CASMI 2026 Phase 0 re-audit — 2026-09-17 17:07 MYT

Executive result

G0 is not unconditionally closed. The rebuilt namespace passes the local core/infrastructure checks, but current root entry points are mixed with legacy artifacts, validation-regime enforcement is incomplete/unproven, and official scorer parity is unavailable. The correct status is:

> G0 CLOSED FOR LOCAL REBUILT INFRASTRUCTURE ONLY; REGIME-A OPERATIONAL CLOSURE, REGIME-B RUNTIME ENFORCEMENT, AND OFFICIAL SCORER PARITY REMAIN OPEN.

Evidence is in current-state-evidence.json; this report is read-only with respect to production data/code/configs. The audit generated only files in this directory.

Verified rebuilt namespace

Train: 2,539,608 rows; SHA-256 9423f90ec08b941a9646b535bf0606183e86ff0c5a297978f5a70693d41bcd98; file size 3,033,286,496 bytes.

Rebuilt cache: 277,566 normalized-SMILES entries and 274,195 scorer identities; metadata contains the corrected identity algorithm, RDKit 2026.03.3, and the train hash.

Rebuilt split: 274,195 identities; streamed row count and per-identity row counts match; deterministic fold recomputation has zero mismatches.

Rebuilt retrieval arrays: 2,539,608 rows, 2,160,035 valid rows, all five fold masks present; every rebuilt mask matches valid & (identity_fold != held_out_fold).

Rebuilt index identity array has zero row mismatches against the authoritative cache. The old root index has 4,703 identity-row mismatches and uses the superseded 274,288-identity namespace.

Rebuilt formate neutral-mass conversion has zero errors above 0.001 Da across 83,188 rows; root legacy index fails all 83,188 checks, with approximately 1.00734 Da maximum error.

Rebuilt Regime-A SQLite: integrity ok; 2,539,608 rows; 274,195 identities; 501 duplicate fingerprint groups, 501 excess rows, maximum multiplicity 2; row-to-identity mapping matches rebuilt index.

Clean-room package previously reported stable manifest/strict hashes and passed its recorded package checks.

Fresh full test run: 19 passed (tests plus audit regressions). The prior 17/18 counts are stale historical records.

Namespace / lineage finding

Resolved root paths are not uniformly rebuilt:

artifacts/identity-cache.sqlite -> rebuilt cache.

manifests/molecule-grouped-5fold-scorer.json -> rebuilt manifest.

artifacts/retrieval-index -> legacy root index.

root external strict-manifest paths expected by src.validate_phase0.validate() are absent, while equivalent rebuilt-staging files exist under artifacts/rebuild-20260917/root/.

This means a generic root-level run can either fail due to missing expected paths or silently consume the legacy retrieval index. Phase 1 must not use root retrieval artifacts until all namespace-safe paths are explicit and verified.

Validation regimes

Regime A — target-domain/library-available: PARTIAL / OPEN

The local parquet contains 1,184 enveda-np-examples rows across 250 identities, while the published description records 1,151 rows/250 compounds. This is explicitly pinned as a current-release/documentation discrepancy in manifests/rebuild-20260917/np-example-reconciliation.json; the hashed current parquet remains authoritative for local evaluation.

The exact available-field fingerprint DB is useful and conservative, but it is not yet a frozen operational A manifest. No committed A query mask/output was found, and evaluate_sample selects generic per-fold rows rather than ingest_lib == "enveda-np-examples". A must hold out target-domain query acquisitions while retaining independent same-identity references from other libraries. Raw acquisition IDs are unavailable, so exact available-field fingerprint exclusion must be explicitly labelled a local proxy.

Acceptance: freeze a JSON/TSV manifest with source hash, NP query row IDs, fingerprint definition, SQLite integrity and row mapping; build and enforce query/reference masks; assert query rows and exact fingerprints cannot score while other-library same-identity rows remain eligible.

Regime B — known structure/no spectrum: identity-wide local runtime enforcement pass

The rebuilt strict manifest is internally conserved and identity/scaffold provenance is persisted for the bounded PubChem cohort. The authoritative retrieval/evaluation path now loads frozen per-fold identity-wide masks and fails closed on row alignment, cohort leakage, and eligible-row counts. Future prototype, embedding, checkpoint, or ranker producers must carry the same fold mask and provenance; bounded PubChem coverage must remain distinct from full external coverage.

Regime C — novelty: proxy only

The scaffold-disjoint manifest is a useful bounded novelty proxy. It is not hidden Class 3 certification. External pretrained-model overlap/provenance is not established; any result involving such assets must remain provisional until audited.

Scorer contract

The local scorer tests pass for rank 1, rank 25, rank 26 cutoff, identity equivalence, constitutional isomers, deduplication, and MRR summary. Organizer tutorial evidence supports per-molecule aggregation, invalid-candidate dropping, first-block InChIKey identity, stereo-insensitive matching, identity deduplication, and the 25-distinct-candidate limit. It does not certify local tautomer canonicalization, MRR@25 arithmetic, malformed/null behavior, or official executable parity. The external scorer remains unavailable.

The current local validate_submission is intentionally structural: empty/null/invalid candidate fields are accepted as rows because the tutorial documents dropping invalid guesses. This behavior must be treated as local policy, not official parity, until organizer reference vectors or scorer confirmation are obtained.

Remaining priority order

1. P0: eliminate root legacy-index ambiguity; make all Phase 1 defaults point to the rebuilt namespace and add provenance assertions.

2. P0: freeze and enforce qualified Regime A query/reference masks.

3. P0: Regime-B identity-wide exclusion now passes for the authoritative retrieval/evaluation path; require the same policy when prototype, embedding, or derived-feature producers are added.

4. P0: run frozen-fold G1 metrics: MRR@25, Top-1, Top-5, Recall@25, candidate recall, pool coverage, conditional MRR, runtime, and peak memory.

5. P1: local NP count is pinned to 1,184 for the hashed current release; published 1,151 remains a documentation/version caveat.

6. P1: obtain official scorer parity evidence, especially tautomer handling and malformed/null/duplicate semantics.

7. P1: preserve Regime C as proxy-only and audit external-model/data overlap before reporting novelty results.

8. Blocked: unconditional official G0 closure until official scorer executable/API/hash or organizer reference vectors are available.

📑 Contents