Label disagreement between topological materials databases is structured, directional, and survives curation

Abstract

High-throughput symmetry-indicator catalogues have labeled tens of thousands ofinorganic compounds by topological class, and machine-learning classifiers trained on theselabels now report accuracies above 90%. The labels, however, are protocol-dependentoutputs rather than measured properties, and the one published attempt to merge thetwo largest catalogues — Materiae and the Topological Materials Database — resolvedits cross-database label conflicts by deleting 212 compounds. Here we re-join completesnapshots of both databases directly on ICSD structure identifiers, one-to-many on bothsides, and audit the provenance of the published merged benchmark against the result. Wefind 1,610 conflicting materials — some seven times the number the merge removed ascross-database conflicts — of which 58.3% disagree on the binary trivial-versus-topologicaldistinction itself; the disagreement is directional, and the merge’s deletions inverted its sign,leaving the published dataset directionally biased relative to its own sources; structuralverification of a stratified sample of conflicts confirms the underlying identifier linkage forthe large majority while flagging roughly one in nine, bounding the corrected conflict ratebetween 7.4% and 8.29% of the comparable population; the archive’s provenance labelsare inconsistent for over a thousand rows even after every documented curation filter isaccounted for, understating the true overlap by 8.9%, and its headline cross-database testset is not disjoint from its training source, 1,455 of its 9,925 test materials being presentin the training source itself. We release the reconstructed conflict tables, the correctedprovenance assignments, and all acquisition and analysis code as a corrected foundationfor cross-database evaluation. These findings establish inter-database disagreement asstructured signal rather than noise, motivating classifiers that model per-source labelreliability during training — the program this diagnostic study enables.