The Graded Merge Family: Separation, Overlap, and Dynamic Range in Hierarchical Sorting

Abstract

Picture a table with thirty-odd columns and millions of rows, and suppose you want a single fingerprint that stands in for each row. The difficulty is that the possible combinations exceed any counting — more than the atoms in the universe by a wide margin — so there is no indexing them and no assigning each one a computed number. The method begins from a simple observation: we do not need the exact value in each column, only its position. So for every column, sixteen ordered slices are drawn, and their boundaries are chosen so that the rows spread across them as evenly as possible — no slice crowded while another sits empty. These boundaries are computed once and stored, and the table is never reopened afterwards. Each column has now become a single question: which slice does this row fall into? Then the answers are combined. But naive addition destroys them: add one slice number to another and you can no longer tell which was which, exactly as the sum of two digits tells you nothing about the digits. So the numbers are spaced apart before they are added — each slice is multiplied by a step that keeps it clear of its neighbour, so nothing bleeds together when the addition happens. This is the same principle that keeps the tens place from being confused with the ones place in any number we write down. The operation repeats upward: two columns merge into a value, two values merge into a value, until one remains. And each time, what merges is not the raw value but its position separated from its remainder — where it sits, and how far it lies from the start of where it sits. The result is one integer a hundred and twenty-eight bits wide, four bits per column. It is totally ordered, so any ordinary search tree will index it. Its leading bits serve to distribute the data across servers. If two keys match, the two rows fall in the same slice in every column, without exception. And more to the point, you can read it backwards: from the key alone you know that this row sat in the seventh slice of the third column, and the twelfth of the tenth, and so on. No black box. All of this runs on binary search, addition, and a table lookup. No training, no randomness, no model to tune. The same row yields the same key today, in ten years, on any machine. Against that stands a clear limit that should not be blurred: the key tells you the slice, not the value. Retrieval is by range, not by number. Anyone wanting nearest neighbours under Euclidean distance is better served elsewhere. But anyone wanting a deterministic fingerprint they can explain, prove, and reproduce — this is where it belongs.