Frege claims that mathematical theories are collections of thoughts, and that scientific continuity turns on thought-identity. This essay explores the difficulties posed for this conception of mathematics by the conceptual development canonically involved in mathematical progress. The central difficulties are (i) that mathematical development often involves sufficient conceptual progress that mature versions of theories do not involve easily-recognizable synonyms of their earlier versions, and (ii) that the introduction of new elements in the domains of mathematical theories would seem to conflict with Frege's view that the original theories involved determinate reference. It is argued here that the difficulties apparently posed to Frege's central views stem from an overly-simple view of Frege's understanding of mathematical objects and of reference. The positive view recommended is one on which Frege's view of mathematical theories is largely consistent with, and helps make sense of, the phenomenon of theoretical unity across conceptual development.