Penelope Maddy (Defending the axioms: on the philosophical foundation of set theory. Oxford University Press, New York, 2011) suggests a new naturalistic account of the objectivity of mathematics, grounding it on the fruitfulness of mathematics both internally, in mathematics itself, and externally, in the applications of mathematical concepts to empirical sciences. In light of the notion of mathematical depth, I will try to defend Maddy's epistemological project from Jeffrey Roland's criticisms concerning the reliability of mathematical beliefs in her account and Maddy's ontological agnosticism between Thin Realism and Arealism. Although Maddy's presentation of mathematical depth could profit from certain clarifications, I suggest that it could nevertheless ground an account of the reliability of mathematical beliefs, in response to the first objection. With regard to the second, I will argue that Roland's criticism is misplaced.