This chapter aims to defend the thesis that we can only perceive what we understand. Such a theory would seem to be unable to account for our learning to perceive what we do not yet understand. To address this objection, the paper presents a non-representationalist, direct realist theory of perception. In this, the sensorimotor theory of Noë and O'Regan plays a crucial role (although one important modification to the interpretation of that theory is proposed). The result is an account of how we are in contact with the world itself during perceptual experi- ence; and this leads to an account of how the world itself guides our understanding, as we move from non-sense to sense.