Science is a constructed narrative of the natural world based on information gathering and its subsequent analysis. In this essay, we develop a novel approach to the epistemic foundations of the scientific narrative, as based on our experiential interactions with the natural world. We first review some of the basic aspects of both Bayesian statistics and Shannon's information theory as applied to the construction of meaningful conceptualization of the natural world. This conceptualization is rendered through the maps we construct of the world based on our limited knowledge of reality. We propose a path from experience to information and physics based on the notion that information is experience that induces change in an Epistemic Agent (EA): the change may be local and focused to a minor aspect of reality or it may be broad and worldview-changing. We illustrate our approach through an analysis of a measure of spatial complexity proposed by one of us called Configuration Entropy (CE), and establish a link between experience at the cognitive level and information content, showing that the CE is a quantitative measure of how much information in spatial-complexity the external world hides from an EA.