Consider a game that is played infinitely often by a set of boundedly rational players. At the beginning of each period, each agent has a belief about the strategies that his opponents will use in that period. These beliefs are derived by observing the play of the game so far, that is, they are conditioned on the information contained in the history of play together with initial conditions such as prior beliefs. We need not assume, however, that the posterior beliefs are derived via Bayes' rule (though they might be). We consider a much more general framework in which beliefs are determined by an arbitrary function that maps initial information and past history to present beliefs (as in Jordan, 1992). We suppose further that players are myopically rational in the sense that they choose a best reply given their beliefs in each period.