According to a much discussed argument, reliabilism is defective for making knowledge too easy to come by. In a recent paper, Weisberg aims to show that this argument relies on a type of reasoning that is rejectable on independent grounds. We argue that the blanket rejection that Weisberg recommends of this type of reasoning is both unwarranted and unwelcome. Drawing on an older discussion in the philosophy of science, we show that placing some relatively modest restrictions on the said type of reasoning suffices to block the anti-reliabilist argument.