A new model for inductive inference
Ronald L. Rivest, Robert H. Sloan · 1988
We introduce a new model for inductive inference, by combining a Bayesian approach for representing the current state of knowledge with a simple model for the computational cost of making predictions from theories. We investigate the optimization problem: how should a scientist divide his time between doing experiments and deducing predictions for promising theories. We propose an answer to this question, as a function of the relative costs of making predictions versus performing experiments. We believe our model captures many of the qualitative characteristics of "real " science. We believe that this model makes two important contributions. First, it allows us to study how a scientist might go about acquiring knowledge in a world where (as in real life) there are costs associated with both performing experiments and with computing the predictions of various theories. This model also lays the groundwork for a rigorous treatment of a machineimplementable notion of "subjective probability". Subjective probability is at the heart of probability theory [5]. Previous treatments have not been able to handle the difficulty that subjective probabilities can change as the result of "pure thinking"; our model captures this (and other effects) in a realistic manner. In addition, we begin to provide an answer to the question of how to trade-off "thinking " versus "doing"--a question that is fundamental for computers that must exist in the world and learn from their experience.