What lies behind the data? How sampling assumptions shape and are shaped by inductive inference

Abstract

The problems of everyday cognition, from perception to social interaction and higher level reasoning, require us to predict future events and outcomes on the basis of past experience. But often (if not always) solutions to the problems we face are under-determined by our experience. So we reason inductively, drawing uncertain conclusions from incomplete information. Yet, despite our lack of first hand data, our reasoning is efficient and effective nonetheless. So how do we close the gap between the paucity of experience and the effectiveness of reason? One way that we do this is by exploiting statistical regularities that we have observed in the world, assuming (contra philosophers’ counsel) that these regularities will continue to hold. In so doing, we leverage the evidentiary value of the data that we do have. This thesis examines our assumptions about what lies beneath the data and how we leverage them to reason beyond it. In particular, it focuses on our mental models of the world – generative models that connect observations to hypotheses through their consequences. I consider the assumptions we make in solving three separate reasoning problems of increasing complexity. Firstly, in a series of related experiments I explore the effect of sampling assumptions in a categorisation task based on low-dimensional perceptual stimuli. Together, these experiments examine how reasoners weigh the value of extra data when deciding how far to generalise, and the extent to which the computations involved are influenced by their representational and sampling assumptions. In addition, I use the same experimental framework to investigate a related question: if people’s sampling assumptions do alter the weighing of evidence, at what stage do these effects manifest – during learning, or only at the point of generalisation? Secondly, I examine the role of sampling assumptions in the shift from percept to concept. A key challenge for the reasoner when reasoning from high-dimensional categorical stimuli is in deciding which of the many dimensions or features represent the appropriate basis for induction. I investigate how the perceived relevance of particular features in the data is affected by people’s assumptions about the representativeness of the sampling process. In almost every sphere of human activity, we reason from data generated by others and we generate data from which others will reason. Equipped with a theory of mind, both senders and receivers of data may exploit recursive “I think, you think, I think...” reasoning to increase the evidentiary weight of data, and improve the utility of communication as a result. But when data is highly leveraged in this way, there is a downside risk. If reciprocal assumptions are not well calibrated, the reasoner may leap to the wrong conclusion. In the final study, I investigate the phenomena of recursive meta-inference in a setting where deception is warranted but lying is not an option – a setting which offers particular advantages. Firstly, when perpetrating or avoiding a deception, some degree of meta-inferential assumption becomes a vital pre-requisite. Secondly, placing the goals of communicating parties at odds offers the potential to more easily distinguish whether people engage in genuine reflection about the assumptions of another or merely respond to constraints implicit in the sampling process. The studies described in this thesis deal with progressively more complex challenges that we face as reasoners: how far should we generalise when the basis of induction is clear, how do we determine the relevant basis for induction in the first place, and how do we calibrate our own inductive inference with that of another. Through a combination of computational modelling and human behavioural experiments I demonstrate how our sampling assumptions influence the way we meet these challenges, and how our solution to each challenge may be inter-related.