Evidence of intelligent neural control of human eyes

Nearly all imaginable human activities rest on a context-appropriate dynamic control of the flow of retinal data into the nervous system via eye movements. The brain’s task is to move the eyes so as to exert intelligent predictive control over the informational content of the retinal data stream. An intelligent oculomotor controller would first model future contingent upon each possible next action in the oculomotor repertoire, then rank-order the repertoire by assigning a value v(a,t) to each possible action a at each time t, and execute the oculomotor action with the highest predicted value each time. We present a striking evidence of such an intelligent neural control of human eyes in a laboratory task of visual search for a small target camouflaged by a natural-like stochastic texture, a task in which the value of fixating a given location naturally corresponds to the expected information gain about the unknown location of the target. Human searchers behave as if maintaining a map of beliefs (represented as probabilities) about the target location, updating their beliefs with visual data obtained on each fixation optimally using the Bayes Rule. On average, human eye movement patterns appear remarkably consistent with an intelligent strategy of moving eyes to maximize the expected information gain, but inconsistent with the strategy of always foveating the currently most likely location of the target (a prevalent intuition in the existing theories). We derive principled, simple, accurate, and robust mathematical formulas to compute belief and information value maps across the search area on each fixation (or time step). The formulas are exact expressions in the limiting cases of small amount of information extracted, which occurs when the number of potential target locations is infinite, or when the time step is vanishingly small (used for online control of fixation duration). Under these circumstances, the computation of information value map reduces to a linear filtering of beliefs on each time step, and beliefs can be maintained simply as running weighted averages. A model algorithm employing these simple computations captures many statistical properties of human eye movements in our search task.