1
Department of Neurophysics, Philipps-Universität Marburg, Marburg, Germany
2
Division of Biology and Division of Engineering and Applied Science, California Institute of Technology, Pasadena, CA, USA
3
Psychological Sciences, University of Melbourne, Parkville, VIC, Australia
The notion of “mind-reading” by carefully observing another individual’s physiological responses has recently become commonplace in popular culture, particularly in the context of brain imaging. The question remains, however, whether outwardly accessible physiological signals indeed betray a decision before a person voluntarily reports it. In one experiment we asked observers to push a button at any time during a 10-s period (“immediate overt response”). In a series of three additional experiments observers were asked to select one number from five sequentially presented digits but concealed their decision until the trial’s end (“covert choice”). In these experiments observers either had to choose the digit themselves under conditions of reward and no reward, or were instructed which digit to select via an external cue provided at the time of the digit presentation. In all cases pupil dilation alone predicted the choice (timing of button response or chosen digit, respectively). Consideration of the average pupil-dilation responses, across all experiments, showed that this prediction of timing was distinct from a general arousal or reward-anticipation response. Furthermore, the pupil dilation appeared to reflect the post-decisional consolidation of the selected outcome rather than the pre-decisional cognitive appraisal component of the decision. Given the tight link between pupil dilation and norepinephrine levels during constant illumination, our results have implications beyond the tantalizing mind-reading speculations. These findings suggest that similar noradrenergic mechanisms may underlie the consolidation of both overt and covert decisions.
In social and financial interactions, timing can be everything. Whether flirting at the bar or playing a game of poker, it is often considered strategic to conceal your choices with periods of apparent deliberation. The privacy of such mental events is generally taken for granted, although it may be violated by measuring the brain’s electrical or metabolic activity (Libet, et al., 1983
; Soon et al., 2008
). Here we investigate whether an outwardly visible measure, pupil dilation, can predict an individual’s decision before it is voluntarily revealed.
Pupillary responses were the focus of numerous psychophysiological studies throughout the 1960’s, 70’s and 80’s (for review see Loewenfeld, 1993
; Andrews and Purves, 1997
; Beatty and Lucero-Wagoner, 2000
). In these studies, the pupil response was considered to be an index of vaguely defined mental states grouped under general labels such as “arousal” (Bradshaw, 1967
; Yoss et al., 1970
) or “mental activity” (Hess and Polt, 1964
; Kahneman and Beatty, 1966
) that were routinely observed during a variety of cognitive tasks. The current study was inspired by more recent theories suggesting that Norepinephrine (NE) may play a specific role in consolidating behavioral decisions (Aston-Jones and Cohen, 2005
; Bouret and Sara, 2005
). Our aim was to show, for the first time, that similar processes may be important for the consolidation of cognitive decisions, using pupil dilation as a measure of NE release.
The neurotransmitter NE, released by the locus coeruleus (LC) and associated nuclei in the brain stem, has been implicated in a number of cognitive functions associated with memory and attention (for review see Sara, 2009
) and reward anticipation (Bouret and Richmond, 2009
). Specifically, it has been suggested that NE may play a critical role in the rapid consolidation of behavioral decisions (Aston-Jones and Cohen, 2005
; Bouret and Sara, 2005
). As exemplified by animal studies using go/no-go tasks, decision-making models have been specifically aimed at motor outputs and an animal’s capacity to rapidly optimize behavioral strategies with respect to reward outcomes (Aston-Jones and Cohen, 2005
; Bouret and Sara, 2005
). More recently, this work was extended to a series of human studies using pupil diameter as an index of NE release to explore the role of NE in behavioral outcomes associated with changes in task utility and reward (Gilzenrat et al., in press
). Extending this research on behavioural decisions, here we used the same measure of pupil dilation to explore whether NE may have an equivalent involvement in consolidating purely cognitive decisions.
In contrast to the pupillary light reflex, which is mediated by acetylcholine and the iris sphincter muscle (Loewenfeld, 1993
), the fluctuation of pupil diameter under constant illuminations appears to be mediated almost exclusively via NE (Koss, 1986
). While the exact path of innervation is not known, it is believed that NE originating from the lower medullary NE cell groups (Levitt and Moore, 1979
) indirectly stimulates the iris dilator muscles via projections from the Edinger-Westphal nucleus (Yoshitomi et al., 1985
). Electrophysiological studies in animals (Rajkowski et al., 1993
; Kalwani and Gold, 2008
) and a recent human fMRI study (Sterpenich et al., 2006
) provide further evidence for a tight association between LC activity and pupillary dilation. Based on this converging data, we use pupil dilation as a reliable and non-invasive proxy for NE release.
Here, human observers performed two basic tasks. In one case, participants pressed a button at a freely chosen time point during a 10-s period (experiment 1: “immediate overt response”). In the second case observers chose a 2-s interval within the same 10-s period, but concealed their choice until the trial’s end (experiments 2a,b: “covert digit choice” with and without reward). In a final condition, the free choice was replaced by an external cue (experiment 3: “instructed pick”). Using pupil-dilation to predict the timing of button press or the chosen interval allowed us to test the hypothesis that NE plays the same role in consolidating covert cognitive choices as it does in the decision to overtly execute motor acts. All versions of the task required minimal mental effort over the trial duration. This simple series of experiments therefore allowed us to identify decision-related pupil responses independent of more general influences of arousal, reward anticipation or task difficulty.
Setup
Throughout all experiments, participants’ pupil diameter of the left eye was measured at rates of 2000 Hz (experiments 1 and 2a) or 1000 Hz (experiments 2b and 3) by an Eyelink-2000 (SR Research, Mississauga, Ontario, Canada) infrared eye-tracking device. Stimuli were presented on a CRT monitor (19.7’ FlexScan F77S, EIZO, Hakusan, Ishikawa, Japan), located 48 cm from the observer, in an otherwise dark room. The background of the screen was a dark gray of 3 cd/m2. Fixation spots signaling trial progress had the same luminance as the background (3 cd/m2), but were either red or blue (CIE-coordinates (x/y): 0.610/0.339 and 0.151/0.065, respectively). The spots had a radius of 1.1° and were presented at the screen’s center. In all experiments, the eye-tracker’s calibration was validated by sequentially presenting spots at 10 random locations that observers had to fixate before each trial. If observers failed to fixate any of these spots within 5 s or after 30 trials had elapsed without re-calibration, the eye tracker was re-calibrated according to standard procedures recommended by its manufacturer. The validation procedure, in addition, served to keep participants engaged throughout. Stimulus presentation and data analysis was implemented in Matlab (MathWorks, Nattick, MA, USA) with its psychophysics and Eyelink toolbox extensions (Brainard, 1997
; Pelli, 1997
; Cornelissen et al., 2002
: http://pyschtoolbox.org
). All observers had normal or corrected-to-normal vision and gave written informed consent to participation. All procedures conformed with national and institutional guidelines and the Declaration of Helsinki. In total, 20 observers participated, 5 in each experiment.
Experiment 1 (immediate overt response)
In Experiment 1, each trial began with validation of the eye-tracker calibration (approximately 5 s) followed by a 5 s “pause” signaled by a blue fixation dot. The dot then turned red for a period of 10 s. Participants were instructed to decide, spontaneously, to push a button exactly once during this 10 s interval. They could win a reward (50 cents) if their response occurred during the “lucky” 1-s interval in this period. Observers were correctly informed that the lucky interval was chosen at random and there was no strategy to determine the lucky interval. Winning or losing was indicated by a smiling or frowning “smiley” figure at the end of the trial (Figure 1
A). Five male observers participated (ages: 21–27) in experiment 1 each performing 90 trials.
Figure 1. Paradigm. Schematic of a single trial, including the preceding validation/calibration period, the fixation period (blue circle), the actual trial (decision and response), and reward indication (details see Materials and Methods). (A) Experiment 1 (B) Experiment 2a. Time is running from bottom to top, display items are not to scale.
Experiment 2a (covert digit choice – with reward)
Experiment 2a (“covert digit choice – with reward”) involved a more cognitive task. Five digits were presented consecutively for 2 s each. During this 10 s presentation sequence, participants were asked to covertly choose exactly one of these five digits while it remained displayed. It is only after the sequence had finished, that observers were able to indicate their choice by pressing a key. In each trial, five digits between 0–9 were randomly selected (without repetition) and presented in ascending order. Providing only five randomly selected digits (rather than all 10) prevented participants from choosing a digit prior to the trial, as each digit appeared in only half of the trials. Presenting the digits in order and without repetitions reduced memory load and task difficulty. If observers hit the randomly defined “lucky” interval (digit), they won a reward (10 cents). In all other respects, experiment 2a was identical to experiment 1 (Figure 1
B). Five observers participated (Three male, Two female, ages 21–35) in experiment 2a, again with 90 trials per observer.
Experiment 2b (covert digit choice – no reward)
Experiment 2b was identical to experiment 2a with the exception that neither reward nor feedback was provided at the trial’s end. Five observers (Two male, Three female, ages 19–23) participated in experiment 2b with 90 trials per observer.
Experiment 3 (instructed pick)
To establish whether any observed pupilary responses reflected the pre-decisional cognitive appraisal component of the decision or the post-decisional consolidation of the selected outcome we performed an “instructed pick” experiment. This experiment was identical to experiment 2b, with the exception that the observer was not free to choose any of the five digits, but the digit to be chosen was indicated during its presentation. To avoid influences on luminance, the “chosen” digit was indicated by a line over it, while the other four digits were underlined. Five observers (One male, Four female, ages 20–26) participated in the experiment 3.
Data Analysis
Trials were excluded from analysis if either: (1) none or more than one button-press was recorded in experiment 1; (2) the observer had chosen a digit that had not been displayed in experiments 2a,b; or (3) selected a digit different from the one indicated in experiment 3. This left 444/450 valid trials in experiment 1, 441/450 in experiment 2a, 446/450 in experiment 2b, and 449/450 in experiment 3. Periods of blinks, in which no pupil dilation information was available, were detected by the Eyelink software. Pupil dilation during these periods was interpolated using cubic interpolation. Unless otherwise stated, data analysis was restricted to the 10-s decision period. To allow comparison across trials, observers and experiments, the pupil trace for each trial was normalized to zero mean and unit variance (z-score) in this period. Note that the data for this normalization were fully available at each decision period’s end, as the normalization did not use any across-trial information. The normalization, furthermore, did not affect the time of the maximum dilation on which the quantitative analysis was based. For visualization only, we subtracted the grand mean over all trials from the individual pupil-dilation traces (panels E through H of Figure 2
).
Figure 2. (A) Valid trials of experiment 1 binned into 2-s intervals based on button-press times; number of trials indicated at the lower right of each panel. Z-score normalization within each trial, but not across subjects or trials; colored lines: mean pupil-dilation averaged over trials with same chosen interval (color-coded); black lines: mean pupil-dilation trace over all valid trials. (B–D) Valid trials of the digit selection experiments, binned according to chosen interval. Notation as in panel A. (B) Experiment 2a (C) Experiment 2b (D) Experiment 3. (E–H) Mean pupil-dilation traces, binned as in (A–D) with grand averages (black lines of panel A–D) subtracted. (E) Experiment 1 (F) Experiment 2a (G) Experiment 2b (H) Experiment 3. Note that the strong reduction of the effect in experiment 2b is mainly a consequence of one observer with opposing trend (see P and text). (J–M) Time of maximum (non-normalized) pupil dilation grouped by chosen interval. Boxes denote quartiles, lines medians, whiskers the whole data range, except outliers beyond 1.5 times the interquartile range (crosses). (J) Experiment 1 (K) Experiment 2a (L) Experiment 2b (M) Experiment 3. (N–Q) Mean time of maximum (non-normalized) pupil dilation grouped by chosen interval, separated by individuals. Each individual encoded by distinct shade of gray. Note that the individuals do not overlap across experiments. (N) Experiment 1 (O) Experiment 2a (P) Experiment 2b (Q) Experiment 3.
For comparisons of experiment 1 to the other experiments, we binned the button-press times of experiment 1 into 2-s intervals, corresponding to the chosen intervals of experiments 2a,b and 3. For ease of notation we refer to both as “chosen intervals” throughout, and denote them by Roman numerals (I, II, III, IV, V).
To predict decisions based on pupil dilation, we used the time of maximum pupil dilation (tmax) within the decision period for each trial. This measure is robust, as it is insensitive to any within trial normalization. Furthermore, it was readily available at the end of the decision period, before the observers revealed their choice in experiments 2a,b. We analyzed the prediction in two ways. First, we performed ANOVAs to test whether tmax statistical depended on the chosen interval. Second, we used a signal-detection-theory measure to quantify how well tmax discriminated chosen intervals, i.e. to what extent statistical differences corresponded to a trial-by-trial predictability of an observer’s choice. For each pair of chosen intervals, we computed the receiver-operator-characteristics (ROC) for tmax discriminating the intervals. For quantification we used the area under the ROC curve (AUC). AUC is at 50% if discrimination is impossible based on tmax, and at 100% for perfect discrimination. We defined the AUC such that values larger than 50% consistently imply that a later tmax was associated with a later chosen interval.
Observers performed one of three tasks, either pressing a button volitionally once during a 10-s decision period (experiment 1), covertly choosing one out of five digits and reporting it only at the trial’s end (experiments 2a,b), or reporting a cued digit at the trial’s end (experiment 3). In all experiments the observer’s choice – time of button press or chosen digit – defined the “chosen interval” on which further analysis was based.
Pupil Dilation Pooled Over all Observers
Pupil dilation showed a general increase during the decision-period, which was independent of the chosen interval (black lines in Figures 2
A–D). As discussed below, this result was expected based on numerous studies showing pupil dilation during cognitively demanding tasks (Hess and Polt, 1964
; Kahneman and Beatty, 1966
; Simpson and Hale, 1969
). On top of this general increase in pupil dilation, however, there were clear increases specific to the chosen interval. In the chosen interval, mean pupil dilation consistently exceeded the average over all trials (colored lines in Figures 2
A–D). This became particularly evident, when – for visualization – the general signal was subtracted from the mean trace (Figures 2
E–H): pupil dilation showed an increase relative to the experiment’s average in all experiments.
Given this apparent time-locking of pupil response to decision timing, we were interested in determining whether the time of maximum dilation in the 10-s decision period could be used to predict the chosen interval in any given trial. Because the time of maximum pupil dilation computed from the raw (non-normalized) data was readily available at the end of the decision period (i.e., in experiments 2a,b and 3 prior to the report) this measure was used for trial-by-trial prediction. In experiment 1, the time of maximal dilation (tmax) depended significantly on the 2-s interval in which the button was pressed (Figure 2
E, F(4,439) = 12.04, p = 2.7 × 10−9). Pairwise significant differences (post-hoc Tukey-Kramer) existed between interval I versus all other intervals, II vs (IV,V)and III vs V. In experiment 2a, a similar dependence was found between chosen interval tmax (Figure 2
F, F(4,436) = 6.51, p = 4.3 × 10−5), with significant differences between the interval I versus intervals (IV,V), and II vs (IV, V). In experiment 2b, the variant of experiment 2a without reward, a similar – albeit slightly weaker – pattern was found [F(4,441) = 3.34; p = 0.01], with significant differences between interval II vs. (IV, V). Aggregating the two covert choice experiments (experiments 2a,b) with a total of 10 observers and 887 trials shows a significant effect [F(4,882) = 8.85, p = 5.3 × 10−7] with significant post-hoc differences between intervals I vs.(IV,V) and II vs. (IV,V). Finally, the “instructed pick” experiment yielded a strong dependence between “chosen” interval and tmax with F(4,444) = 13.49, p = 2.2 × 10−10 (Figure 2
H) and significant differences between intervals I vs. (III, IV, V), II vs. V and III vs. V. In summary, in all experimental conditions there is a significant dependence of tmax, the time of maximum pupil dilation, on the chosen interval, no matter, whether the response is immediate or concealed, in the absence or presence of reward or whether the choice is voluntary or instructed.
Inter-Individual Differences
So far, we have pooled the data in each experiment and neglected the fact that they stem from distinct observers. Similar results as in the pooled analysis were obtained when the factor observer was treated as fixed effect in a 2-factor ANOVA with factors observer and chosen interval (Table 1
, first rows of each experiment). When accounting for inter-individual variance by treating the factor observer as random effect, significance prevailed in all experiments, with the exception of considering experiment 2b in isolation and including second-order interactions in the model (Table 1
, second rows). Experiment 2b also shows a significant effect, when only main effects are considered (Table 1
, third rows) or the “outlier” observer (see below) is excluded (Table 1
, caption). In all experiments but experiment 1, there was a significant main effect of observer and no interaction between observers and chosen interval. This shows that, although tmax is a robust predictor of chosen interval, substantial individual differences between the exact timing of the pupil response relative to the choice exist.
Table 1. Overview over 2-factor ANOVA results.
To further visualize these differences, we plotted the mean tmax of each observer depending on chosen interval (Figure 2
, bottom row). Across all experiments observers exhibit a trend for a later tmax in case of later chosen intervals, although slopes and lags between chosen interval and peak of response vary among observers and across experiments. The sole exception to the general trend is found in one observer in experiment 2b, who exhibits an opposing trend (medium gray in Figure 2
P). This outlier is the main cause for the substantially weaker effect in the population data of experiment 2b as compared to experiment 2a. By excluding this outlier, the effect of chosen interval in the pooled analysis of experiment 2b (see Pupil dilation pooled over all observers) indeed becomes comparable in effect size to experiment 2a (p = 4.1 × 10−6, F(4,353) = 8.15), and also all other reported analyses become significant for experiment 2b in isolation (see captions of Tables 1 and 2
).
In summary, with the exception of the immediate response condition – there is substantial variation across individuals in the lag between chosen interval and time of peak pupil response. Given this inter-observer variability, it is even more remarkable that tmax is a comparably reliable predictor of the chosen interval in covert choice and instructed pick.
Pupil-Response Function
Comparing results from the two experiments, the effect in experiment 1 seems clearer than in the covert choice experiments (experiments 2a,b) and even slightly more pronounced than in the instructed pick condition of experiment 3. This may be attributed partly to the fact that participants in experiment 1 were more likely to press the button towards the beginning of the trial (number of trials per chosen interval see Figure 2
A through D), providing more data for early intervals. Furthermore, because the button response immediately signals the participant’s decision to act it may provide a stricter time locking of decision to pupil response than the digit selection condition. This interpretation is in line with the larger inter-subject variability in the covert choice experiments (see above). Alternatively, this result may reflect actual differences between the overt and covert decision process. Irrespective of the relative magnitude of the results, in all cases, a time-locking of pupil response to decision timing is evident. This result provides the first evidence suggesting that NE may play the same role in consolidating cognitive decisions as has previously been proposed to be involved in the rapid selection of competing motor acts.
In experiment 1, the binning of button-press times was arbitrary and done to ease comparison with the other experiments. Sorting the trials of experiment 1 by button-press times (Figure 3
A) allowed a more precise evaluation of timing of pupil dilation relative to decision making. This representation shows that the timing of maximum pupil-dilation (tmax) follows the button press generally by 1–2 s (red color to the right of the black line). Furthermore, tmax is indeed significantly correlated with the button press time (r = 0.30, p(443) = 1 × 10−10). This confirms the tight coupling between peak pupil-dilation and time of decision. Aligning the pupil-dilation traces of experiment 1 to the time of button press revealed the time-course of the average pupil response (Figure 3
B): about 1 s prior to the button press pupil dilation starts to rise, peaks at 420 ms after the response, and has relaxed back to baseline after about 2 s thereafter. For the covert conditions, comparable pupil-response kernels cannot be obtained, as the precise decision time has an uncertainty of 2 s (the duration of a digit). The representations in the top two rows of Figure 2
, however, present the best approximation when aligned by chosen interval. The fact that even the smeared kernel of experiment 1 (Figures 2
A,E) has a more pronounced peak than the analogous representations of the other experiments, is further evidence that the variability between the time of decision and the pupil response within and between observers is the main cause for the weaker (and more delayed) effects in the covert choice experiments. Nonetheless, as in the overt case of experiment 1, the peak succeeds the decision, it seems likely that this also holds for the covert cases. That is, although pupil dilation generally follows a decision with some lag, it nevertheless may precede and predict its volitional revelation.
Figure 3. (A) Valid trials of experiment 1 sorted by time of button press (black line) and presented in ascending order (rows); columns denote time since start of the decision period, colors the pupil dilation. For display purposes, pupil diameter was z-normalized within individual trials, but not across trials or subjects. The timing of maximum pupil dilation generally occurring 1–2 s after the button press (red color to the right of the black line), is unaffected by this normalization and thus corresponds to the data used for the main analysis and prediction. (B) Average pupil-dilation in experiment 1 aligned to time of button-press. Gray: raw (z-score normalized data); black: grand mean subtracted from each trial before aligning and averaging.
Trial-By-Trial Predictability
To test the extent to which these statistical differences corresponded to a trial-by-trial predictability of an observer’s choice, the area under the receiver-operator-characteristics (ROC) curve (AUC) was computed. This signal-detection-theory measure confirmed the statistical results, both pooled over observers and for individuals: the timing of peak pupil dilation discriminates the chosen intervals, in particular whether early or late intervals had been chosen (Figure 4
). When performing this analysis separately for each of the five individuals of experiment 2a, a mean AUC of 0.70 (SD: 0.11) was found for discriminating interval I from V. AUCs were larger than chance in all observers, and the mean was significantly so (t(4) = 4.05, p = 0.02, t-test). All pairwise discriminations, besides IV vs V, were above chance on average, with significant effects for I vs. IV [t(4) = 6.33, p = 0.003; AUC: 0.71 ± 0.08], I vs. III [t(4) = 8.70, p = 0.0001; 0.63 ± 0.03], II vs. V [t(4) = 4.44, p = 0.01; 0.65 ± 0.08] and II vs. IV [t(4) = 6.49, p = 0.03; 0.66 ± 0.05]. Similar results were obtained for the other experiments, in particular when discriminating the first (I) from the last (V) chosen interval. Again with the exception of the “outlier” observer in experiment 2b, all observers had ROCs substantially above chance (50%) being at least 62% (Table 2
). This implies that – given that either interval V or I is chosen – these intervals can be distinguished in 62% to 100% of the trials. This discrimination is done merely on the basis of a single measure tmax, which can be robustly measured, and without any individual-specific classifier training. These data show that our findings are not an effect of averaging over many observers, but hold for individuals and are informative on a trial-by-trial basis. Hence, pupil dilation is sufficiently powerful to consistently predict an individual’s choice before it is openly revealed. Furthermore, the fact that such inferences could be drawn using the same measure in all individuals and despite substantial inter-subject variability, illustrates the robustness of the reported effects.
Figure 4. Pairwise discriminability of chosen intervals based on the time of maximum pupil dilation. Values above 50% (chance) consistently indicate that the later chosen interval corresponds to later pupil dilation maxima, yielding a symmetric plot. (A) Experiment 1 (B) Experiment 2. The reduced discriminability between intervals IV and V in experiment 2 may be due to the latency of the maximum response, which is effectively truncated by the end of the 10-s decision period; it reaches 55% if an additional second is included (maximally up to the report). (C) Experiment 2b (D) Experiment 3.
Table 2. Trial-by-trial prediction for individuals.
Analysis of Choice Frequency
In experiment 1 we found a preference to press the button early in the 10 s (on average after 3.9 s ± 1.9 s), as one would expect for an experiment with a fixed deadline. To test whether any of the participants favored one, or a small set, of “preferred” digits in experiments 2a,b, we compared the frequency at which the most frequently chosen interval was chosen with the respective value expected from choices according to a uniform distribution. As the maximum is considered, this number falls slightly above the intuitive 20% or 10% for the selected interval or digit respectively. We obtained the expected value by simulating 106 draws of 90 numbers uniformly from [1,2,…,10] and [1,2,…,5] and measuring the incidence of the most frequently chosen number. This yielded expected values of 13.9/90 = 15.4% (uniformly chosen intervals) and 23.1/90 = 25.6% (uniformly chosen digits). Compared to this baseline, we found no evidence in experiments 2a,b that any of the participants strongly favored one or a small set of “preferred” digits. In experiment 2a, for each observer, the most frequently chosen digit was selected in 16.5% ± 3.5% of trials, resulting in the most frequently chosen interval to be picked in 25.4% ± 2.4% of trials. In experiment 2b, these values were 16.5% ± 3.5% (digit) and 27.1% ± 1.8% (interval). None of these numbers were significantly different from the 15.4% and 25.6% expected from a uniform distribution (experiment 2a digits: p = 0.51; experiment 2a intervals: p = 0.85; experiment 2b digits: p = 0.51; experiment 2b intervals: p = 0.13). There was some evidence of repetition avoidance with respect to the chosen digits: the same digit was chosen subsequently in 7.4% ± 4.3% (experiment 2a) and 2.3% ± 2.1% (experiment 2b) of trials, the latter being significantly (p = 0.001, t-test) below chance (10%). However, this did not yield a significant repetition avoidance with respect to chosen intervals (20.7% ± 4.9% in experiment 2a, 17.4% ± 4.5% in experiment 2b, p = 0.76 and p = 0.26). In experiments with reward (experiment 1 and 2a) we found no influence of the previous trial being rewarded on the subsequent trial, neither behaviorally nor with respect to any of the pupil analysis described above. It should be noted, however, that the fraction of rewarded trials (10% on average) was low and the delay between reward and onset of the subsequent decision period was substantial (about 12 s: reward indication, eye-tracker validation, fixation period). We are therefore confident, that there is no systematic preference for a given interval; in particular, we can rule out that the predictive value of the pupil dilation is an artifact arising from any such bias.
Here we show that pupil dilation can betray an individual’s decision before it is openly revealed. Pupil diameter was found to increase at the time of a decision irrespective of whether the choice related to the voluntary execution of a motor act or the selection of a digit from a series of five numbers.
In experiment 2 we disentangled the effect of the decision to act and the action itself by enforcing a delay between the two. Using the five digits as a “clock” we nonetheless gained access to the approximate timing of the covert decision. While the magnitude of responses were reduced in the covert tasks, the same overall pattern of results was observed for the covert cognitive decision as was seen in the execution of the overt motor act. This finding extends the proposed role of NE in behavioral decisions (Aston-Jones and Cohen, 2005
; Bouret and Sara, 2005
). Our results suggest that NE may serve the same function in situations requiring a purely cognitive decision (in this case the selection of an arbitrary digit symbol). This finding suggests that the applicability of these models may be extended from behavior to cognition, as in both cases, NE may help to rapidly consolidate the chosen selection. Taken together with the recent proposal that NE consolidates perceptual interpretations of ambiguous signals (Einhäuser et al., 2008
), we suggest a generic consolidating function of NE for situations of high uncertainty – in perception, cognition and behavior.
The time-locked pupil response observed in the current studies appears reminiscent of decades old research showing pupil dilation during tasks involving cognitive (Kahneman and Beatty, 1966
; Beatty and Lucero-Wagoner, 2000
) and attentional effort (Bradshaw, 1967
) or periods of emotional arousal (Sterpenich et al., 2006
). From inspection of the top panel of Figure 2
, it can be seen that the mean response (indicated by the black line) plateaus towards the middle of the trial. This general, and quickly saturating, increase of average pupil dilation during a decision period closely matches earlier data. Simpson and Hale (1969)
obtained results similar to the mean trace seen here, when they asked observers to decide, during a 7-s decision period, whether to press one of two levers directly after the completion of the decision period. Using an additional no-decision control, these authors convincingly showed that the bulk of the increase in pupil diameter is linked to the decision process itself. Their paradigm lacked, however, any possibility to narrow down the time-point of the actual decision further. Here, in contrast, we could demonstrate that the pupil response is specific to the particular choice (for example, the top panel of Figure 2
when the subject responds in the initial 2-s epoch), peaks during the corresponding time period and then subsequently decreases.
Given that NE, and the associated pupil response, is inextricably linked to general levels of arousal it is important to disentangle our claim of decision-specific effects of NE from these more generic examples of heightened arousal. Since the initial version of the covert choice task involved a reward payout at the end of each trial (experiment 2a) and it is known that reward expectancy leads to increased arousal (for review see Wise, 1978
; Weinshenker and Schroeder, 2007
), it may have been possible that our pupil dilation response reflected the subjects’ anticipation of the upcoming results. In order to explore this effect directly, we conducted a second version of the task without any reward or feedback (experiment 2b). In the absence of reward the same, though weaker, overall pattern was observed in both the group (Figure 2
L) and the individual data in all but one observer (Figure 2
P). In the instructed pick condition of experiment 3, we actually see stronger pupil responses compared to experiment 2a, despite the absence of reward in this condition as well. Together, these results show that reward anticipation is unlikely to explain the observed pupil dilation. We do not deny, however, that reward anticipation is likely to engage the same noradrenergic network and may underlie some proportion of the pupil response observed in the rewarded conditions (experiments 1 and 2a). The fact that the digit sequence always consisted of 5 digits randomly selected from a possible 10 digits together with the analysis of choice patterns, makes it unlikely that subjects pre-selected their favorite number and then became aroused or excited when the digit was presented on the screen. Finally, as the cognitive load is minimal, and identical across the duration of all trials, it is difficult to see how the pupil dilation could result from a temporary increase in cognitive or attentional demand associated with the task. For these reasons, we are confident that our result does not merely reflect general changes in reward-based arousal, mental effort or cognitive demand. However, we do not dispute the obviously close links between arousal and NE. Indeed, we believe that transient increases in arousal may play a central role in consolidating a decision by increasing the perceived salience of the winning selection and directing the individual’s attention away from the competing alternatives. Our results are thus consistent with the proposed role of NE in temporarily facilitating the neural networks supporting the winning selection (Aston-Jones and Cohen, 2005
; Bouret and Sara, 2005
) and show that this facilitation is irrespective of whether the decision is between abstract digit symbols or available motor contingencies. Importantly, our results from the instructed pick condition (experiment 3) are consistent with the proposed role of NE in the consolidation component of the decision, rather than the preceding appraisal component.
Independent of the neural mechanisms underlying our results, there are a number of important implications arising from the fact that we used pupil responses to predict an individual’s decision before it was revealed. Unlike previous reports of “mind reading” based on changes in the brain’s electrical or metabolic activity recorded using cumbersome equipment such as electroencephalography (EEG) (Libet et al., 1983
) or fMRI (Soon et al., 2008
) here we instead used a measure that is accessible to the naked eye and thus of potential relevance in social interactions. Arguably the best-known example for usage of pupil dilation as a social cue, is the fact that the natural source of pupil-dilating atropine is the plant commonly named “bella donna” (beautiful woman). Indeed, the pupil size in images of faces modulates several emotional judgments, brain activation as well as the observer’s own pupil dilation (Harrison et al., 2006
). It is, therefore, tempting to speculate that the use of pupil dilation to provide insights into a competitor’s strategic decision making processes may be one reason poker players like wearing sunglasses. One further implication of our study is that pupil dilation may provide a means of assessing mental function and communicating with non-responsive patients in a vegetative state or suffering from locked-in syndrome (Owen et al., 2006
). While the pupil responses observed were relatively small, for such applications, prediction performance could likely be improved further if inter-observer variability in the pupil response was accounted for, and – instead of using the same marker (tmax) for all observers – the system was trained for each individual to exploit idiosyncratic features of the pupil response function. Even if pupil dilation may never provide sufficient single trial reliability for complex communication it could nevertheless provide a measure of mental function. Given the accessibility of the signal, pupil dilation could thus complement the use of functional brain imaging methods currently being explored, which are similarly reliant on statistical inference.
In summary, the time locking of pupil dilation with cognitive decisions, provides evidence for the same involvement of NE in optimal cognitive performance as has previously been suggested for behavior (Aston-Jones and Cohen, 2005
; Bouret and Sara, 2005
). However, while there are converging links between pupil dilation and NE release (Rajkowski et al., 1993
; Sterpenich et al., 2006
; Kalwani and Gold, 2008
; Gilzenrat et al., in press
) direct physiological measurements of LC-NE activity in our task are needed to firmly establish this conclusion. Independent of the physiological basis, however, we have for the first time demonstrated the prediction of seemingly private decisions by a measure accessible to the naked eye.
This research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.
This research was funded by the Mathers Foundation to Christof Koch and NHMRC(Aust) grant-no.: 368525 to Olivia L. Carter.