CORE Metadata, citation and similar papers at core.ac.

uk
Provided by AIS Electronic Library (AISeL)

Association for Information Systems

AIS Electronic Library (AISeL)
Special Interest Group on Human-Computer
SIGHCI 2016 Proceedings
Interaction

12-11-2016

InfoVis: The Impact of Information Overload on

Decision Making Outcome in High Complexity
Settings
Lisa Falschlunger
University of Applied Sciences Upper Austria, [email protected]

Othmar Lehner
University of Oxford, [email protected]

Horst Treiblmaier
Modul University Vienna, [email protected]

Follow this and additional works at: http://aisel.aisnet.org/sighci2016

Recommended Citation
Falschlunger, Lisa; Lehner, Othmar; and Treiblmaier, Horst, "InfoVis: The Impact of Information Overload on Decision Making
Outcome in High Complexity Settings" (2016). SIGHCI 2016 Proceedings. 3.
http://aisel.aisnet.org/sighci2016/3

This material is brought to you by the Special Interest Group on Human-Computer Interaction at AIS Electronic Library (AISeL). It has been accepted
for inclusion in SIGHCI 2016 Proceedings by an authorized administrator of AIS Electronic Library (AISeL). For more information, please contact
[email protected].
Falschlunger et al. InfoVis: The Impact of Information Overload

InfoVis: The Impact of Information Overload on Decision

Making Outcome in High Complexity Settings

Lisa Falschlunger Othmar Lehner Horst Treiblmaier

University of Applied Sciences Said Business School, Modul University Vienna
Upper Austria University of Oxford [email protected]
[email protected] [email protected]
operations (Huang et al., 2009). Extensive research has
ABSTRACT
been conducted on the applicability of tables and graphs
Dealing with an ever increasing amount of information is but no empirical evidence can be found on whether one
a major challenge in decision making. This especially outperforms the other, a quest which the theory of
pertains to information overload in managers, which is cognitive fit is trying to answer (Vessey, 1991).
associated with impeding cognition and thus impairs
objective decision making. Using visualizations to Cognitive fit theory investigates how information needs to
mitigate this effect has been widely discussed as a be presented in order to reduce cognitive load and thus
potential countermeasure. The theory of cognitive fit is enhance efficiency and effectiveness of decision making.
far from being consistent or holistic when it comes to This theory suggests that a fit between visual
explaining information overload and leaves ample room representations and tasks needs to be established (Vessey,
for data driven advancements. In this paper we thus report 1991). If a presentation is congruent with existing
the results of an experiment utilizing eye tracking that schemas (i.e., knowledge structures) stored in long term
investigates how information overload alters the memory, decision makers can more easily process
relationship between task complexity and decision information (Chandra and Krovi, 1999) and also higher
making outcome. It is shown that information overload amounts thereof. The theory recommends using tables to
acts as a mediating variable between task complexity and achieve better results for symbolic tasks (e.g., the search
decision making outcome and that it occurs less often for a specific value) and graphs for spatial tasks (e.g.,
when graphs instead of tables are being used. This also identification of trends, patterns or sequences) (Vessey,
improves decision making outcome. 1991).
In this study we investigate high levels of data density and
Keywords high levels of task complexity and their implications on
Information Visualization, Information Overload, Task mental models. Especially such models are rather
Complexity, Data Density, Structural Equation Model unexplored as the theory of cognitive fit does not account
for such subtleties in the theoretical model (Speier, 2006).
INTRODUCTION We propose that using graphs instead of tables in cases of
high data density and high task complexity better supports
Technology allows the collection of data from hitherto decision makers irrespective of the task type. We assume
untapped sources in a much more efficient way, and, as a that (1) visual representations of data (i.e., graphs) trigger
consequence, the amount of available data has increased processing with the human visual system, which operates
tremendously, bringing with it issues of complexity and faster and with less constraints in comparison to plain text
ambiguity (Lemieux et al., 2014). Former research shows or plain numbers, (2) the increasing use of graphs
that decision making quality is enhanced by an increase of presuppose the better applicability of graphs
information (Schroder et al., 1967). However, a turning (Falschlunger et al., 2015) and (3) using graphs enhances
point exists where adding further information deteriorates human processing capabilities irrespective of the data
decision making quality because the amount information presented because it reduces the demand on working
surpasses the cognitive abilities of decision makers memory (Huang et al., 2009).
(Tortaso-Edo et al., 2014). The benefits of a high volume
of information are often outweighed by the occurrence of HYPOTHESIS DEVELOPMENT
information overload (Lurie and Mason, 2007).
In this part of the paper we use the two variables task
A widely accepted method to assist managers in complexity and information overload, and put them in a
understanding high volumes of data is the use of tables new perspective when applying cognitive fit theory:
and graphs. They provide cognitive support by
representing information visually and transforming Task complexity is used as an independent variable,
laborious cognitive processes into few simple perceptual (instead of a moderator or mediator) representing a

Proceedings of the Fifteenth Annual Pre-ICIS Workshop on HCI Research in MIS, Dublin, Ireland, December 11, 2016
1
Falschlunger et al. InfoVis: The Impact of Information Overload

combined measure for visualization type, data density and H3: Information Overload is moderated by working
task type. To understand the complexity and applicability memory capacity of the decision maker.
of certain information presentation modes which support By using these two variables and by introducing high task
decision making these three influential factors need to be complexity levels we focus our attention on the
considered and aligned (Speier, 2006; Wood, 1986). The applicability of graphs. Tables and graphs are often seen
most useful concept for task complexity with respect to as competing modes of presenting information (Chan,
these requirements was introduced by Wood (1986), who 2001). However, recent studies focus on graphs rather
presented a formula to objectively calculate task than tables because visual representations are supposed to
complexity (TC = αTC1+βTC2+γTC3). It is defined as “(1) further reduce the demand on working memory because
a function of the number of distinct information cues that the visual system operates in a highly automated manner
must be processed (TC1); (2) the number of distinct (Mostyn, 2012). “Our senses, particularly the visual
processes that must be executed (TC2); and (3) the sense, are able to handle a huge amount of input, and to
relationship (i.e., interdependence and change over time) identify significant patterns within it” (Bawden and
between the cues and processes (TC3)” (Speier et al., Robinson, 2009, p. 180-191). Visualizations allow
2003; p. 1117). patterns to materialize and focus the attention on specific
Only when task complexity is low, the probability of aspects of the data depending on the visualization type
achieving cognitive fit and a high-quality decision- being used (Lemieux et al., 2014). They boost
making outcome is high (Falschlunger et al., 2016; information processing by relying on the human
Chandra and Krovi, 1999). This formula accounts for perceptual system which is highly developed and allows
information cues (dependent on data density levels) and multiple processes to be executed at the same time
behavioral acts necessary to fulfill a task dependent on the (Mostyn, 2012). Visual representations are therefore often
task type and the type of information presentation mode. said to be an “external memory to reduce demand on
For instance, behavioral acts change when comparing human memory” (Huang et al., 2009, p. 139). We
tables to graphs, when comparing different graph types, hypothesize:
and also when comparing various task types. In our H4: The information presentation mode moderates the
research, therefore the focus shifts away from matching effects of the relationships between task complexity,
task types and visualization types, to receiving the best information overload, and decision making outcome.
possible outcome on the objective task complexity
measure. RESEARCH METHODOLOGY
H1: High task complexity reduces decision making We used a laboratory experiment with a 4x3x3 within-
outcome. subjects and between-subjects design. Within-subject
The second variable of interest is information overload. effects were tested with 4 different visualization types
Hwang and Lin (1999, p. 213-218) describe information (two graphical designs as well as two tabular designs) and
overload as an inverted u-curve function based on the 3 different task types with various complexity levels
original work of Schroder et al. (1967): “... in response to (accumulation, recognition, estimation/projection).
increases in information load, decision makers will Between-subjects effects were tested with 3 different
increase their information processing initially. However, levels of data density (dashboard with 3 key performance
if the information load keeps increasing and finally indicators (KPIs); dashboard with 2 KPIs; dashboard with
exceeds the capacity of decision makers, information 1 KPI). In total, 2,184 individual observations were
processing will cease being increased. Instead, decision gathered using validated questionnaires and an eye
makers will decrease information processing as they tracking study.
experience a phenomenon termed ‘information
overload’.” As soon as the amount of information exceeds Subjects and Design Setup
working memory capacity limits, biased or even irrational 91 international students participated in this experiment,
decisions can be the consequence (Tortaso-Edo et al., who either received credit points or a small financial
2014). Each individual has a different working memory compensation. Participants were randomly assigned into
capacity which depends on experience, knowledge and two out of three sessions. Only two levels of data were
the respective topic. Information overload occurs if tested with the same participant due to time constraints.
working memory capacity is outreached. A high task The 91 students were recruited from a university with a
complexity, caused by performing highly complex tasks focus on business and economics thus representing the
and/or using huge amounts of data, increases the future target audience of financial reports.
likelihood of information overload (Mostyn, 2012). This
indicates a mediating effect of information overload on For measuring working memory capacity a computer-
the relationship of task complexity and decision making based test using E-Prime 2.0 was conducted (Foster et al.,
outcomes. We therefore conclude: 2015). Data was collected at two different points in time.
First, we measured working memory capacity and
H2: Information Overload mediates the relationship gathered demographical information. Second, the actual
between task complexity and decision making outcome. eye tracking experiment was carried out. Data collection
Proceedings of the Fifteenth Annual Pre-ICIS Workshop on HCI Research in MIS, Dublin, Ireland, December 11, 2016
2
Falschlunger et al. InfoVis: The Impact of Information Overload

was done on an individual basis (eye-tracking sample rate setting. A score of 1 indicates that both answers were
120 Hz, experimental software: SMI Experiment Center correct, 0.5 was given for 1 correct answer and 0 in case
3.6; analysis software: SMI BeGaze 3.6). The participants both answers were incorrect. RT was measured as the
were not allowed to use external devices for solving the time span between stimuli onset and offset. As a low
tasks and no feedback on the accuracy of the task or their response time is favorable we reversed the scale for RT.
performance efficiency was given during the session. For
Information Overload (IO) – formative construct: Pupil
the experimental tasks participants slipped into the role of
size and fixation count are used to measure cognitive
the CEO of a fictitious company. The tasks the
load. A high pupil diameter and a high amount of
participants had to perform and the decisions they had to
fixations indicate high cognitive load (Granholm et al.,
make were common in business. In total, the participants
1996). To measure pupil diameter, controlled lighting set-
had to answer 48 questions. One individual session took
tings are of utmost importance. Our laboratory did only
from 15 to 30 minutes.
have artificial light which was positioned in an angle that
caused minimal reflections in glasses or contact lenses.
Experimental Design
When measuring pupil diameter possible confounds exist.
Figure 1 shows our final research model and in the The unaffected pupil diameter is different for each person
following paragraphs we briefly describe the and an initial high or low diameter results in a lower
manipulations used to change task complexity levels. Due variance than a medium-sized pupil. Furthermore,
to space constraints the test material is not provided in different levels of screen brightness cause variations that
this paper, but can be downloaded on the main author’s do not relate to differences in cognitive load. To account
web site. for these changes in pupil diameter we introduced two
control variables, namely pupil diameter difference and
Visualization Type Manipulation: The most frequently
color intensity. The tracking ratio per stimulus, which is
used graphs in business communication are bar, line, and
the time being recorded by the eye tracking system
pie charts. Although the use of graphical aids is gaining
divided by the time of the stimulus, needed to be above
importance, tables are still the preferred mode
95% to be included in further data analyses. In the case of
(Falschlunger et al., 2015). In our study we used a
missing or excluded values mean replacement was used.
dashboard with two graphical and two tabular layouts for
each business line. All four visualization types were used Working Memory Capacity (WMC) – formative construct:
within one session and each business line was presented Working memory capacity is used as a moderator for the
differently and in random order. relationship between task complexity and information
overload. We used the shortened automated operating
Data Density Manipulation: Three different data density
span and symmetry span tests. Combining blocks of
levels were tested: high (3 KPIs within one dashboard;
operation and symmetry span is recommended since
180 distinct data points), medium (2 KPIs; 120 distinct
multiple indicators should be used to draw conclusions
data points), and low (1 KPI; 60 distinct data points).
about working memory capacity (Foster et al., 2015).
Task Complexity Manipulation: Task types were applied These two tests helped us to draw conclusions about
in accordance with Hard and Vanecek (1991) resulting in spatial and mathematical thinking. The operation span
three different complexity levels for each type of infers the cognitive ability in a test where one has to
dashboard: (1) Recalling, reading, retrieving of one item remember letters and at the same time calculate math
of information (e.g., How high was the actual throughput- problems. The symmetry span tests the ability to recall
time in August?); (2) Comparing several items of colored areas on the screen while simultaneously judging
information, recognizing patterns (e.g., Which month had the symmetry of figures. These two different procedures
the highest negative deviation between the actual value allowed us to draw conclusions about the applicability of
and the budget?); (3) Retrieving multiple items of symbolic and spatial information visualization types from
information, developing trends (e.g., When you compare a working memory capacity perspective.
the actual customer satisfaction index with its outlook,
Information Presentation Mode (IPM) – formative
which statement is true? (multiple choices were given)).
construct: Information presentation mode is a
Measurement models
dichotomous variable based on the used visualization type
manipulation (table or graph) splitting the dataset for
Task Complexity (TC) – reflective construct: Task evaluation in half (1,092 observations per IPM).
complexity was operationalized using the formula of
Wood (1986) shown above. RESULTS
Decision Making Outcome (DMO) – formative construct: For data analysis we used SmartPLS3 which is based on
Decision making outcome was measured in two ways: partial least squares modeling. PLS is able to
response accuracy (RA) and response time (RT). RA simultaneously estimate all the proposed relationships
accuracy pertains to the correct completion of a task. The while taking the existence of measurement error into
scores for accumulation, recognition and estimation account and it allows for a complete representation of the
included two tasks with the same complexity level in each influences (Hair et al., 2012). We chose PLS SEM over

Proceedings of the Fifteenth Annual Pre-ICIS Workshop on HCI Research in MIS, Dublin, Ireland, December 11, 2016
3
Falschlunger et al. InfoVis: The Impact of Information Overload

CBS SEM because formative and reflective

measurements are used, asking for a composite factor
model. Model fit is measured by SRMR (standardized
root mean square residual) which is 0.077 for our model
and below the recommended upper threshold of 0.08
(Henseler et al., 2014).
Table 1 shows that hypotheses 1-3 are corroborated. Task
complexity significantly impacts decision making Figure 1: Decision Accuracy Model
(IPM: Information presentation mode; WMC: Working memory capacity; IO:
outcome (H1, p<0.05). Additionally, this relationship is Information Overload; TC: Task Complexity; DMO: Decision Making Outcome)
mediated by information overload (H2, p<0.05) and the
dataset shows an indication for information overload to be DISCUSSION
moderated by working memory capacity (H3, p<0.1). The
This paper shows that individual differences result in
bootstrapping results are shown in Table 1.
varying information overload levels, which have to be
Original Standar T-Value P-Values
considered when testing for cognitive fit. Our findings
Sample d Error add to the ongoing discussion of information visualization
TC => DMO -0.200 0.018 11.303 0.000
as well as to the decision making literature which
(accept H1) focusses on information processing of high amounts of
TC=>IO 0.622 0.019 32.228 0.000 data. Our model predicts information overload and
IO=>DMO -0.726 0.029 25.431 0.000 decision making outcome independently while it
(accept H2) acknowledges and considers their interrelatedness.
WMC=> IO -0.086 0.030 2.836 0.005
Moderating Effect -0.057 0.031 1.814 0.070 Combining various eye tracking measures such as pupil
(accept H3) dilation and fixation count, standardized span tests as well
as questionnaire based research, our model has
Table 1: Bootstrapping Results
considerable power in explaining and predicting decision
Decision making outcome has an R² of 0.726 (R² making outcome (R²: 0.709). This paper shows
adjusted: 0.726) indicating that task complexity and shortcomings of the theory of cognitive fit which suggests
information overload are the main drivers of DMO. The using spatial modes for spatial tasks and symbolic modes
model’s out of sample predictive power (or the predictive for symbolic tasks respectively. It presents an alternative
power) was measured using the blindfolding routine method using the task complexity formula introduced by
(0.391 for DMO). VIF, HTMT as well as AVE were well Wood (1986) and accounting for individual differences by
within the proposed thresholds proving construct taking working memory capacity into account. We show
reliability and discriminant validity. that information overload mediates the relationship
between task complexity and decision making outcome.
Multigroup analysis was used to test H4. Splitting the data
into the two fundamental presentation formats graphs and In line with previous research this experimental study
tables makes it possible to test for the moderating role of highlights visualizations as a presentation mode that
visualization mode. A multigroup comparison based on supports the comprehension of large amounts of
the Welch-Satterthwaite test was used and the results are information, and that enhances the ability of humans to
shown in Table 2. detect patterns, trends, and sequences (Gettinger et al.,
2012). Therefore, an effective and efficient visualization
(accept H4) Ip(1)-p(2)I T-Statistics P-Values of information can be seen as a precondition and a
TC => DMO 0.058 1.660 0.097 possible first step in reducing cognitive load and
TC=>IO 0.088 2.240 0.025
IO=>DMO 0.000 0.009 0.993 enhancing decision making quality (Dilla et al., 2010).
WMC=> IO 0.028 0.404 0.686
Moderating Effect 0.034 0.438 0.661 Limitations
Table 2: Multi-Group Analysis (tables vs. graphs) Limitations of this study include a potential sample bias
due to a population solely based on students. However,
When task complexity increases the dataset indicates a
this population includes extra-occupational as well as full
higher effect when decisions are based on tables.
time students of various age groups, varying work
Additionally a significant effect can be identified when it
comes to the occurrence of information overload (H4, experience and different cultural backgrounds.
Additionally, the amount of information and data cues in
p<0.05). Tables seem to trigger information overload
our study might not have induced a state of information
more frequently than graphs. The higher the complexity
overload in all participants. Looking at the nature of these
of a task and the higher the data density, the more
important the use of graphs as decision aids is (see Figure limitations it is possible that actual effect sizes might be
even higher if more data sets or a higher task complexity
1).
were used. Furthermore, some limitations stem from
using eye tracking data: First, while eye tracking provides
a good approximation for cognition, it is not ensured that

Proceedings of the Fifteenth Annual Pre-ICIS Workshop on HCI Research in MIS, Dublin, Ireland, December 11, 2016
4
Falschlunger et al. InfoVis: The Impact of Information Overload

the perceived information is actually being processed Rönkkö and Evermann (2013), Organizational
(“eye-mind hypothesis” - Just and Carpenter, 1980); and, Research Methods, 17, 2, 182-209.
second, eye tracking cannot capture every occurring 12. Hwang, M. I., and Lin, J. W. (1999) Information
peripheral vision (Kim et al., 2012). dimension, information overload and decision quality,
Journal of Information Science, 25, 3, 213-218.
13. Huang, W., Eades, P., and Hong, S. H. (2009)
REFERENCES Measuring effectiveness of graph visualizations: A
1. Bawden, D., and Robinson, L. (2009) The dark side of cognitive load perspective, Information Visualization,
information: Overload, anxiety and other paradoxes 8, 3, 139-152.
and pathologies, Journal of Information Science, 35, 2, 14. Just, M. A., and Carpenter, P. A. (1980) A theory of
180-191. reading: From eye fixations to comprehension,
2. Chandra, A., and Krovi, R. (1999) Representational Psychological Review, 87, 329-355.
congruence and information retrieval: Towards an 15. Kim, S. H., Dong, Z., Xian, H., Upatising, B., and Yi,
extended model of cognitive fit, Decision Support J. S. (2012) Does an eye tracker tell the truth about
Systems, 25, 271-288. visualizations? Findings while investigating
3. Dilla, W., Janvrin, D. J., and Raschke, R. (2010) visualizations for decision making, IEEE Transactions
Interactive data visualization: New directions for on Visualization and Computer Graphics, 18, 2, 2421-
accounting information systems research, Journal of 2430.
Information Systems, 24, 2, 1-37. 16. Lemieux, V. L., Gromly, B., and Rowledge, L. (2014)
4. Falschlunger, L., Eisl, C., Losbichler, H., Greil, A. M. Meeting big data challenges with visual analytics,
(2015) Impression management in annual reports of the Records Management Journal, 24, 2, 122-141.
largest European companies: A longitudinal study on 17. Lurie, N. H., and Mason, C. H. (2007) Visual
graphical representations, Journal of Applied representation: Implications for decision making,
Accounting Research, 16, 3, 383-399. Journal of Marketing, 71, 1, 160-177.
5. Falschlunger, L., Lehner, O, Treiblmaier, H., and Eisl, 18. Mostyn, G. R. (2012) Cognitive load theory: What it is,
C. (2016) Visual Representation of In-formation as an why it’s important for accounting instruction and
Antecedent of Perceptive Efficiency: The Effect of research, Issues in Accounting Education, 27, 1, 227-
Experience, Proceedings of the 49th Hawaii 245.
International Conference on System Sciences (HICSS),
19. Schroder, H. M., Driver, M. J., and Streufert, S. (1967)
Kauai, USA, 5-9 January 2016, 868-877.
Human information processing: Individuals and groups
6. Foster, J. L., Shipstead, Z., Harrison, T. L., Hicks, K. functioning in complex social situations, New York,
L., Redick, T. S., and Engle, R. W. (2015) Shortened Holt, Rinehart and Winston.
complex span tasks can reliably measure working
20. Speier, C., Vessey, I., and Valacich, J. S. (2003) The
memory capacity, Memory & Cognition, 43, 2, 226-
effects of interruptions, task complexity, and
236.
information presentation on computer-supported
7. Gettinger, J., Koeszegi, T. S., and Schoop, M. (2012) decision-making performance, Decision Sciences, 34,
Shall we dance? The effect of information 4, 771-797.
presentations on negotiation processes and outcomes,
21. Speier, C. (2006) The influence of information
Decision Support Systems, 53, 1, 161-174.
presentation formats on complex task decision-making
8. Granholm, E., Asarnow, F., Sarkin, A. J., and Dykes, performance, International Journal of Human-
K. L. (1996) Pupillary responses index cognitive Computer Studies, 64, 11, 1115-1131.
resource limitations, Psychophysiology, 33, 457-461.
22. Tortaso-Edo, V., López-Navarro, M.A., Llorens-
9. Hair, J., Ringle, C. M., and Sarstedt, M. (2012) Monzonís, J., Rodríguez-Artola, R., M. (2014) The
Editorial - partial least squares: The better approach to antecedent role of personal environmental values in the
structural equation modeling?, Long Range Planning, relationships among trust in companies, information
45, 312-319. processing and risk perception, Journal of Risk
10. Hard, N. J., and Vanecek, M. T. 1991, “The Research, 17, 8, 1019-1035.
implications of task and format on the use of financial 23. Vessey, I. (1991) Cognitive fit: A theory-based analysis
information”, Journal of Information Systems, (5:2), of the graphs versus tables literature, Decision
pp. 35-49. Sciences, 22, 2, 219-240.
11. Henseler, J., Dijkstra, T. K., Sarstedt, M., Ringle, C. 24. Wood, R. E. (1986) Task complexity: Definition of the
M., Diamantopoulos, A., Straub, D. W., Ketcher Jr., D. construct, Organizational Behavior and Human
J., Hair, J. F., Hult, G. T., and Calantone, R., J. (2014) Decision Processes, 37, 1, 60-82.
Common beliefs and reality about PLS: Comments on

Proceedings of the Fifteenth Annual Pre-ICIS Workshop on HCI Research in MIS, Dublin, Ireland, December 11, 2016
5