Direct and Indirect Measures of
Assessment
University of North Carolina at Chapel Hill
Ways to Assess Student Learning?
Direct Measures
Means of assessing student learning which require
students to display their knowledge, skills, or thought
processes in an observable or tangible way.
Indirect Measures
Means of assessing student learning which ask students to
reflect on or self-assess their own knowledge, skills,
thought processes. For example, survey questions
generally entail self-reflection.
Palomba and Banta, 1999
Direct Measures
Evaluations of student learning based on observable displays of
knowledge or skills.
How do we observe knowledge or skills?
Presentations
Document Analysis (Reports, Blogs, Journals, 1 Minute-Paper, Photos, Videos)
Short Answer Quizzes & Pre-test/Post-test
Role Plays, Mock Scenarios, Case Studies, Interviews*
Portfolios, Projects, and Work Samples
Rubrics
Direct Measures
Strengths
Provide rich information in terms of depth and detail
Multidimensional (students can demonstrate competence in different ways)
Can be unobtrusive
Some data is readily available
Avoids survey fatigue
Can be difficult to quantify or norm data (rubrics)
Application is integrated with assessment
Process affords both teaching & learning opportunities
Challenges
Requires fair amount of planning ahead (outcomes, criteria, rubrics, training)
Consistency (who is/are evaluators, timeframe, training)
Can take significant time to gather information and analyze
Capacity building (writing learning outcomes, constructing/using rubrics)
Indirect Measures
Evaluations of student learning based on students
own perceptions of their knowledge or skills.
Surveys or questionnaires that focus on students perceptions
of their own learning.
Students select a response from a scale. (e.g., strongly agree, agree,
neither agree nor disagree, disagree, strongly disagree)
Focus groups or interviews* in which students reflect on
their own learning.
Indirect Measures
Strengths
data relatively quick to collect & analyze
can use existing data or piggyback
can potentially access large samples
minimal resources are needed
helpful in understanding perceptions, beliefs, thought processes
Challenges
survey fatigue & low response rates
limited in terms of what you can measure
requires capacity building in survey/question design/data analysis
can incur significant costs (testing services)
data relatively lacking in depth or detail
does not provide information regarding real world application skills
Activity #1
Direct or Indirect Measure?
Examples of Direct Measures in
Student Affairs
Direct
NSCPP- Retrospective Analysis- Student Leader Training,
Journaling, and Performance Review
Carolina Union Project Union- Supervisor feedback from
Rubric
University Career Services Pre and Post Assessment for
EDUC 131
Examples of Direct Measures in
Student Affairs
RHA HeelPrint Training Assessment
RHA One minute paper
EDUC 318 Peer Leadership Course Rubrics
Multicultural Advisors Pre and Post Tests
Examples of Indirect Measures in
Student Affairs
Indirect
CHS focus groups and satisfaction survey
Campus Recreation- Campus Impact Study (Benchmarking)
Project Union Event Evaluation from participants, team evaluation and
a post project survey after each event, and individual reflections.
HRE- Needs assessment of re-contracting of upper-class students on
south campus
HRE- Consortium for Resident Life (Benchmarking)
Examples of Indirect Measures in
Student Affairs
NSCPP - The New Student Experience: Review and Analysis
across Four Years
DOS- tracking categories of students served, student issues,
etc.
Student Wellness National Collegiate Health Assessment
(Benchmarking)
Accessibility Resources Satisfaction with Test Proctoring by
students and faculty
Examples of Indirect Measures in
Student Affairs
CAPS- Bounce Back Program
Campus Recreation Program evaluations
FSLCI Recruitment feedback
Interactive Theater Evaluations
One Act and Haven Training Post Assessment
LeaderShape Post Assessment
Selecting a Method
Is your outcome
manageable, meaningful, and measureable?
Before Choosing a Method
Make sure you have a program that can actually deliver
the outcome.
Think about what meeting the outcome looks like.
Describe how your program is delivering the outcome.
Think about collecting data.
Choosing an Assessment
Tool/Measurement
It is important to choose tools based on what you are
trying to assess, not on what tool is most appealing to
you i.e. survey!
Consider what will provide you with information to
make decisions.
Be able to justify your choice of tool and method.
Choosing an Instrument
What outcomes are you measuring?
What criteria will determine if the outcome is met?
Who is being assessed? How often do I have access to them?
Do I know who they are?
What is my budget? Timeline?
What type of data is most meaningful to me: direct/indirect
and qualitative/quantitative?
Who will analyze the data and how?
Activity #2
Choose an Outcome & Work through the Process
How is this outcome delivered? (program)
What does a student who meets this outcome look or act like?
What does a student who meets this outcome able to know or
do? How will you know when the outcome has been reached?
How will the outcome be measured? (method)
Are their opportunities during the program to capture data?
Direct, Indirect, Qualitative, Quantitative
Rubrics, Surveys, Pre/Posttest, Focus Groups, Minute Papers, etc.
Will this data demonstrate student learning? And that I have
met my outcome?
Will this kind of evidence help me make the decisions I need to
make?
Take Away Messages
You do not have to assess everything you do every year.
You dont have to do everything at once-start with 2 or 3
learning outcomes.
Think baby steps.
Be flexible.
Acknowledge and use what you have already done.
Borrow examples from other institutions to modify as
appropriate.
