Uncovering impactful solutions through user research

The purpose of user research is to understand the needs and behaviors of your users so you can develop a product or service that supports them. Combined with user testing, it helps ensure that your product or service meets the needs of your users.

Types of user research

There are two types of user research: generative, and evaluative.

Generative research helps you discover behaviors and problems in the current environment. This is the type of research you’ll use most when you first start a user research exercise. Examples of generative research include observations (i.e., watching the user do something) and user interviews.

Evaluative research takes place after you have a potential solution in mind. You use this type of research to pilot your solution with real users to see how it works. Examples of evaluative research include usability testing, where you watch a user try out your solution to see how they interact with it.

Types of usability testing

When conducting these two types of user research, be sure to apply some qualitative testing with users. As with the research types, there are two approaches here, each of which has its own unique benefits.

Unmoderated user testing. In unmoderated testing, your users all receive the same questions (think of it like a script). This approach works best with a large sample size where you can gather quantitative data to identify the challenges users face when navigating through your digital offerings.

Moderated user testing. In moderated testing, you have a moderator who can facilitate sessions to follow the needs and the questions of your testers in real time. This can help you discover more qualitative data from users, such as: what are your users thinking, feeling, and saying as they work with your digital content? This type of data can reveal themes and help you empathize with the people who will be accessing your content. Using this approach means you can also get a smaller group of people together and generate less paperwork, driving the goals of the Paperwork Reduction Act.

Remember that the American public is not necessarily your end customer; they might be someone from another agency or even within your own agency.

When conducting user testing, some important best practices to keep in mind include:

• Be empathetic to the needs of differently-abled users.
• Only ask questions that you really need answered. This shows respect for users’ time.
• Make it clear from the beginning that the tester is driving the session, and check in often to make sure that they are comfortable proceeding.
• Highlight that what they are sharing is valuable, and that they are in a safe environment, helping to inform trauma-responsive design of your digital content.

Guidance within policy

Recent memos from the Office of Management Budget (OMB) provide agencies with compliance and implementation guidance for two laws: Section 508 of the Rehabilitation Act of 1973 (as amended), and the 21st Century Integrated Digital Experience Act (21st Century IDEA).

Strengthening the federal government’s commitment to digital accessibility (M-24-08): Guidance for agencies to incorporate digital accessibility in their services, specifically building in guidelines and guardrails.

Delivering a digital-first public experience (M-23-22): Guidance for agencies to test their digital solutions with individuals who have disabilities and people using different forms of assistive technology.

Both policies are important to consider when conducting user research.

If you have questions about either memo, contact OMB’s Office of the Federal CIO at [email protected].

User testing resources

Utilize existing resources across the government for help with accessibility-focused user research journeys, such as identifying testers. There are online guides and communities to help you get started, like the usability testing card from 18F Methods and the Digital.gov User Experience Community.

Your own agency may even have its own guide to conducting user research. For example, the Department of Homeland Security published Basics of User Research: What is User Research and worked with Digital.gov on a training session, DHS Usability Testing Kit — So Easy, Anyone Can Do It.

Remember to use your contracts and associated documents — like a Quality Assurance Surveillance Plan (QASP) — to document accessibility requirements and identify the mandates in applicable laws and memos to ensure that contractor-driven developments meet accessibility needs.

Learn more in the 18F De-risking Guide: Sample Quality Assessment Surveillance Plan (QASP).

While there are automated tools available to help test systems for accessibility, they can only get you so far. A thoughtful plan of action for user research and testing is the only way to calculate for the human element and to design digital services that meet a variety of accessibility and inclusivity needs.