The study participants evaluated the early prototype by doing many of their everyday Gmail tasks, such as reading, sending, and replying to emails. We then looked at how easy or difficult it was to complete those tasks, and made changes based on this feedback. For example, one of the things we found with our prototype was that we had put too much emphasis on conversation level actions at the expense of per-message actions. As a result, our study participants had difficulty finding the reply button on each message. In response, we changed its appearance, size, and location to make the reply button easier to see.

As we continued to develop the new look, we evaluated our progress through additional usability studies with even more people. Much of what we captured from these studies was users’ first reactions to the new look. But Gmail is something people use repeatedly, not just once. So in addition to first impressions, we were also interested in seeing how people adjusted to the new look as they used it repeatedly in their daily lives. To find that out, we conducted a different kind of study called a longitudinal study. Longitudinal studies are used to observe the longer term effects our products have on people’s usage.

The longitudinal study consisted of turning on the new look for a group of Gmail users. We captured their initial reactions after their first experience. Then we let them use the new look in Gmail as they normally would as part of their everyday lives. As the days and weeks passed by, we periodically checked with them to see how they were adjusting to all the changes. Like with anything new, there were some changes that our participants initially needed time to adjust to, but later came to prefer as they used the new look more. On the other hand, problems that were not seen during the first couple times of use later emerged after more prolonged use. For example, many of the changes we made to the new left navigation were the result of people reporting their repeated experiences using labels and the chat area over time.

These studies have been absolutely critical in helping us build Gmail’s new look. Much of how it looks and behaves is a result of people participating in these studies and giving us their feedback. If you are interested in becoming a participant in a research study about Gmail or any of Google’s products, you can sign up at google.com/usability.