User-Centric Research - Making Mobile UX Testing Mobile

November 26, 2012

Mobile devices, by definition, go everywhere users go. Depending on location, mobile contextual engagements are almost limitless. Yet, mobile users still expect the same robust experience they get from their desktop. With the global shift towards mobile, creating user experiences that meet these inherent challenges of limited real estate and unlimited contexts can seem daunting.  Addressing the changes in user expectations and behaviors specific to mobile makes UX research a bigger imperative than ever, and a much bigger challenge.

To meet these new challenges, current research paradigms need to be re-examined.   Firstly, it can no longer be assumed that user interactions with mobile devices can be measured in the same ways we measure desktop experiences. How often have you painstakingly set up a sled, or set up lab cameras just so, to make sure you capture user screen interactions, then heard participants say, “I never hold my phone this way”?   By assuming that user goals and interactions with devices are the same for everybody all the time, we as researchers hinder the contribution we make to mobile UX best practices. Instead of imposing controls on participants to fit testing equipment specifications, methodology should adapt to the variability of mobile experiences. 

While it is impossible to account for every context, mobile UX methodology can account for two main use cases: transactional and inspirational.  Impatiently attempting to get directions from Google maps before the traffic light changes from red to green is a typical transactional mobile use case.  Conversely, killing time on Zappos.com while sitting on the beach or waiting for the bus is inspirational. Each of these use cases involves very different user goals, attitudes, and behaviors.  While we of course should always adapt our research questions to fit our client’s business goals, when we apply methodology to mobile UX testing we need to take into consideration that, coupled with location, each of these two main use cases can greatly affect how users interact with their device.

At OHO, we recently began using a new mobile testing app called UX Recorder. With this app, we record screen interactions and participant facial expressions more naturally, since participants can hold the device any way they like. This freedom from the constraints of the lab also means we can test anywhere our client’s customers go, while taking advantage of all the equipment features mobile has to offer. By making mobile–testing mobile, we not only test contextually, we can also test connectivity to bridge devices such as phone versus stereo, TV, hands free devices, etc.

Of course, the equipment we use to test mobile user experiences is only one aspect of the refinement needed to further mobile UX methodology. As researchers, we should constantly strive to improve the ways in which we gather and report findings. Mobile UX research and reporting should be framed so that each new engagement informs the way we approach future projects–– as we develop new research paradigms we develop new design best practices. To do so, we need to not only adapt to the audiences whose behaviors and preferences we are testing, we need to understand and adapt to the stakeholders we are reporting to.

Since mobile computing is still relatively new, each testing engagement should be used as a means of refining methodology and reporting, and of building trust between researchers and stakeholders. One way of increasing stakeholder investment in research is to make reporting more interactive. Participant data can be shared directly with clients and developers via video presentations that tell user stories. By presenting findings as stories, recommendations can then be framed as dialogue between researchers and stakeholders. Rather than focusing on design shortcomings, discussion should be focused on improving mobile experiences, so that each new engagement adds value to future projects, and drives new best practices in research and design.

Related Reading

three people sit on a bench looking at their cell phones
Mobile Marketing for Higher Education

“Thanks to mobile, micro-moments can happen anytime, anywhere.

14 Digital Marketing Mistakes Explained in GIFs

Digital marketing is becoming an increasingly critical component of the overall business strategy of companies, big and small, across industries.

Why Today Is Already Too Late To Make Your Website Mobile-Friendly

If you’re still thinking about how to make your website more mobile-friendly, you better stop thinking and start doing beca

Top Problems Implementing iBeacon Marketing Solutions

We recently surveyed 35 of the top beacon CMS and hardware providers who are on the front lines of implementing this emerging technology and leading the push toward proximity marketing.