While some user experience research (UXR) problems can be solved using one research method, others benefit from mixed-methods. In this approach, the data collected via one method complement the data collected via another method to provide a more comprehensive picture of customer or user needs. A mixed-methods design uses the best combination of data collection and analysis methods to answer a complex research question. These methods can be qualitative, quantitative, or a mix of both qualitative and quantitative methods. The key thing that distinguishes a mixed-methods approach from a random combination of research methods is that each method’s data can be triangulated with data from the other methods to fill any gaps that one method would have. Part of the reason mixed-methods approaches are not more common in UXR, or commercial research generally for that matter, is that mix-methods research is hard to do and can yield muddled insights if not done well. Even so, the insights gleaned and improvement to product or experience is worth the time and effort in many cases. The rest of this article highlights some case studies from my own research experience and then elucidates the value of mixed-methods approaches to solving UX and other business problems.
Case Studies
The following case studies illustrate how a mixed-methods research design can improve the experience of a service, technology, or program for its target audience.
Dietary Health Service Design Project in Honduras
Several years ago, I spent a year working in two communities in Roatán, Islas de la Bahia (Bay Islands), Honduras, on a service design project to improve access to food and dietary health for residents. The research design for addressing this problem blended an ethnographic methodology. It included immersive observation, semi-structured interviews, and contextual inquiry with quantitative data from a dietary diversity survey I created, a validated food insecurity survey, and body measurements such as height, weight, and skinfolds. This design triangulated quantitative data about food access and obesity with qualitative data about local people’s lived experience, mainly how they made a living and obtained food. With this blended approach, I was able to complement the extent of food insecurity and obesity in the two communities with rich narrative stories about how people made a living and the challenges they faced. Such a comprehensive picture would not have been possible using only qualitative or quantitative methods. I found that obesity was highly prevalent in both communities and that steady jobs in tourism were more stable and better paying than small-scale enterprises that depended on tourists’ tips. Based on these findings, I crafted recommendations to local leaders for programs that would make residents more competitive for tourism jobs and also programs that would give more reliable access to healthy foods for local households.
Water Quality Service Design Project in Costa Rica
For a service design project in the Monteverde Zone in Costa Rica, the team I was part of used a mixed-methods design to assess local perceptions of surface water quality in the local environment. The team used a demographic survey that included questions about perceptions of stream water quality and everyday pollutants, a semi-structured interview guide that allowed a smaller sample of respondents to delve into their perceptions of water quality and common pollutants more deeply, and detailed notes and photographs from immersive observations of streams in the area, the local water treatment plant, and a local community meeting. This approach allowed the team to combine thick data about water quality problems from interviews, insights from a survey about the general state of water quality and everyday pollutants, and vivid images of streams burdened with laundry detergent that had been flushed out of homes directly into the waterways. This project led to further research into water pollution and recommendations for improving surface water quality.
Tech Startup UXR Project
Of course, mixed-methods designs are a powerful way to answer research questions in the tech space. At Javelyn Technology, the team used interviews, surveys, and netnography in a discovery research project to conceptualize and inform the design of a property-tech application intended to help residents of large apartment complexes get to know their neighbors better. The team conducted the semi-structured interviews and surveys in the UK’s original target market, with the in-depth interview responses from a small number of respondents being used to refine the survey to uncover better local perceptions of trust and social capital that were crucial in the initial concept of the application. Later, when the company decided it would launch an MVP in a few apartment buildings catering to expatriate professionals from North America and Western Europe, the research team conducted a netnography, an ethnographic study adapted specifically to observing online interaction. In this case, we watched the feed of a few Dubai expatriate Facebook groups. Analysis of images and text from these feeds surfaced critical themes of interest to members of the expatriate groups yielded insights that were instrumental in creating features for the application. For this project, a mixed-methods approach allowed us to gain a more comprehensive understanding of potential users of the product and to pivot to a slightly different user group to adjust to evolving business needs.
Adapting Mixed Methods to UXR
While UXR projects have different time and budget constraints than other research types such as market research, program evaluations, or academic research, a mixed-methods approach to addressing business and research problems can be adapted to UXR projects. In some cases, the greater speed and lower cost of using only one method to collect user data might make more sense for a project. Still, often there is significant added value in collecting complementary data by triangulating multiple methods. Mixed-methods approaches can produce comprehensive and accurate insights when one method’s weaknesses are mitigated by another method or methods’ strengths. The specific considerations for UXR mean that methods may need to be adjusted from their original research context and tailored to time and budget constraints. For instance, ethnographic designs typically include fewer primary respondents and much shorter immersive observation sessions than market research or academic ethnography. In UXR, data collection via the different methods is often done in parallel in one data collection session. A contextual inquiry may combine immersive observations, a semi-structured interview about the current state of the workflow, time on task, and possibly the number of clicks and a hover heat map. A usability test may incorporate a semi-structured interview about user needs and time on task metrics. It is possible that a research design to aid in the development of a digital product may incorporate a usability test of a prototype and then a system usability scale (SUS) assessment to gain a basic understanding of user perceptions on a larger scale, and web analytics to assess potential friction points through click analytics, hover analytics, or drop-offs.
Wrapping Up
For some business problems, a mixed-methods approach to UXR is a substantial value add. Mixed-methods research can yield more profound and more comprehensive findings that lead to better design recommendations in a competitive timeframe. Radiant has deep expertise in mixed-methods research designs. Reach out for a UXR plan to inform your business strategy today!
