Why UX workshops rock: comparing UX discovery methods

The bottom line: Kickoff projects right by choosing the discovery method that fits your team resources and supports your project goals for optimization or innovation. 

Compared to other UX discovery methods, workshops combine several attributes that help set projects up for success. I’m a huge fan and begin every engagement with a 2-4 days of UX workshops. We gather a cross-functional team of business, technology, strategy, and UX specialists from both our company and the client to run through a broad set of activities running from broad goals and vision all the way through sketches of specific screens and interactions.

However, that doesn’t mean workshops represent the best way to generate shared vision, ideas, and requirements. User experience professionals have a huge arsenal of methods for generating insights.

You can facilitate workshops co-located or remotely, and they allow you to collaboratively generate both a breadth of information as well as dive deep into multiple areas with participants from across silos and multiple stakeholder groups from the business to users to different business lines. 

Enter your email and I'll send a quick note when the next post is up.

Of course, workshops have some constraints. Workshops generate qualitative data, and their advantage working collaboratively with multiple participants also means you must have access to multiple stakeholders at the same time.

If you have different needs for generating insights, observations, and data, then workshops may not be the right tool for the job. In the chart below, I’ve listed eight common discovery methods and show how they compare to workshops to offer some rough guidance on how to choose the right method. I’ve tried to sort the methods by how easily you can perform them.

Low-investment discovery methods

  1. Competitive/sector research
  2. Expert review (and heuristic analysis)
  3. Metrics/analytics review
  4. Surveys

Medium-investment discovery methods

  1. Workshops
  2. Interviews

High-investment discovery methods

  1. Contextual inquiry
  2. Ethnography

11 attributes for comparing discovery methods

I used the following 11 attributes to compare the eight methods.

  1. Co-located: Can you conduct the method in the same room with others
  2. Remote: Can you conduct the method remotely
  3. Synchronous: Can you conduct the method with participants at the same time
  4. Asynchronous: Can you conduct the method when stakeholders participate at separate times
  5. Collaborative: Does this method allow you to work collaboratively with stakeholders
  6. Cross-silo: Does this method provide access to stakeholders across silos, business lines, and expertise
  7. Qualitative: Does this method generate qualitative data
  8. Quantitative: Does this method generate quantitative data
  9. Reach: Does this method reach a broad set of stakeholders
  10. Breadth: Does this method generate data across a broad range of topics and issues
  11. Depth: Does this method allow deep exploration of specific topics or issues

The following chart compares the eight methods along these 11 attributes. 

A comparison of low-, medium-, and high-investment UX discovery methods. Click to view a larger version of this chart, or you can download a 40kb PDF.

This comparison focuses on discovery methods and does not look at methods for evaluation. Reviews of user profiles, flows, and interfaces with any stakeholder groups, from the client to the tech team to end-users, are part of a different set of activities during the experience design and development process.

Low-investment discovery methods for small, UX teams of one

You can conduct several of the eight discovery methods both remotely and asynchronously with little to no assistance from your client. These methods allow resource-limited UX teams to tailor discovery to their existing team, budget, and timeframe making them ideal for agile and lean teams.

Competitive/sector research

remote, asynchronous, qualitative, quantitative, breadth

Research into competing services, similar interaction solutions, or industry sectors provides an easy way to begin discovery. You can conduct competitive research remotely, off and on across several days or weeks (asynchronously). Based on your approach, you can generate a mix of qualitative and quantitative data across a broad range of topics (breadth).

Expert review (and heuristic analysis)

remote, asynchronous, qualitative, breadth, depth

With expert reviews and heuristic analysis have you sit down and walk through a service, site, or application to discover insights, issues, and constraints. You can conduct expert reviews remotely, off and on across several days or weeks (asynchronously). 

Although some approaches appear to generate quantitative data, expert reviews only generate qualitative data. However, because of the qualitative approach, you can generate data across a broad range of topics (breadth) as well as dive deep into specific areas (depth).

Metrics/analytics review

remote, asynchronous, cross-silo, quantitative, reach, breadth

Unlike competitive research or expert reviews that you conduct at your own volition, a metrics review typically requires some assistance from the client, so that you can get access to the metrics and analytics you would like to analyze. Still, you can get access to a wealth of data from only one or two client contacts. 

You can conduct metrics reviews remotely, off and on across several days or weeks (asynchronously). Unlike expert reviews and competitive research, metrics and analytics can provide access to data from across multiple silos (use and flow, search analytics, customer support, sales, top-line KPIs, etc.). Metrics and analytics can provide both a broad range of topics (breadth) from across a very wide range of stakeholders like all the users of a service, all the support issues for a year, all search queries, etc.

Surveys

remote, asynchronous, cross-silo, qualitative, quantitative, reach, breadth

Next to workshops, surveys provide the most versatility when compared to the other discovery methods. You can conduct surveys remotely, off and on across several days or weeks (asynchronously). Based on respondent groups and survey design, you have access to both qualitative and quantitative data across a broad range of stakeholders (reach) and a broad range of topics and issues (breadth).

Medium-investment discovery methods

Workshops and interviews require greater investments from UX teams and clients in terms of both team members and time. Of the two, workshops offer the greatest amount of flexibility in discovery. In my practice, I have almost completely abandoned interviews for discovery in favor of UX workshops.

Workshops

co-located, remote, synchronous, collaborative, cross-silo, qualitative, breadth, depth

Workshops allow you to gather stakeholders from across multiple silos into one room for collaborative discovery and discussion. Because of this collaboration, you must facilitate workshops with everyone in the same place at the same time (synchronous). However, you can still facilitate workshops in the same geographic place (co-located) or remotely with tools like Mural.ly

The cross-silo collaboration allows for qualitative discovery of data across a broad range of topics (breadth) as well as deep dives into any issue that seems interesting (depth). Unfortunately, workshops do not allow you to reach much more than about 15 participants at a time. You can counter this limitation if you combine workshops with another discovery method that allows better reach like surveys or metrics/analytics reviews.

Interviews

co-located, remote, synchronous, cross-silo, qualitative, breadth, depth

In many ways, interviews run like workshops with one participant. Because they include one stakeholder at a time, you can schedule interviews more easily. But you miss the collaboration you get in workshops. For me, collaboration adds so much value and generates so much more breadth and depth, I choose workshops whenever possible.

You can facilitate interviews while co-located or remotely. Being in the same space, interviews are synchronous. (Asynchronous interviews are surveys.) Based on your participants, interviews allow access to multiple silos and expertise, and you can generate both a broad range (breadth) of qualitative data as well as explore specific issues at length (depth).

High-investment discovery methods

Contextual inquiry and ethnography both require considerably more resources than any of the other discovery methods in this list. They require more planning, more scheduling, and generally take place over longer periods of time than low- and medium-investment discovery methods.

Contextual inquiry and ethnography seem much less flexible than the other methods in this list. Their strength comes into the breadth of discovery that allows you to discover things you didn’t know you didn’t know. The low- and medium-investment discovery methods work best when identifying solutions for known or familiar problems. Contextual inquiry and ethnography allow you to discover unknown problems and solutions to support innovation and step-changes in products and services.

Contextual inquiry

co-located, synchronous, cross-silo, qualitative, breadth

Contextual inquiry requires a good deal of up-front planning, recruiting, and scheduling to support the co-located, synchronous observation of users in their natural habitat. Contextual inquiry generates qualitative data across a broad range of issues (breadth). Being able to interrupt the participant to inquire about something allows you to explore issues at greater depth as well (though not as much depth as other methods on this list).

Ethnography

co-located, synchronous, cross-silo, qualitative, breadth

Ethnography typically requires much longer collection times. You must observe participants in the same space (co-located) at the same time (synchronous). Based on the users you observe, you can discover qualitative information about users from across stakeholder groups (cross-silo). And based on observation periods and context, ethnography allows you to gather observations about very diverse issues (breadth).

The diverse set of issues, especially the unknown unknowns, that ethnography can reveal help identify the kinds of unknown problems and solutions that lead to step-change innovations in products and services.

Choose the right UX discovery method

Although I recommend you kick off projects with UX workshops whenever possible, this comparison illustrates strengths and advantages of other common discovery methods. Comfortable process should not drive your choice of discovery methods. Don't always start with workshops or interviews because that's what you're used to. Your resources identify what discovery methods are most possible given your current team, timeline, and budget. Your project goals help guide you towards methods that work best for optimization or for innovation.

Download the chart comparing UX discovery methods as a 40kb PDF, so you can hang it in your cube :-)