Thursday, July 24, 2008

How are Focus groups different from Usability Testing?

In today’s fast paced life, people tend to avoid products that frustrate them. Satisfied customers would be ready to buy good products and services. This also helps in word-of-mouth marketing.

Used together, focus groups and usability testing help in building great products and services which are highly desirable, satisfying, fun and easy to use and also worth being recommended to others. However, both, focus groups and usability testing have different focuses and must be used in conjunction with each other. Focus groups and usability testing must never replace each other.

A focus group speaks about the extent of promises that can be made to the customer while usability testing helps shaping the products based on those promises. Usability testing is special as it focuses on eliminating end-user frustration by turning it into satisfaction and pleasure.

This article is a continuation of a series of comparisons of various user research techniques. Here I’ve contrasted focus groups with usability testing.

Focus Groups

Usability Testing

What is the meaning of focus groups?

Focus group is a brain storming session group administered a moderator.

The goal of a focus group is usually to get a variety of broad opinions of what people want a product to do for them along with what will drive the end users to buy the product? This helps the moderator to begin considering the features of the product.

Focus groups are opinion based and their motivation is to find out what the desires and wants of the users are. .

What is the meaning of usability testing?

Usability testing is a one-user-at-a-time testing by users but moderated by a seasoned usability professional.

While the user interacts with the product in a structured manner, the usability professional notes down the stumbling blocks, hits, and reactions of the users while they use the product. This help usability professional to guide product development decisions. The key focus is what will make the end users of the product to learn about and use the product easily.

Usability testing is observation based and tells if or not something works.

What is the output of focus groups?

The output of a focus group is a collection of various opinions, attitudes and preferences of the users.

What is the output of usability testing?

The output of usability testing is observational and behavioural measures based on the ease with which the users handle the products.

When should focus groups be used?

Focus groups are best used in the early stages of product or service development to provide wide-ranging information on opinions and attitudes of the target user group. This is used primarily when important investment decisions are to be made.

Focused on opinions: Focus groups have many voices at the same time, so it is impossible to observe the users in action and record the steps. The moderator can, at best, get opinions about their intended usage With this lack of focus and precision, it only elicits vague opinions.

Consider what to build: The purpose of a focus group is to decide what kind of product to consider building.

Decide Costs: A company uses focus groups to decide the extent of investment of product development. This results in large near term costs in the hope of future profits.

When should usability testing be used?

Usability testing is used throughout the development process to help in finely focused development decisions and it also guides in the realization of the big picture.

During the early development stages, paper-prototype based Formative Usability Testing is used to test ideas, navigation, and task flow.

Post development, Summative Usability Testing is used to benchmark navigation, task flow, measuring user performance, errors, and subjective satisfaction.

Focused on tasks: In usability tests, the users are asked to complete many tasks one after another. The difficulty with which the users complete the task is carefully recorded. This clear focus helps them to get more specific and actionable results.

Making products better: The purpose of a usability test is to improvise the product to make it work better.

Increase profits: The usability test is used to improvise the products resulting directly in cost savings, greater sales, and increased profits.

How many people do I need for a focus group?

Focus groups involve 6-10 people interviewed together in the same location at the same time.

How many people do I need for a usability testing?

Ideally about 5-8 users are tested per sub-group individually with usability professional.

Why does usability research requires only a few users?

Usability research is usually qualitative, or driven by insight (why users don't understand). Qualitative research follows different research rules to quantitative research and it is typical that sample size is low.

The end result of usability testing is not statistical validity per say but verification of insights and assumptions based on behavioral observation - quantitative vs. qualitative.

· We are looking for behavioral based insight, like what people do.

· Statistics tell half the story and often are devoid of context, like why did they fail?.

· Research clearly shows that even with low numbers, you can gain valid data.

· Usability testing is being used industry-wide and has been for past 25 years. Experts, authors and academics put their reputations and credentials behind the methodology.

As mentioned earlier, Usability research is behavior-driven: You observe what people do, not what they say.

In contrast, market research is largely opinion-driven: You ask people what they think and what they think they think. For this, big samples are needed for market research. This is why phone or web surveys require hundreds or thousands of responses.

Behavior-driven research is more predictable. Basically, if 10/15 users are confused you can assume that many more will also be confused as well.

How much would a focus group cost me?

To conduct a focus group, expenses will be incurred on recruiting, incentivising, focus group facilities such as a place where a focus group can be conducted, a place where clients and others can view the proceeding, and audio-video equipment must be readily available.

Also expenses have to be incurred to carefully prepare the focus group protocol and also to analyze the findings.

Focus groups are less expensive when compared to usability testing.

How much would usability testing cost me?

Preparation of testing protocol, hiring a usability lab with one way mirror, recruiting and incentivising people from a target user group, conducting tests one-by-one, and analysis is time consuming and expensive.

Usability testing requires lot more time to plan and organize.

Usability testing is one of the most expensive and most effective usability evaluation methods today.

How long does it take to conduct focus groups?

A focus group involves a detailed protocol scripting and participant recruiting.

However, it is not time consuming when compared to usability testing. Ideally a focus group session should not exceed 1 ½ hours.

For a couple of focus groups involving 14-20 people, Kern would ideally take 2 ½ weeks to recruit, conduct tests, and present analysis.

How long does it take to conduct usability tests?

Usability testing method involves detailed protocol scripting, participant recruiting, and conducting the test one by one, followed by data analysis and report generation, it is a time consuming process.

For a usability testing involving 2 sub-groups (about 16 users), Kern will take about 3 ½ weeks for the entire process.

What are the advantages of focus groups?

Focus group is one way to gather information about a target audience that you have very limited information about.

They help us understand users’ opinion, attitudes, preferences and initial reactions.

Group discussions often spark new ideas. They are excellent if you are looking for multiple and a wide range of opinions, ideas and points of views in a short period.

Focus groups are cheaper and faster to conduct.

What are the advantages of usability testing?

The results of usability testing give a truer picture of the real problems people encounter as they’re derived from observations of real users.

They help us to understand the different behavioural patterns of the users while using the products and services.

Videos of user sessions provide a very powerful persuasion tool for reluctant stakeholders. It’s hard to argue with recordings of people in tears of frustration. Understanding points of frustration usually trigger practical solutions.

What are the disadvantages of focus groups?

The findings do not lead to statistically significant results.

As there are more members, one or two members’ opinions usually dominate others and this does not help in arriving at clear conclusions. Opinions.

What are the disadvantages of Usability testing?

Usability testing is expensive and time consuming.

The validity of test findings depends heavily on identifying the right target group, accuracy of usability testing protocol to recognize key user tasks, and accurate user recruiting.



Carley said...
This comment has been removed by the author.
Carley said...

Excellent article! I work for Macadamian Usability, and am always delighted to read blogs that mention usability and evaluation methods. I'm glad to hear that you found our site useful. I noticed the link you cited in your references was broken, so I have included a working link:

sowdamini said...

hey carley if you want to know more about usability and evaluation methods you must read our company blog. where you can find lots of info on usability.