What Is Concept Testing?

Concept testing is a process used to evaluate a new idea or concept with the target audience to understand how well it resonates with them. The primary goal is to validate the concept and identify potential issues or areas for improvement before investing significant time and resources in further development.

A concept can be almost anything you’re about to commit budget to: a new product or feature, a price point, a package design, a name, a campaign message, or an ad. The earlier you test, the cheaper it is to change course.

Why Concept Testing Matters

Most teams fall in love with their own ideas. Concept testing replaces internal opinion with evidence from the people you actually want to buy. Done well, it helps you:

• Kill weak ideas early, before they consume development and launch budgets.
• Choose between competing concepts with data instead of debate.
• Discover which features, benefits, or messages drive interest and which ones confuse people.
• Find the audience segments most likely to respond, which informs targeting and positioning.
• Refine pricing, naming, and packaging before they’re locked in.

Concept Testing Methods

The most important design decision is how respondents see your concepts. Each approach trades depth, cost, and bias differently:

• Monadic testing: Each respondent sees and evaluates only one concept. Because people aren’t comparing options, scores are clean and independent, but you need a separate sample for each concept, which increases cost.
• Sequential monadic testing: Each respondent evaluates several concepts one at a time and answers the same questions about each. It’s more efficient, but the order in which concepts appear and respondent fatigue can affect results, so rotate the order randomly.
• Comparative testing: Respondents see two or more concepts side by side and choose or rank them. It’s a fast way to pick a winner, but it tells you less about how each concept would perform on its own.
• Proto-monadic testing: Respondents evaluate concepts sequentially and then choose a favorite, combining diagnostic scores with a direct preference.

Two related techniques go deeper on features and trade-offs:

• Conjoint analysis: Respondents choose among bundles of features and prices, revealing how much each attribute contributes to preference and which combination performs best.
• MaxDiff: Respondents repeatedly pick the most and least important items from a list, producing a ranked view of which benefits, claims, or features matter most.

For visual and digital concepts, qualitative methods are useful early on. Preference tests ask people to choose between design alternatives and explain why, five-second tests check whether a design communicates its main message at a glance, and moderated interviews or focus groups uncover the reasoning behind reactions. Once a concept is live, in-market experiments such as an A/B test of a landing page measure actual behavior instead of stated intent.

What to Measure in a Concept Test

Use the same core questions for every concept so results are comparable. Common measures include:

• Purchase intent: How likely the respondent is to buy, usually on a five-point scale. Many researchers report the share of respondents choosing the top two boxes.
• Uniqueness: How different the concept is from what’s already available.
• Relevance: Whether the concept solves a problem the respondent actually has.
• Believability: Whether the claims are credible.
• Value: Whether the concept seems worth the price shown.
• Comprehension: Whether respondents understand what the concept is and does.
• Likes and dislikes: Open-ended responses that explain the scores.

Collect demographic and behavioral data as well, so you can see whether a concept performs especially well, or poorly, with specific segments.

How to Run a Concept Test

1. Clearly define your concept: Describe the product, service, or idea, including key features, benefits, and price if relevant, in language your audience understands. Present every concept in the same format and level of polish so presentation quality doesn’t skew results.
2. Set your objective and decision criteria: Decide in advance what result would make you move forward, revise, or stop.
3. Identify your target audience: Recruit people who represent the customers you eventually want to sell to, whether that’s existing customers, prospects, or a representative panel.
4. Choose your method: Use monadic, sequential monadic, comparative, conjoint, or MaxDiff based on the decision you need to make and your budget.
5. Write the survey: Combine closed-ended questions, such as rating scales and multiple choice, with open-ended questions that capture why respondents feel the way they do.
6. Include a benchmark or control: Test against an existing product, a past concept, or a control version so you can tell whether scores reflect the concept itself or outside factors.
7. Pre-test the survey: Run it with a small group to catch confusing questions, broken logic, or design problems.
8. Field the survey: Launch it to your audience through a research panel, your customer list, or channels such as email and social media. Consider timing, since seasonality and current events can influence responses.
9. Analyze the results: Compare scores across concepts and segments, check statistical significance, and read the open-ended feedback for themes.
10. Refine and retest: Use what you learned to improve the strongest concept, then retest if the changes are significant.

Concept Testing Tools

Online survey platforms have made concept testing faster and more affordable, and many now include purpose-built concept testing workflows:

• Qualtrics: An experience management platform that supports concept testing programs, as well as conjoint and MaxDiff projects.
• SurveyMonkey: Its LaunchPad product concept testing uses monadic testing to evaluate up to 10 concepts, with automated scorecards, statistical significance testing, key driver analysis, industry benchmarks, and AI-generated insights.
• Pollfish: A survey platform for reaching consumer audiences quickly for quantitative tests.
• Lyssna: User research software for preference tests, which compare up to six design alternatives, and five-second tests.
• UserTesting: A platform for recorded and moderated sessions that show how people react to prototypes and experiences.

Can AI and Synthetic Respondents Replace Concept Testing?

AI is changing concept testing in two ways. First, it speeds up the work around a test: drafting survey questions, summarizing thousands of open-ended responses, and highlighting key drivers. Second, some teams are experimenting with synthetic respondents, which are large language models (LLMs) prompted to answer as a specific type of consumer would.

The research is promising but comes with real caveats. A study by PyMC Labs and Colgate-Palmolive, LLMs Reproduce Human Purchase Intent via Semantic Similarity Elicitation of Likert Ratings, compared synthetic consumers with 57 personal care concept surveys involving 9,300 human responses. When the models were asked for numerical ratings directly, they mostly answered in the middle of the scale. When the researchers had the models answer in words and mapped those answers to a rating scale, the synthetic panels reached 90% of human test-retest reliability in ranking concepts. Without demographic personas in the prompt, the ranking signal dropped sharply.

The authors also cautioned that some subgroups, such as gender and region, weren’t reliably replicated, and that synthetic respondents only work in categories the model has learned a lot about. The practical takeaway is to use synthetic respondents to screen and improve early ideas, then validate finalists with real people before making expensive decisions.

How to Know If Your Concept Test Results Are Statistically Valid

To determine whether concept test results are statistically valid, assess the sample size and the margin of error associated with the results.

The sample size is the number of participants in the test. Generally, larger samples produce more reliable results, but the size you need depends on the precision you want, the size of the population you’re sampling, and the expected variation in responses. Monadic designs need a sample for each concept, so plan your budget accordingly.

The margin of error is the amount of uncertainty in the results due to sampling variability. It’s expressed as a percentage and indicates the range within which the true population value likely falls. The margin of error decreases as the sample size increases, and you can calculate the margin of error and required sample size before fielding your survey.

Calculate Your Sample Size

Once you know the margin of error, compare it to the differences between your concepts. If the gap between two concepts is smaller than the margin of error, you can’t confidently say one outperformed the other. Also consider other factors that affect validity, such as how representative the sample is, the wording and order of questions, and response bias. A well-designed concept test accounts for these factors and uses appropriate statistical methods.

A Concept Testing Example

Suppose a company has three package designs for a new product. In a monadic test, three matched groups of target consumers each see one design alongside the same product description and price, then rate purchase intent, uniqueness, and how well the package communicates the product’s main benefit. They also explain what they like and dislike. If one design leads on purchase intent by more than the margin of error and the open-ended comments explain why, the company has a clear winner and specific feedback to refine it before production.