What is A/B testing and how does it support decision making?

• A. It compares more than two variants at once.
• B. A/B testing compares two variants to determine which yields better performance using statistically significant results.
• C. A/B testing guarantees a positive outcome.
• D. A/B testing is only used for qualitative judgments.

Answer: (B)

A/B testing is a controlled experiment that compares two variants to see which one performs better on a specific metric. You randomly split your audience into two groups, show each group a different version, and measure how they respond. The goal is to determine, with statistical significance, whether the observed difference is real or just due to random variation. This provides concrete evidence to guide decision making rather than relying on guesses or intuition.

This approach supports decisions by showing which version delivers a better outcome, so you can implement the winning option with greater confidence. It also helps manage risk because you only scale a change if there’s solid data behind it. It doesn’t guarantee a positive outcome—even with a clear winner, the improvement might be modest or not worth the cost. And it’s a quantitative method, not just qualitative judgment; it relies on numeric metrics and statistics. It’s also distinct from approaches that test many variants at once, which would be a different testing method.