How many customers should we survey for reliable results?

How many responses does a customer survey need?

About 400 responses are needed to estimate an overall response proportion within ±5 percentage points, and about 1,000 to estimate it within ±3 percentage points. These figures assume randomly selected customers and a 95% confidence level. However, who does not respond and how the questions are asked often affect results more than the number of responses. Check both issues alongside sample size.

How does the margin of error change with the number of respondents?

Sampling error describes a range for the difference between asking all customers and asking only a sample. The margin is largest when the response proportion is close to 50%, so calculations commonly use that value. The formula is 1.96 times the square root of 0.25 divided by the sample size. If 60% of 400 respondents report being satisfied, the underlying proportion is read as roughly between 55% and 65%.

Maximum sampling margins by sample size: simple random sampling, 95% confidence

Halving the margin requires quadrupling the sample. Increasing 400 responses to 1,600, for example, reduces ±4.9 percentage points to about ±2.5 points. Choose an appropriate balance between cost and precision. In business-to-business settings with few customers, applying a correction for population size reduces the sample required.

How should the sample be designed for comparisons between customer groups?

If 400 respondents are divided into five age or regional groups, each group averages 80 people and its margin rises to around ±11 percentage points. Sample size should therefore be based on the smallest group you want to compare, rather than the overall total. If you want to compare new and lapsed customers but have relatively few lapsed customers, you can sample more of that group separately and adjust its weight in the analysis. Define the grouping criteria before the survey. Trying many ways of dividing the data after receiving the results makes it easy to mistake a chance difference for a meaningful finding.

Does a low response rate make results unreliable?

Not necessarily. A meta-analysis of 59 studies that independently measured nonresponse bias (Groves and Peytcheva, 2008) reported a very small correlation between response rates and bias. A National Research Council report (2013) also cited this study in explaining why response rates alone are poor guides to bias.

What matters is whether nonrespondents think differently from respondents about the subject of the questions. If highly dissatisfied customers are less likely to complete a satisfaction survey, the reported satisfaction level will look better than reality even with a high response rate. Compare records such as purchase value and relationship duration for respondents and the entire customer base to see whether the groups are similar. This does not mean efforts to improve response rates are pointless. It means that survey quality should not be judged by a single response-rate number without considering who is missing.

Which errors should be checked before increasing the sample?

Sampling error is only one of several sources of survey error. Increasing the sample does not reduce the others.

Sources of survey error and ways to reduce them

Article 108 of the Public Official Election Act, which regulates election opinion polling, also requires selecting respondents to represent all segments of the target population and avoiding biased language or leading questions. These are not legal requirements for corporate surveys, but they provide useful checks. Try the wording on a few customers before the main survey. Questions that people interpret differently or struggle to answer often become apparent at this stage.

What information should accompany reported survey results?

The same Act requires publication of election polls to include information specified in election polling standards, and requires advance reporting of sample size, methodology and the full questionnaire. Internal reports can likewise state the target population, sample size, response rate, margin of sampling error, methodology and exact question wording so readers can judge the evidence for themselves. If the difference in response proportions between two options falls within the margin of error, reporting them as similar is more accurate. Stating both the evidence and its limitations reflects objectivity, a core value of the Korea Research Institute of Business & Branding.

What precautions are needed when using surveys for management decisions?

Surveys show what customers say; they do not guarantee what customers will do. Even when purchase intention is high, confirm it through reservations or trial sales. Linking survey responses with transaction records to check whether satisfied customers actually bought again also helps assess predictive value. Questions about cause and effect require a comparison-group design, such as the randomized experiments discussed in the effects of management consulting. Being drawn to a few striking responses follows the same principle discussed in using other companies' success stories. See business research methods for combining fieldwork and data analysis, and market research and competitive analysis for customer research topics.

Frequently asked questions about survey samples

References

1. Public Opinion Quarterly, “The Impact of Nonresponse Rates on Nonresponse Bias: A Meta-Analysis” (Groves and Peytcheva), 2008. https://doi.org/10.1093/poq/nfn011
2. National Research Council, “Nonresponse in Social Science Surveys: A Research Agenda,” The National Academies Press, 2013. https://www.nationalacademies.org/read/18293/chapter/4
3. National Law Information Center, Public Official Election Act, Article 108 (Prohibition of Publication of Public Opinion Poll Results, etc.), effective July 1, 2026. https://www.law.go.kr/법령/공직선거법