key	News	Sept. 11, 2(X30 25
RESEARCH t 4ples and oranges
Don't read research. by the numbers

Hy STEPHEN J. HELLEBUSGH
R
ecently, a new oxymoron has arisen in marketing research--qualitative numbers. Used incorrectly, this technique can be dangerously misleading.

Sp

Qualitative research uses small, nonrepresentative samples of respondents to help refine concepts, obtain general reactions, learn consumer language or explore new areas of opportunity. It specifically does not use numbers in reporting, and for good reason: The.discipline was developed to explore these nonstatistical concepts in research. The results usually are presented in the foa in of verbal statements or conclusions, sup-ported by respondents' direct quotes. Such research generally has been done via one-on-one interviews or focus-group discussion conducted in-person, over the telephone or via computer.
Sometimes, the sample sizes get quite large by qualitative standards-for example, 50 one-on-one interviews or 10 focus groups. But the research still is qualitative; asking the same question in 10 focus groups of 10 respondents each presents the question in 100 undeniably different contexts, and it is not the same thing as doing structured inter-views with 100 respondents because the questioning is not controlled or standardized. Even question wording may differ from occasion to occasion.

Knowing people's reasons and motives for responding
is qualitative research's biggest benefit

Qualitative numbers involve deriving tables and graphs representing numerical data from qualitative research. But a graph (especially one with no base sizes shown) depicting, say, the responses from 20 people in two focus groups can be a dangerous thing, representing misinformation of the highest order.
To take one example, suppose the qualitative numbers from two focus groups of 10 each show that 50% like Option A best, 30% like Option B best, and 20% pick Option C as best. Clearly, Option A is the big winner. Or is it?
The two main aspects of sampling are quality and quantity, where quality refers to how representative the sample is of the population with which the researcher is concerned. Typically, strict representation is not a concern in qualitative research, and the samples are somewhere in between being representative and being picked for convenience. In our example, unless the people were selected to be representative, the results could be seriously distorted by the ways in which they are not. For instance, suppose the concepts being tested concerned a new computer design. If 80% of the respondents were low in computer literacy, and those with real computer savvy all loved Option C, then Option A's win is suspect, at best. But as the end-users, we may have no idea that this is the case.
Quantity is the other sampling issue. These qualitative numbers may not differ from one another, statistically speaking. It would take an expert in statistical testing to determine if they do, and even then, because of the quality issues, the results would be in doubt. A good sample for estimating or testing demands both quality and quantity.
Many large companies have marketing research departments, and often-in addi‑

don to a subcontractor's report-the researchers in those departments interpret and recommend courses of action based on the information. But the process may not stop there. To continue our example, suppose the graph of these results is lifted out of context by an assistant brand manager and presented to key decisionmakers as a justification for going with Option A. At this point, marketing research truly has done the company a disservice by generating and disseminating bogus data.

Examining the number of respondents who prefer a given option can be useful to the interpretation of qualitative research. A researcher can report that the most preferred option is Option A, and the numbers are unnecessary. But the true benefit of qualitative research is understanding respondents' motives and reasons for responding as they do because such information is virtually impossible to capture in such depth from an open-ended question in a quantitative survey.

I suspect qualitative numbers are here to stay. I am told that some clients really like focus group reports with graphs and tables, and once Pandora's box has been opened, there is no closing it. I can only hope that those who understand research will make every effort to avoid misleading the end-users with them.

New York +1 (212) 447 5300	Cincinnati vi (513) 771 7111	San Francisco +1 (415) 288 4141	Mexico City 52 (5) 536 3061
Repiona9 Headquarters New York +, (no) 441 5300 London 044 (o)171 644 0.0 Sydney +61 (o)7 9954 5650 arww.spSS.eom%spSSr, Sales-uSOspssmaspss.tom	MOOD 060::6044

1 of ~ntF n tional	S-. -C.:	le

	f fflrl9 r, iYs i3.:r TTt3i14 't w~f;ci:Y l {}I	xr)ii	7r	Ulutl0

(a1 rnsri a?.i	tnaaci. s	u~ the irledl sIot)xi[ partiP for a
ridin a~ttra d.i~nerro

Stephen.. Helleb ch is president of Q2 Marketing Research Inc., a Cincinnati-based marketing research firm.