Recently, I completed focus groups talking with core NPR listeners. We wanted to hear from those who listen to NPR at least weekly and, in this case, specifically listen to Morning Edition or All Things Considered. Screening for those behaviors, we expect the listeners in the focus group all have a good understanding of NPR programs and a reasonably positive attitude toward NPR. After all, if they didn't like NPR, they wouldn't listen very often, right?
Well, as I learned at a focus group facility in Chicago, apparently, that's not always true. In one group, the discussion was moving along fine as we talked about specific sponsor brands. However, when the talk turned to what the listeners thought of NPR, one participant dominated the discussion. He very frankly shared how much he hates NPR and how he frequently listens just so he can see what we are, in his opinion, doing wrong so he can then spread the word to others. A few other participants spoke up to share what they like about NPR, but most were clearly taken aback and uncomfortable sharing their thoughts in the aftermath of his diatribe.
We have seen in other quantitative research that a small percentage (less than five percent) of frequent listeners can be described as NPR "detractors", either by speaking badly about NPR to others or rating NPR poorly overall.
There is definitely a need for research to understand the concerns of our detractors but a mixed group of detractors and fans made for a very uncomfortable and unproductive discussion. Since detractors are such a small part of our audience, we had never taken participants' attitudes about NPR into consideration in focus group screening.
Hopefully, we've learned our lesson. Mixing listeners with strongly opposing opinions prevents us from having a productive discussion in the focus group setting. In the future, when focused on understanding the attitudes and concerns of our typical listener, we'll be adding a new screening question: the respondent's attitude about NPR.
We don't want groups dominated by rabid fans either; listeners with more neutral attitudes can be a great source of insights into where we need to improve. This screening will only apply to focus groups. In quantitative research, of course, we want to include the full range of NPR listeners: positive, neutral, and negative.
All you researchers out there, have you ever been surprised in the viewing room to find the participants who you recruited were not at all what you wanted? What did you learn?
Susan Leland is the Research Manager for NPR's Corporate Sponsorship and Development.