Much of the research I do here at NPR focuses on demonstrating the value of the NPR to our sponsors. We try to answer the questions:
- Do our listeners think better of companies that sponsor NPR?
- Are they more likely to seek out information about sponsors?
- What is the value of sponsorship for those companies?
- To answer these questions, it is critical to show a comparison: "Better" than what? "More likely" than whom?
To answer these questions and make the case for sponsoring NPR, we rely on control groups.
Recently I've been discussing with a couple clients the strengths and shortcomings of control groups. In one conversation we were trying to figure out the best way to establish a control group for a study using our in-house listener panel. Just a couple days later, I explained to another client the necessity of including the control group.
For those who only may have a vague memory of control groups from long-ago science classes, the control group is, to quote Dictionary.com, "a group of subjects closely resembling the treatment group in many demographic variables but not receiving the...factor under study and thereby serving as a comparison group when treatment results are evaluated."
In the research we do for on-air sponsors, our "treatment" is the on-air sponsorship announcement. So the treatment group is NPR listeners while the control group is people who don't listen to NPR but have a similar profile in age and gender and education. We ask both groups about awareness of brands, opinion of brands, and other questions relevant to the particular sponsor. We've conducted this type of research for many sponsors and generally find that listeners are more likely than non-listeners to be aware of the sponsor brand, to have a higher opinion of the brand, and to seek out information about the brand.
Sometimes we have to take a different approach to establish a control group. For example we wanted to measure the impact of sponsorships on the iPad by only testing our in-house listener panel. Since our in-house panel, not surprisingly, only includes NPR listeners/users we didn't have any "non-listeners" to include as the control group. Instead we created a control group with NPR website users who don't use the iPad.
Sometimes, for a new sponsor, we can do a pre- or post-test, where the measurement of listeners before a campaign starts serves as the control group. We have also simply compared brand attitudes among those who specifically recall the sponsorship verses those who do not. This, however, is not a true control group since many of those who don't consciously recall the on-air message have likely been exposed to it.
It can take more time and expense to include the control group in a study, but designing our research with a clean control group helps us effectively make the case that sponsoring NPR provides a clear benefit for the sponsor.
Susan Leland is the Research Manager for NPR's Corporate Sponsorship and Development.