-
Ceril Shagrin
December 8, 2009
Have an opinion? Add your comment below. -
One of the more outspoken radio representatives who testified at the recent House Committee panel on the PPM was Univision EVP/Research Ceril Shagrin. A veteran of over two decades of research experience while at Nielsen, Shagrin outlined five key issues facing the PPM -- issues she believes are "entirely fixable."
They are: 1) recruiting from the wrong sample frame; 2) including cell phone-only households from a second frame, which limits the CPO percentages; 3) under-representation of Black and Hispanic listeners in total numbers; 4) the PPM panels still aren't big enough; and 5) panel members aren't adequately trained.
Here, Ms. Shagrin elaborates on those points, as well as touches on other pertinent ratings matters.
Please summarize your testimony before the House Committee regarding Arbitron's PPM.
In my testimony before the House Oversight Committee, I said that what Arbitron did was roll out a 21st Century technology paired with aged 20th Century methodology. They've got to make the methodology catch up to technology in terms of a five things. I commented that Arbitron issuing the wrong sample frame in the context of that. Phone samples were just fine 25 years ago ... maybe even 20 years ago. But today, those sample frames exclude too many people from different types of homes -- and those who are excluded tend to be a higher percentage of minorities
In Arbitron's most recent PPM client conference call, they said that since June, they've increased participation of cell phone-only Black and Hispanic households by well over 20%. Overall, Black and Hispanic participation has increased as well. Aren't they currently doing what you are suggesting?
It's not just a question of having the "right number" of minority panelists; it's a question of having a representative sample. They may have right number of Hispanics, but if the Spanish dominant households are all 50+ and the English-dominant households are mostly 18-49, that may look really good when you put them together, but when you look underneath the surface, that sample doesn't accurately represent the Hispanic audience. When they over-sample in certain areas to make up for an under-sample in other areas that does not create a representative sample.
In regards to cell phone-only households, the way they measure makes it extremely hard for them to get people to cooperate. Their methodology limits CPOs to 10-15% of the total sample. Then they are dependent on the CPOs to get back to Arbitron. Those types of homes tend not to cooperate on a consistent basis, which results in a greatly under-represented CPO sample.
Just how can you make this segment of the sample cooperate in terms of using the PPM regularly?
The way to do it is the way they did it in Houston. Instead of a phone-frame sample, use an address-based sample with in-person recruitment, and have in-person recruiters encourage those who say no to participate, and those and to use the PPM consistently and properly. The diary is simple to use and not very frightening, but with the PPM you have to wear a device that knows your whereabouts for two years. Young people won't be bothered to do this, and many Hispanics will not understand how it works and would decline participation. This is a difficult task - getting enough people to agree to carry or wear a meter.
What's happening right now in Riverside is a very good example of what happens when you get the sample right, but then ignore it. That's why Riverside sample degraded considerably in recent months.
It may be easier to get a representative sample with a diary, but it's also inhibited by the users' temptation to simply write down their favorite station for a considerable length of time, when in reality, they're channel surfing.
The diary isn't perfect, but neither is the PPM. At least with larger samples, you received a more representative sample in the diary than you do with the PPM. However, improving the same frames in diaries is necessary as well. But here's the key point: The PPM sample is significantly smaller than the diary sample - by up to two-thirds smaller. For example, if you have 3,000 diaries in one market; you'll only have 1,000 PPMs used in the same market. Arbitron says they're increasing their sample by 10% -- that would make it 1,100, which is still a small sample size that's far more difficult to get a representative sample when people refuse to cooperate, which makes it harder to accurately compensate when your sample drops for any reason ... and that's not acceptable.
The in-person recruitment that was used in Houston was paid for by both Arbitron and Nielsen. How can you expect Arbitron to do that alone in a market without passing along the extra cost to its clients, who are already chafing at the current fees?
Yes, more money to will have to be spent to collect data accurately, but it needs to be done as clients will not want to spend their money on bad data. I think working together, we can find the most efficient and cost-effective way get the data that's needed for a quality sample. If you do a better job at recruiting people, you may be able to get away with a smaller sample. It's no use increasing the sample size if you don't increase the quality of the sample by capturing those who are not accurately being measured.
Just how big should the PPM sample size be?
Realistically, having the same size as the diary sample will not be cost-effective, but you have to figure out what the trade-offs are. You need to make to get a representative sample in a cost-effective way. Until we sit down and try to work this out, having a small sample that is representative would be much better than a huge sample that's not representative. If a sample is made up of just 50% of a certain demo or ethnic group should be, no matter what is done to compensate for that discrepancy, it wouldn't be representative. I'd much rather see Arbitron work more at getting a representative sample and increasing it to some extent than just ensure that the sample size is a bigger. If we can get a representative sample that provides usable data every day, that would be acceptable.
Unfortunately, we've seen Arbitron samples where more than 25% of people don't supply usable data every day. That's unacceptable. If they could get 90% of the people using it every day to provide usable data, that would be beneficial. Some of the current samples have a certain demo under-represented by 20%. Whether you're under-represented by 20% or over-represented by 20%, they're both bad. You have to match Universe Estimates.
Have you examined Nielsen's new sticker diary - and if so, how does it stack up to the PPM?
I've only looked at some of the data, so I don't have the full picture yet. It seems that they have work to do as well, but starting out with an address-based sample, and working to get higher cooperation by examining a lot of ways to get higher cooperation, they're doing a lot of things right. Maybe some combination of a meter and diary would work, but we still don't know that what we have with PPM until every market is rolled out like Houston.
Bottom line: Univision's view of PPM data is...
We currently use it in Houston, but we will not use PPM data in any other market until that market has been MRC accredited and it meets minimum quality standards. Other than that, we're not using any ratings. We would love to have ratings -- if we could believe in them to make programming decisions and sales. Until we have numbers we can believe in, our advertisers will rely on us to deliver what we've successfully delivered in the past. But I'm hopeful that Arbitron finally gets a decent measurement and offers true value.
What do you say to advertisers who have been calling for electronic measurement to supplant the diary - even when the PPM records passive listening, while the diary's flaw is that it's susceptible to brand recall?
The PPM measures exposure, which is not necessary an accurate measure, while the diary may suffer from those who overstate stations they're very loyal to. As for advertisers, I would believe that they want to advertise on a station that people remember they listen to, not just happened to be exposed to. For example, a person with a PPM can be in a car with the windows open and another car pulls up beside it with a radio station blaring; the PPM would pick up the exposure from the other car and not give an accurate measurement of that person's listening habits. This presents a problem. This happens often, as many business establishments expose their customers to English-speaking radio. Rarely are there cases where English speaking PPM carriers are exposed to Hispanic music at a retail outlet.
I think advertisers want to know exactly who are really listening to their commercials - and there's a better chance the listener will remember your ad if they're listening to a station they chose. Otherwise, the advertisers' message will not resonate with them.
But that brings us back to the original question - why have advertisers been pressuring radio to adopt electronic measurement?
My guess is that advertisers and their agencies talked about the need to get more accurate measurement with reliable data, but that nobody asked, "Just what are we giving up?" I absolutely will tell you there is a big difference. My background ... working at Nielsen for 27 years, I've been through changing methodologies in TV before this. The difference is TV with a meter records the station someone has actually tuned in, while the PPM is more of a passive-exposure audience.
I've done some real in-depth research in terms of the overstatement of listening in the diary as to the overstatement of reach of PPM, and I came to the conclusion that both have weaknesses, but the diary delivers a stronger message for the advertiser's commercial to a more interested audience.
-
-