George Ivie

While Arbitron's PPM continue to roll out nationwide amidst the outspoken criticism of various radio group executives, minority broadcasters and some in Congress, the Media Ratings Council is essentially the eye of this hurricane. The Council, which has already accredited three markets for the PPM, continues to work with Arbitron while independent CPAs audit the other markets. Because PPM accreditation is a long, complicated process, the MRC prefers to stay out of the public eye as it monitors the proceedings. Fortunately, MRC Executive Director George Ivie has been kind enough to offer All Access an overview into the accreditation process and how it works in evaluating the PPM.

How long have you been at the MRC ...and how has its role evolved over the years?

I've been here since January 1st, 2000. When the MRC was formed back in the '60s, it concentrated on broadcast TV and radio measurements. Those measurements have evolved as new media emerged; we also expanded into other areas, such as print and the Internet. There have been all kinds of change ... literally tons of changes. Even when you look at when I first started in January of 2000, we did just a handful of audits that year. Last year we did almost 60. We had 50-something members back then; now we have 115 members. The scope of what the MRC does have expanded a lot.

Who exactly audits the ratings services?

The audits we rely on are conducted by independent CPAs. We hire these independent CPAs and our members assist in directing the CPAs in building the audit plans and scope of testing. The MRC provides oversight, but because of the CPAs our process is very scalable. While the membership of the MRC has grown, the staff itself remains very small. We basically have four employees, which is a little deceiving because the MRC has 115 member organizations that get involved in our process, participate in audit reviews and assists the MRC staff in deciding on the accreditation of measurement services.

Please describe the process of accreditation that Arbitron is following.

The MRC operates on several different levels. Each MRC member organization has a seat on our Board of Directors, which makes general policy and operating decisions for the MRC. Our members are also divided into committees by media type; this includes a TV committee, print committee, radio committee Internet committee and an out-of-home committee.

Each of our 115 members can decide which media committee to serve on. Naturally, they want to be on the committee based on their interests -- and they can serve on more than one committee. Those committees are responsible for the media audits that pertain to them; the radio committee is responsible for Arbitron, for example.

Continuing with the radio example, interested members of the radio committee organize into an audit committee for each radio measurement service. This audit committee reviews the audits with the applicable CPAs and they interact with the measurement services. The CPA meets with the committee; the committee then evaluates the audit, makes recommendations and communicates with Arbitron. There's a lot of back and forth during these meetings; there are often many rounds of communication before there's a recommendation for accreditation.

Once there is a recommendation for accreditation, the media committee - in this case, the radio committee - has to ratify it before it can go to the full board of directors. If the board then recommends accreditation, it's voted on by the full membership of 115, where it needs a majority decision for approval.

If an accreditation is denied, the measurement service can appeal, which leads to an extensive due process, including potential arbitration and, if necessary, a direct appeal proceeding with the MRC Board. Upon appeal, enhanced voting rules are employed to ensure extremely diligent decision processes such as super-majority voting rules. For example, related to the PPM services, Arbitron has not appealed any of MRC's decisions thus far.

Is there a time limit on accreditation decisions, or do you allow measurement services such as Arbitron unlimited opportunities to tweak their process to gain accreditation?

MRC has the right to announce the granting or denial of accreditation; those are official statuses, the end products of an audit. But there's a lot that goes on in an audit before it reaches that point. The committee reviews an audit report; it goes to Arbitron. The report may tell the service what's lacking or needs to be corrected. Arbitron has the chance to respond and/or correct things. There's continuous dialogue, action plans, etc.

A lot of interim work goes on for a period of time, so we refrain from announcing the status of a market if we think it's close to accreditation. If we see progress and things changing, we might hold off on finalizing the audit until 1) the certain parameters are met ... or 2) they're so far away, we might as well put our pencils down -- that would result in a denial. But we don't automatically announce our decision every three months ... especially when a certain market may be close to accreditation.

If there is a denial, the service basically has to start over. That's why we don't deny accreditation on a lot of audits; we prefer to work behind the scenes to move the service towards accreditation. Through it all, we make sure we're not misrepresenting anything to the marketplace. This is a very complex situation that's difficult to summarize in a brief article.

How does the MRC audit so many PPM markets at once?

We're currently auditing PPM markets quarterly on a rotating schedule that is worked out with Arbitron in advance. We don't do all 30 markets during the same quarter; certain markets are audited the first quarter, other markets are audited the second quarter, and so on.

When we execute an audit, the data goes to an audit committee, which reviews it and interacts with Arbitron. There's a lot of back and forth there, and when this auditing and follow-up process is complete we vote on accreditation, which then goes to our board for ratification. At that point we notify Arbitron on whether accreditation has been approved.

How different is auditing the PPM as opposed to the diary?

The audit of the PPM is very different because it's metered technology, which uses a panel of respondents vs. one-time contacts. We've done a number of meter service audits, which are completely different than a diary audit. Although we're very experienced in auditing meters, the PPM is unique in that we went through a process to build a custom audit for PPM. As we continue to audit PPM data, the process and procedures are continually revamped. It's a custom-designed audit that has to be responsive to the methodology, sampling and quality of service.

The diary is much easier to audit. All the evidence is there; diaries are sent to the respondent, who only has to fill them out and return them. There's a whole host of issues -- for example, literacy bias -- that can be a concern, but generally these can be overcome with proper treatments and simplification. Concurrently, meters have certain burdens associated with them, too. This is not to say that a diary is a better measurement. It's just easier to audit.

What type of burdens must Arbitron address with the PPM?

One of Arbitron's burdens with the PPM is associated with making sure the respondents carry the PPM with them. For example, they're supposed to wake up in morning, undock their PPM and carry it with them as they go around the house, have breakfast and so on. They're supposed to carry it around with them when they go shopping. It's a burden and a tradeoff. The results you get from that measurement mode are much different than what you get from a diary.

That seems like an incredibly daunting challenge.

It is expecting a lot. Arbitron has developed very extensive coaching and motivational processes to get people to do that. The thing is, different people react differently to those motivations and incentives. People who are employed use the PPM differently than those who are not employed. There are differences between young and old, age by race, ethnicity, etc. ... they all react to these burdens differently.

At first glance, it seems that the recruitment and sampling methods Arbitron used in Houston, which is now accredited, are far more successful than the Radio First method employed in Philadelphia and everywhere else, which has yet to be accredited in the lion's share of markets. Is it in the MRC's power to recommend the best ratings methods to achieve accreditation?

The auditing process is confidential, so I can't discuss specific results. I can say that sampling is a very important part of an audit. The MRC audit follows through the entire process of a measurement service. As with any measurement service we audit, the MRC looks across all the procedures -- the sample frame, how they sample, field recruitment and data collection, how they edit and weigh the data ... all way through a myriad of procedures. Our audit follows all the way to the reporting of the audience data.

The Radio First methodology, which is being used in all markets other than Houston, has been accredited in two markets, too, so in certain circumstances it can be accredited.

Arbitron just announced in a recent SEC filing that it is planning to do in-person recruitment to bolster its Black and Hispanic representation in several markets. Were you aware of this? Was this change a result of the give-and-take between the MRC and Arbitron?

Yes, I have been aware of this. The MRC staff has been involved in discussing these enhancements with Arbitron.

When this new process is incorporated in July, how long do you expect it to take before you can accurately judge whether these changes are having the desired effect ... 30 days? Three months? Six months or longer?

These new procedures will gradually impact the production PPM services over the next several months ... into 2012. We will be able to see impact, we believe, within the first six months despite the fact that the full impact will not have developed yet.

How is it that the Minneapolis audit -- which was initiated well after audits for New York, Los Angeles and Philadelphia, among others - can get accredited while the others are not? Essentially, what did they do right?

There were differences in how those markets performed. If you looked at New York and Minneapolis, you see two distinct demographic profiles and market environments. Our audits found out how well Arbitron sampled the demographics in each market and how the samples performed. We examined all those different parameters, and because each market is essentially different, how they perform in relation to the sample is why we can accredit some and not others.

Each market is different. There are some consistent problems with the PPM in the majority of markets, but there are also some issues that are unique to certain markets. You just can't say each market has the same problems. They vary.

Arbitron itself has disclosed publicly that there are several things the MRC is focusing on - the response rate or SPI, the tabulation rates of the panels, and what is known as differential non-compliance ... how certain groups of the population may or may not adequately comply with carrying the PPM. Those three issues tend to cross markets.

Once a market has been accredited, such as Riverside, does the MRC continue to monitor the process there to ensure that its data maintains a desired level of quality?

Yes, we audit each service annually and we update the accreditation status at that time.

Looking at the challenges Arbitron still faces with the PPM, both with the consistent problems and the unique market-specific issues, can you safely say that everything is fixable and that all markets could be accredited?

Yes, I can tell you that they are fixable. Arbitron has actually made great progress over the past few months. Granted, it's a difficult process, one where we have literally spent thousands of hours with Arbitron, working with its PPM service, but one that can and has proven to be accreditable.

Several radio groups and interested parties have used the MRC's lack of accreditation as a reason to not use the PPM at all. Some parties have asked the FCC to get involved in the accreditation process. Does it bother you when the MRC is being used as a political football?

The MRC doesn't initiate government activity; we have never have gone to a government agency and said, "We need your help with something." Other parties have done it in the past; customers of Nielsen did several years ago with the LPM service.

We have dialogued with the FCC and been in meetings to talk about the MRC process and PPM audits. We've also prepared written testimony, so we feel we have been communicating well. I don't know what role the FCC will choose to play. We know we execute our process well. We know how to audit, accredit research and make good decisions about it.

We have 115 members who think our accreditation is important, so we don't say anything during the process that would detract from our decision. People always have a right to go to the government if they feel a ratings process is hurting them, but we believe in our accreditation. If you're a user of a measurement service that is not accredited, you know it either hasn't been audited or doesn't meet our standards.

Has the MRC audited Nielsen's new sticker diary?

No, we haven't audited Nielsen's book yet, but we have discussed it with them. Nielsen is preparing for an audit in the future.

Finally, are there any misconceptions about the MRC that you'd like to dispel?

Not that I know of. We're not a very public facing organization. We have a very simple function -- conduct audits and verify compliance with the standards we publish. The outside world can look at our accreditation and consider our standards. We don't try to be political; we're not out to denigrate any measurement service or popularize any measurement service. We just accredit research that meets our standards and hopefully the industry will rely on the assurance that accreditation brings.