BBM Canada improves forecasting accuracy, saves costs with SAS®
BBM Canada supplies radio and television audience ratings services to the Canadian advertising industry. BBM's membership includes TV and radio stations and networks, advertising agencies and advertisers. With huge volumes of data to process and analyze from its ongoing audience survey sweeps, BBM Canada relies on a powerful SAS® analytics solution with automated modeling capabilities to improve accuracy in survey sample forecasting and provide deeper data analysis.
Have you ever wondered how radio and television stations find out who's tuning into their programs? BBM Canada, founded in 1944 as the not-for-profit Bureau of Broadcast Measurement, conducts audience research across the country to provide this information to its members, who include television and radio stations and networks, advertising agencies and advertisers.
The company collects audience data using Portable People Meters (PPM) in a carefully selected panel of homes, and diary surveys conducted in more than 100 radio and television markets. A few times a year, paper diaries are sent to thousands of Canadian survey participants who use them to track their viewing or listening over a seven-day period before mailing the surveys back to BBM for analysis that determines program ratings.
"Ratings are the broadcast industry's currency – they determine the value of commercial time," said Derrick Gray, who is Director of Research Sample Design and Control for BBM. "Our members depend on us to provide accurate audience data because our research has a direct impact on advertising costs and revenues."
Gray and his team of analysts are responsible for sample management and data quality for all of BBM's electronic and paper-based surveys. His group manages the entire survey process – including breaking up the country into geographic sample strata, determining sample targets based on station numbers and population size, enumerating households to select participants and managing electronic panels, then taking in the results and analyzing the data with SAS® Enterprise Guide® to determine ratings.
"In a given survey-diary sweep we could have 90,000 respondents. The datum is captured at the quarter hour for an entire week per respondent – so 90,000 times seven days times 24 hours times four quarter hours," said Gray. "That is a huge volume of data, and SAS makes it a lot simpler to manipulate. It gives us a single, user-friendly solution for modeling, analysis and reporting."
Not every household completes the survey diary or returns it, so BBM must account for this gap by over-recruiting participants. The company uses the time series forecasting in SAS/ETS® software to predict how many diaries it needs to send out for accurate sampling.
"Using SAS time series forecasting to predict how many diaries we need is extremely useful; we've set up return rates at each sampling strata and we model these for each survey. Based on our models we know how many diaries we need to get back for what we want," Gray explained. "The problem is that sending out too many diaries incurs extra costs, but if we don't send out enough, we don't meet our sample target and standard errors increase."
Gray said BBM tries to be between 4 percent below and 6 percent above target for each survey sample, and with SAS, the company is seeing an improvement in accuracy. "With the models SAS allows us to build, we're getting about 80 percent of our markets in that range. Before we were less than 60% accurate from survey to survey; a big benefit of SAS is that it's helped us become consistent."
"In a market like Toronto, for example, you may have 4,700 diaries as a target. If you're 20 percent over, it's about 1,000 extra diaries at a cost of $22 per diary, or $22,000. So when you have total sample levels of 90,000 per survey, being 5 percent over is better for costing," Gray said. "We've definitely seen dollar savings as a result of better targeting using the modeling functionality in SAS."
SAS also helps make the modeling process simpler and faster. With about 380 sampling strata for radio and another 400 for TV, BBM needed an automated solution that builds models quickly and accurately so analysts can devote their time to fine-tuning forecasts.
"Part of a good forecast is the level of adjustment done by the analyst. With all the automated model-building processes we have using SAS, a 300 time series linear regression can be built out in 10 or 15 seconds. I've got that output automatically – I don't have to worry about doing the math and building an Excel spreadsheet," Gray said. "SAS will fit every single appropriate model and then spit out the one that produces the lowest error base, for example – or whatever I've set the criteria for in the software."
The results illustrated in this article are specific to the particular situations, business models, data input, and computing environments described herein. Each SAS customer’s experience is unique based on business and technical variables and all statements must be considered non-typical. Actual savings, results, and performance characteristics will vary depending on individual customer configurations and conditions. SAS does not guarantee or represent that every customer will achieve similar results. The only warranties for SAS products and services are those that are set forth in the express warranty statements in the written agreement for such products and services. Nothing herein should be construed as constituting an additional warranty. Customers have shared their successes with SAS as part of an agreed-upon contractual exchange or project success summarization following a successful implementation of SAS software. Brand and product names are trademarks of their respective companies.
Copyright © SAS Institute Inc. All Rights Reserved.
Derrick Gray, A.Stat.
Director, Research Sample Design & Control
Provide accurate audience data on millions of Canadian households so television and radio stations, networks, advertising agencies and national advertisers can determine the value of radio and TV commercial time in order to price it accordingly.
SAS Enterprise Guide
With SAS, BBM Canada has automated the forecasting process while ensuring more accuracy so that its clients can better price commercial time.
“With the automated model-building capabilities in SAS, a 300 time series linear regression can be built out in 10 or 15 seconds.”
Derrick Gray, A.Stat.
Director, Research Sample Design & Control