ScottishPower rises to the challenge

Using powerful SAS technology to facilitate compliance and spark massive improvements in the customer journey

ScottishPower is one of the top six energy providers in the UK, serving more than five million households and businesses. Founded in 1901, the company is a fully-owned subsidiary of Iberdrola, one of the world’s largest utilities.

Meeting tough compliance requirements

Energy is one of the UK’s most heavily regulated industries, and providers are obliged to keep pace with a constantly evolving set of rules and regulations. When the Competition and Markets Authority (CMA) launched a comprehensive inquiry into the UK energy market, utilities were faced with yet another challenge.

Duncan Bain, Senior Data Scientist for the Market Investigation and Business Intelligence Team at ScottishPower, begins: “The CMA asked all utilities to supply data about the tariffs paid by current and former customers on a monthly basis going back more than five years. Given that we serve around 5.5 million customers, the quantities of data involved are immense.

“To further complicate matters, we had recently migrated from our legacy mainframe platform to a new SAP billing system. Because there had been no business requirement to transfer all historical data over to the new platform, we needed to extract data from both systems to get the full customer history.”

Initially, ScottishPower relied heavily on manual processes to extract the relevant data. Sourcing data from the mainframe was relatively easy, because the records were purely historical and not subject to change. However, refreshing live customer data from SAP proved much more challenging.

“To maintain high performance, the SAP system limits the amount of data that can be extracted at any one time,” explains Duncan Bain. “The IT team wrote code to extract the relevant data in chunks, and each batch had to be run separately. To maximize efficiency, the team undertook a tiresome trial-and-error process to find the optimal size of chunks of data for extraction.

“Before submitting information to the CMA, we needed to reassemble the chunks, then cleanse, analyse and verify the data. Because the extraction was taking so long, we were going to be left with little time to complete those other tasks before the deadline elapsed. If our submission was late or inaccurate, we risked incurring significant penalties, as the CMA can include the timeliness and quality of data-provision by each company in the final report.

“To overcome the challenge, we had to find a better way of working.”

Cutting-edge SAS technology helps us stay ahead of the curve in analytics.

Duncan Bain
Senior Data Scientist for the Market Investigation and Business Intelligence Team

Lightbulb moment

ScottishPower decided to deploy SAS® Data Surveyor, which supports tight integration with enterprise applications, alongside SAS Data Integration Studio, which provides powerful tools for building, implementing and managing data-integration processes.

“We selected the SAS proposal because it was backed by a proven track record of successful implementations,” adds Duncan Bain. “Additionally, the solution offered our analytics team full control of which data was extracted, and when – enabling us to make changes such as adding new database tables and fields on-the-fly, without consuming substantial IT resources.”

An expert team from SAS helped ScottishPower implement the solution and integrate it with the core SAP systems.

“SAS provided full support during the implementation. We have been working with SAS for years, and we’re always happy with the service.” With the implementation complete, ScottishPower used the SAS solutions to pull data from the SAP application into the analytical platform. The SAS solutions helped ScottishPower match records from the mainframe and SAP systems to gain a single view of each customer’s tariff history. Next, the company used SAS to join up the chunks, cleanse and verify the data, then convert it into the format stipulated by the regulator, ready for submission.

“SAS Data Surveyor has a very light touch on the system of records, so unlike many competing solutions, it has a minimal impact on SAP performance,” remarks Duncan Bain. “That was a critical factor, because many of our employees rely on SAP for their day-to-day work, so slow response times hamper their productivity.

“Using SAS enabled us to provide much more accurate data to the CMA, because retrospective changes – such as corrections to contract dates – are detected automatically using change data capture functionality. What’s more, because we capture only the data we need, we have accelerated extraction from two weeks to just two hours.”

Mining for golden insights

To help it get ahead in the competitive energy market, ScottishPower seized the opportunity of digging out historical data to perform valuable business analysis.

“For years, our customer data had been trapped in silos,” comments Duncan Bain. “Now, we are bringing it all together to gain a 360-degree view of the customer lifecycle, and using SAS Enterprise Guide to analyse it. This consolidated customer view also enables the creation and execution of predictive models, using SAS Enterprise Miner, which are used to drive customer engagement.

“We can unlock insights into what makes customers churn, enabling us to pinpoint accounts we risk losing and intervene early on. By working out which brokers sell to small and medium-sized companies most effectively, we can make smarter decisions about which ones we work with. And by examining how and why customers contact us, we can spot opportunities to solve their problems first time around. Long term, our sights are set on predictive account maintenance, which would enable us to resolve issues with customer accounts before they even realise that there is a problem.”

Duncan Bain concludes: “The priceless insights that we have uncovered with SAS have sparked a broader appreciation of the value of data across ScottishPower. Cutting-edge SAS technology helps us stay ahead of the curve in analytics, and be perceived as a leader among Iberdrola subsidiaries.”


Faced with a tough compliance requirement, ScottishPower found that slow data extraction threatened its ability to supply information to the Competition and Markets Authority (CMA) on time, risking fines and reputational damage.


SAS® Data Surveyor
SAS Data Integration Studio
SAS Enterprise Guide
SAS Enterprise Miner


  • Accelerates data extraction from two weeks to two hours
  • Enables timely provision of all relevant data to the regulator, facilitating compliance
  • Reveals new insights into customer journeys, highlighting opportunities for improvement
The results illustrated in this article are specific to the particular situations, business models, data input, and computing environments described herein. Each SAS customer’s experience is unique based on business and technical variables and all statements must be considered non-typical. Actual savings, results, and performance characteristics will vary depending on individual customer configurations and conditions. SAS does not guarantee or represent that every customer will achieve similar results. The only warranties for SAS products and services are those that are set forth in the express warranty statements in the written agreement for such products and services. Nothing herein should be construed as constituting an additional warranty. Customers have shared their successes with SAS as part of an agreed-upon contractual exchange or project success summarization following a successful implementation of SAS software. Brand and product names are trademarks of their respective companies.

Back to Top