Canadian businesses are at a crossroads. As the world economy continues to struggle, and competition intensifies, companies must work smarter to survive. But this is no simple task as, in our knowledge-based information economy, sifting through the vast amount of data required to make smart, insight-driven decisions takes time that many companies don’t have.
Numerous recent reports point to Canada lagging in both productivity and innovation compared to other countries. A report from the Conference Board of Canada awarded our (Canada’s) performance for productivity a C rating and a D rating for innovation, thereby making it increasingly difficult for our businesses to succeed globally and for our economy to grow.
Unfortunately, at the same time, Canadian businesses are at risk of falling behind international competitors in the era of big data as a recent survey conducted by IDC Canada and commissioned by SAS shows our businesses are failing to adopt the advanced technologies that can help them harness the power that big data holds.
Big data presents both a challenge and opportunity to businesses. IDC estimates that by 2020 the amount of data created and replicated will rise to 90 zettabytes. Social interactions, mobile devices, facilities, equipment, R&D, simulations and physical infrastructure are all part of this.
Specifically, the IDC survey found that while 96 percent of Canadian companies say the ability to process and act on data in real time is important, less than half (48%) have invested in the technologies to do so, and 15 percent have no adoption plans for the future. That’s dramatically lower adoption than the two thirds (76 percent) of international companies interviewed in another recent survey by SAS that have already adopted big data technologies.
Demystifying big data
What is behind the low adoption of big data technologies? IDC has suggested that lack of understanding on the part of Canadian businesses of what big data is and the technologies that can overcome the challenge is the likely culprit.
While one of the hottest business buzz terms, big data is nothing new. Companies large and small have long been faced with massive amounts of data. Big data is a popular term used to describe the exponential growth, availability and use of information, both structured and unstructured. Here are some key components that help define what big data looks like:
- Volume: Many factors contribute to the increase in data volume – transaction-based data stored through the years, text data constantly streaming in from social media and increasing amounts of sensor data being collected are just a few sources. In the past, excessive data volume created a storage issue. But with today’s decreasing storage costs, other issues emerge, including how to determine relevance amidst the large volumes of data and how to create value from data that is relevant.
- Variety: Data today comes in all types of formats – from traditional databases to hierarchical data stores created by end users and OLAP systems, to text documents, email, video, audio, stock ticker data and financial transactions. By some estimates, 80 percent of an organization’s data is not numeric, but it still must be included in analyses and decision making.
- Velocity: The speed at which data can be analyzed and delivered is another critical factor. Reacting quickly enough to deal with velocity is a challenge to most organizations. While 48 percent of Canadian respondents in the IDC survey said that the speed at which their organization processes data has increased over the past 12 months – and of those, 38 percent said that it was significantly faster – that rate of change still pales in comparison to our global peers. Around the world, 64 percent of organizations indicate that their speed was increasing.
- Variability: In addition to the increasing velocities and varieties of data, data flows can be highly inconsistent with periodic peaks. Daily, seasonal and event-triggered peak data loads can be challenging to manage – especially with social media involved.
- Complexity: When you deal with huge volumes of data, it comes from multiple sources. It is quite an undertaking to link, match, cleanse and transform data across systems. However, it is necessary to connect and correlate relationships, hierarchies and multiple data linkages or your data can quickly spiral out of control. Data governance can help you determine how disparate data relates to common definitions and how to systematically integrate structured and unstructured data assets to produce high-quality information that is useful, appropriate and up-to-date.
IT in the C-Suite
A further challenge brought to light by the survey is the value Canadian businesses’ place on IT. By and large, strategic data decision making is being relegated to mid-level IT managers, rather than being viewed by the C-suite as a critical matter. In Canada, mid-level IT managers are close to six times more likely than the international average to be primarily responsible for data management strategy, with a quarter of Canadian companies placing such decisions in their hands versus 4 percent internationally. Worldwide, Chief Information Officers are the key strategy drivers for a third of organizations.
The result? International companies have a longer big data technology adoption track record and more defined plans for adoption in the near future of these technologies that are assisting organizations become more innovative and productive.
This is also troubling as due to the complexities of big data, a solid data governance strategy is important to ensure all teams are equipped with well-managed, integrated and high-quality enterprise data that can provide the foundation for good business decisions. When mid-level managers are charged with overseeing data strategy, we frequently see a lack of enterprise-wide data governance.
Faster, more fulsome analysis with high-performance analytics
Whether it’s a retailer processing customer transactions, a financial institution looking at millions of records to identify fraudulent activities, or a telecommunications company attempting to predict churn risks, big data technologies can prove an invaluable ally.
Powerful analytics technologies such as SAS High-Performance Analytics can turn data into knowledge in near real time, reducing the time it takes to analyze billions of pieces of data from days or hours to minutes or seconds. It also helps businesses make more informed decisions by enabling them to analyze all the data they have, not just a sample as has only been possible to date.
What does this look like in action? A sporting goods retailer recently used an HPA solution to increase its direct marketing response rates by 60 percent by studying the geographies of customers most likely to generate the greatest possible incremental sales. The retailer now also has a more complete understanding of the value of promotional marketing, and how much – and where – to spend those promotional dollars.
The IDC survey found Canadian companies are only using a fraction of the vast amount of information available at their finger-tips. An overwhelming majority of respondents (76 percent) are using internally produced data, but other valuable sources such as social media (32.7 percent), web data (34.7 percent), RFID tags (26 percent) and GPS (16.7 percent) have yet to gain significant traction. With big data technologies, analyzing these data sources is faster and simpler than ever so decisions take into account all the data, not just a sample of it, which results in more sound decisions.
Of course timely decision making often means the difference between business success and failure. Canadian companies polled see great value in real-time data as of the benefits respondents attributed to faster data processing, the ability to generate operational efficiencies was most important (52 percent), followed by enabling informed decision making (24.7 percent) and improved customer service (23.3 percent).
As Canadian businesses face the crossroads between conducting business as usual with decisions based on gut, and speeding ahead to embrace the potential of big data, it will be those who see the potential of big data technologies who will see benefits to productivity, growth and innovation.