Better Storage for Better Business Intelligence

Driving down the hidden costs of data warehousing by using the right storage at the right time

By Mark Torr, SAS

Online transaction processing systems, or OLTPs, have been the heart of many organizations for years - handling a constant deluge of transactions from operational applications such as airline reservation systems, point of sale systems or inventory control systems.

There's no doubt about the critical role these systems play. But OLTPs are extremely complex and require significant disk overhead to maintain data integrity during updates. This complexity has led to increased overhead in terms of the database administrators required to manage them and the large collections of costly subsystems used to store the data.

The typical data storage of choice for operational applications that require OLTP capabilities is relational database management systems (RDBMSs) because they have been designed to address the "always-on,"  constant stream of small transactions required by  operational systems – and still maintain data integrity during the many updates they endure.

Because RDBMSs were functioning well for these operational applications, IT managers at some companies thought it was logical to apply them to other types of applications. 

But when you begin to deploy business intelligence applications or take advantage of advanced analytics throughout the organization, relational database management systems fall sorely short as a storage strategy. Today's organizations are seeing increasing data volumes. They're maintaining more historical information. And they're supporting a growing number of business intelligence initiatives with applications that generate large numbers of queries.

It comes down to this: The storage you have in place to support intelligence activities in your organization is now critical to attaining a competitive edge and keeping costs down. Data storage directly affects the speed and agility of your organization and may even be a contributor to costs that you could easily eliminate. As such, it deserves a strategic focus. And if using relational database management systems for these purposes isn't already posing a problem, it soon will, in terms of both cost and performance.

The risks for business intelligence
RDBMSs were never designed to function outside the world of operational applications. RDBMS vendors continue to claim that customers can overcome the limitations by throwing more hardware – including CPUs, costly storage subsystems and memory – at the problem, which causes an escalation of hardware and software expenses that most organizations didn't plan for. 

If we dive down further under the covers of the RDBMS, it becomes obvious why it is not a good strategic storage option:

• RDBMSs were designed to support large numbers of transactions and are required to be able to reverse any failed or truncated transaction to maintain data integrity during update. Their workings, therefore, are extremely complicated, which adds significant disk space overhead. In the BI world, there is little or no need for most of this overhead. This becomes a major issue when you get into terabyte levels of storage because your disks are not being used by your data, but by the RDBMS for no apparent reason. You're actually paying twice: once for the storage the RDBMS uses over and above your raw data and once for its software. This also significantly complicates administration and maintenance.
• RDBMSs must be maintained by specialized database administrators. As you add more RDBMSs into your organization, you'll need to add significant numbers of DBAs. Also, most database administrators are skilled at tuning these systems for operational applications, not strategic BI applications. This isn't the fault of the DBA; try to find information on tuning an RDBMS for the analytics required for business intelligence applications!
• RDBMSs are designed for a high volume of quick updates and small numbers of simple queries. This means that the high volume of simple queries and the higher-intensity analyses to satisfy business intelligence and analytics requests, such as full table scans or large numbers of long, complex queries from many users, often overload the RDBMS, causing significantly slower response times. RDBMS vendors will suggest you build different tables for different tasks (again increasing complexity and disk utilization) and will insist on different indexes for different styles of usage or simply advise you get a bigger piece of hardware to hide their shortcomings. Again, this complexity will require the help of more database administrators. 
• Most RDBMSs are confined to a limited number of platforms, which restricts the architecture's flexibility. Some RDBMSs are even residing on proprietary hardware that, once committed to, can require costly upgrades later.

A better storage landscape
In the business intelligence world, it's common to load massive amounts of data and support large numbers of users who issue lots of queries and can often start long, complex queries. So while you can choose to use an RDBMS for things like business intelligence or analytics, it means you're taking technology designed for one purpose and applying it to a task it was not designed for.

Think of an RDBMS as a truck – a big, bulky, resource-guzzling machine that constantly needs help to keep running. It is entirely out of place in the Formula One car race of business intelligence.

SAS Intelligence Storage is the highly tuned race car. With SAS Intelligence Storage, you can lower your costs and bring an end to the problems associated with using an RDBMS in ways it was never designed to be used.

SAS Intelligence Storage is not an individual offering. Rather, it is a collection of interoperable data stores that have been designed with the intelligence landscape in mind. Each store has been built to suit the needs of the organizational area and the typical skill level of the person who needs to create, manage and use it. For example, SAS has storage options that support:

• Desktop storage, so business analysts can query data locally and data miners can build models that can be deployed against other data stores.
• Subject-specific data marts.
• Multi-dimensional storage to deliver pre-aggregated data to the organization. 
• Enterprise data warehouses or certain aspects of the ODS layer.

SAS Intelligence Storage differs from an RDBMS because it has been designed to handle the workload that the operational RDBMS cannot. SAS Intelligence Storage was designed to provide for the loading and storage of vast amounts of data and to support both regular and ad hoc queries from large numbers of users - without the overhead of an RDBMS. In fact, with SAS Intelligence Storage installed at the data warehouse or data mart level, organizations have seen the following benefits:

• Four times less disk space utilization.
• Performance increases by up to a factor of 40. 
• Hardware life extended where previously there was an impending cost.
• Newfound freedom to move platforms, which allows the organization to meet modern business requirements at a significantly lower cost. 
• Four times less support staff required.

Organizations need to re-examine their storage landscape and eliminate unnecessary overhead and costs. At SAS we stand ready to prove our performance and our lower disk footprint, and show how our ease of maintenance will help you get the costs associated with your data warehouses and storage back under control.

In addition, SAS Intelligence Storage provides the platform agility and flexibility that allows you to take advantage of the most optimal hardware platform and operating system environments – now and in the future.

Bio: Mark Torr is a Technology Strategy Manager for SAS EMEA.