Data Warehousing: The Foundation for All Analyses

by Susan Cohen

No matter what analytical tools you use – customer relationship management, decision-support systems, online analytical processing (OLAP) or others – your insight depends on complete and accurate data. Data provides the underlying foundation for every single analysis you perform. So you should be asking if you can make your data more complete and accurate.

If your data sits in silos – each division or group owning its own collection of information and making decisions based only on that data alone – then the answer to your question is a resounding YES!

Why should you care if your data is incomplete? You can still use data mining and predictive modeling techniques, right? Well, you can, but you can also buy a house with a faulty foundation. Would you want to? In either situation, your results are not dependable.

Using incomplete data, multiple divisions of the same company might contact customers, annoying them with inconsistent messages and offers, and leaving your company vulnerable to a more "coordinated" competitor.

Many organizations rely on weak data foundations as a result of poor data integration, data collection, data naming and data definition practices. Inconsistent data translates into inconsistent actions across the organization. Ideally, all areas of the company should be making decisions from the same data, branching out from the same solid foundation.

What can you do?

Build a warehouse
A data warehouse is likely to be part of the solution. The warehouse can provide a single repository for data collected throughout your organization as well as third-party data. It collects data from a variety of sources – transactional systems, customer service systems, manufacturing processes, ERP systems, do-not-call lists and more. The data is stored in a manner that is optimal for analysis, usually with redundancy to improve analytic capabilities. Typically, data warehouses are located away from vital real-time systems to avoid slowing down day-to-day operations.

Here’s how a data warehouse might work: data is collected from across the organization, as well as from third-party sources. All this data is stored in a single warehouse, where it is cleaned and organized as it enters. Various systems and reports then call on the data to provide information for better customer service, decision making and other purposes. Employees are armed with data from throughout the organization at every customer touchpoint and within every report.

Document information about the data
One of the challenges of a data warehouse is communicating information about the data. Managers who interact with the warehouse need to understand how the data is collected, constructed and reported. Metadata – or "data about the data" – is critical to a successful data warehouse solution. Metadata provides information about what data is available in the warehouse and defines the information. (For example, what are the definitions of the variables and the data?) It also reveals the data collection process. (For example, data about recency comes from transactions from seven of the divisions and is updated daily.) Keeping metadata current can help ensure that managers fully understand what the data is saying. Using metadata, managers might learn when a new line was added in a factory (explaining sharp increases in output capacities) or that certain data is updated on Mondays (leading the manager to run reports on Mondays instead of Fridays).

Unleash the arsenal you’ve created
With the data warehouse in place, managers throughout the organization can access the data to make better decisions. Typically, they use decision-support systems to uncover information stored within the warehouse. These systems include OLAP, data visualization, data mining and executive information reporting systems. Using such systems, as well as a complete data warehouse with explicit metadata, most managers can unleash the power of the data from their desktop. Not only is the data now more complete, it is also more accessible.

Some organizations are empowering their managers even further with collaborative business intelligence technology that allows companies to store reports, graphics and other intelligence with complete documents, e-mail and even customer notes. Armed with all this information, personnel can maximize each area of customer contact.

Realize the gains
Creating a single repository for data and integrating data from across the organization and beyond gives a business an incredible competitive edge. (In fact, your warehouse might even include competitive information that is collected and used throughout the organization.)

Providing this solid foundation for data mining and predictive modeling results in increased insight. Since a warehouse includes all information collected about all customers, the organization can act in a more coordinated way. Offers can be delivered in a rational sequence, so the company doesn’t compete with itself and so the customer receives a clear, unified message. Note that even the best data warehouse does not ensure improved customer communications – managers need to learn how to use the data warehouse to coordinate all activities.

By consolidating data and looking at a customer’s interactions with all channels and products as a whole, companies can perform more sophisticated analytic techniques, such as optimization, to ensure that customers get the best mix of offers. The end result? Maximum profit potential.

Respect thy customer
A data warehouse can be an extremely powerful tool. Often, it includes very personal and timely information about customers. It is critical that businesses adhere to strict privacy policies to build stronger relationships with their customers.

Many organizations get excited about analytics. But beware of analytics built from incomplete data. Like a home built on a weak foundation, analytics built on incomplete data may collapse, potentially causing more harm than good. Be sure to start any analytic project with a strong data foundation.