You Can't Do That with BI – Or Can You?

Three emerging uses for automated business intelligence

By Allan Russell

Traditionally, business intelligence has been used to improve corporate performance and alignment by producing metrics that – if attained – will lead to organizational success. One common use of BI has been to improve decision making in an organization by providing data that will influence decisions within a strategic, tactical or operational framework.

Traditional types of BI activities have focused on helping decision makers improve their understanding of how the organization works, with the expectation that decision makers can effect changes that would lead to better results.

But what if you want to improve business processes themselves and use the BI system to intelligently monitor, influence, adapt and change the operational systems without manual intervention? Is that possible?

With SAS, it is – and the benefits include reduced costs, improved decisions and efficiency gains.

This column will look at three emerging uses of BI that provide automated or embedded business intelligence to business processes themselves: feedback loops, real-time BI and business-activity monitoring.

The classic BI infrastructure
A typical BI infrastructure includes operational system elements, their associated databases and possible replications of their data, such as data warehouses or operational data stores. On top of the data warehouse, you will often see managed data marts, which are tailored to support specific analytic applications.

In classic BI, an analyst uses an interface to access data from reporting and analytics engines. Warehouse data flows into these engines, which prepare forecasts, format reports, build optimization models and complete many similar tasks.

The analyst is responsible for communicating any knowledge gained from the reports or analytics to the business process owners or information consumers, who presumably know how to improve their processes based on the information provided.

Provide feedback to your BI infrastructure
Feedback loops offer tighter integration between the BI infrastructure and the applications that implement business processes. In this model, some of the results that flow to the analyst from the reporting and analytics engine also flow automatically back into the operational systems. Results from forecasts, cost models and optimizations are integrated at the data level and returned as inputs to the application databases, or possibly back into the data warehouse or managed data marts.

This technique is most suitable for applications where there is a cyclical business process such as monthly, biweekly, or even daily planning or buying. Information consumers will benefit almost immediately because the application considers the new results each time it makes a new recommendation.

For example, a global manufacturer of bacon, sausages, pies and pâté recently implemented feedback loops into its biweekly buying cycle for pig purchases. Typical factors in the buying cycle include the different types of breeds, the price on the market, the product demand and the processing methods used. The BI system considers the results from the previous buying cycle and determines how many pigs to buy and how to process the pigs.

After each buying cycle, the optimization analyses are rerun and fed back into the manufacturer's SAP R/3 system. After 12 months of using the system, the manufacturer continues to see improved savings on every purchase cycle.

To implement feedback models, the results of analyses are yielded by a batch-type process and should be inspected by controllers or data experts before being pushed back into the operational application. The main challenges to overcome include understanding the appropriate format for the data that the operational application will expect and making any necessary modifications to the operational application so it can understand and present the incoming data correctly. The good news is that many standard applications already have the necessary hooks and APIs to allow for this way of working.

Support on-the-spot decision making
How can BI assist with decision making on the fly? Can BI respond to an external event such as the swipe of a credit card or the arrival of a telephone call? When organizations set up BI systems to support on-the-spot decisions, they can help employees make faster, more informed decisions that improve the ability to cross-sell products, halt fraudulent activity and more.

In these scenarios, the business process or operational application will need to "call out" to the BI infrastructure for help. This call will trigger the scoring or recommendation engine to execute very quickly and return a result in the form of a recommendation or suggestion. For example, the result could be a go/nogo decision for a potentially fraudulent transaction, or it could be a "next offer" suggestion based on a customer rating.

In these cases, prescoring techniques are commonly used, but prebuilt models do not respond dynamically to changes in the customer situation because the scores typically get calculated some time before the previous push cycle.

The steps of classic BI are still necessary to build the real-time models based on historical data. The difference is that now the operational application calls out to get access to the results of the model, instead of calling up a file of stored results that may have been sitting in the reporting mechanism for several weeks or months.

A number of international banks have successfully implemented this type of system. One retail banking customer uses it to generate a response of either "yes," "no" or "refer" within milliseconds of each swipe of a credit card. Refer means customers are passed on to the case management system for further processing.

In this example, the retail bank constantly updates customer changes in the customer information control system and built a data mining model to detect fraudulent transactions. The model analyzes data in the operational system that has not made it into the data warehouse yet, and the system uses message queuing technologies to communicate the decision back to the system instantly. The result for this bank was increased fraud detection and decreased loss.

Real-time analyses involve interapplication communication, so we recommend an open infrastructure. One way to achieve this is to implement a services call from the operational application to the BI infrastructure. Another alternative is to run the model "inside" the operational database. This latter option allows for more data to be pulled together in order to make the decision. If the model runs "inside" the database, it can quickly access all the data available for analyses.

Organizations must take care to keep track of the different models that exist for analyses. They need to monitor model performance as the real world drifts or shifts against the model's representation. As soon as such changes are identified, we recommend creating a challenger model to see whether performance improves. Depending on whether the challenger beats the champion model, a switch may be needed - all these stages of a model's life cycle will need to be tracked and managed, and model management tools are emerging to do that work.   

Send alerts – and suggestions – to improve business processes
When the BI infrastructure looks out for abnormalities within a business process and then takes action to correct those abnormalities, this is called business-activity monitoring (BAM). In a typical BAM or alert situation, the business application either sends an event message to a decision maker when something abnormal occurs or the BI infrastructure is set up to automatically listen for internal message events from the business application.

In the second scenario, the BI infrastructure monitors events in real time and applies rules to determine when the process experiences an abnormality, such as a very high load of a particular transaction type or an unacceptably long delay in processing particular transactions.

A European post and parcels organization that wanted to optimize resource and route planning for parcel deliveries found success with BAM. Resource allocations needed to be determined in real time for each shift based on the activity volumes that day. The company's system automatically sorts and records the movement of each package. The BI infrastructure listens to the message queue and fills the data mart with volume information that is always up to date. Business rules determine allocations just before each shift, and that information is communicated directly to the operational system as alerts, and automatically schedules shifts and delivery routes for optimum speed.

To achieve business-activity monitoring, some new elements will appear in the BI infrastructure, including the alerts engine and the business rules repository, which drives the alerts engine. The alerts engine pushes events to a decision maker who may be using a dashboard that refreshes key metrics regularly or who may be notified by e-mail when a threshold is exceeded. Assuming that the operational application can consume changes, the alerts engine can also push actions to the operational application to cause change in behaviors, such as changing routing parameters based on changed loads or causing particular classes of customer transactions to receive higher priority.  

Managing the emerging BI infrastructure
Today, there is a shift in BI from looking exclusively at the data to understanding the role of BI in the business process. While a number of styles exist that allow for event-driven or periodic access to the BI infrastructure, the core of that infrastructure will remain the same.

As organizations begin to use new styles of BI, however, we must recognize that simply deploying one of the styles discussed above will not suffice. The key challenge will be repeatedly building and managing many systems with a mix of styles. Areas of management and governance to focus on include:

• Classic BI activities, such as building OLAP cubes or managing multiple ETL flows.
• Model lifecycle management as the number of models increases.    
• BI services, which will become more important and will be managed in the same way as other services. 
• Business-activity monitoring strategies, which should be developed to manage the many alerts generated from various rules and governing multiple interlinked operational systems.

When implemented correctly, embedded BI inserts business intelligence into the processes that affect the day-today operations of an organization. It does not replace traditional BI but helps BI become easier to use, more effective and more pervasive.

Bio: Allan Russell is a SAS Fellow who researches and reports on emerging trends in the software industry.