Supercharging capital markets with real-time data visualization

Better manage market risk, liquidity and counterparty credit risk during the trading day

By David Wallace, Global Financial Svcs, SAS

The payoffs from rapidly analyzing big data are well-known in trading. However, a number of instructive events over the past few years show failure of global financial services firms in evaluating their data in time to avoid negative consequences.

Fines from regulatory compliance lapses, collusion and violations of international laws totaled more than $10.7 billion in 2012, according to That total does not include write-downs from trading losses ($6 billion at one firm in 2012) and from other financial impacts, including reductions in market capitalization ($9.9 billion in 2012 at another firm).

These regulatory challenges are accelerating much-needed improvements in firms’ information management practices. “The data guys are getting their say at the corner office and getting the budget to drive many of these regulatory initiatives,” said Larry Tabb, Chief Executive Officer of the TABB Group, a financial markets research and advisory firm.

Optimization is no longer a quarterly or monthly reporting cycle; it is an almost immediate response to market, capital and risk factor changes as they happen.

The technologies implemented for compliance have the added advantage of driving business improvements. If done right, it can achieve additional benefits. Under pressure for greater transparency and risk awareness, firms stand to gain a lot from the data management revolution.

Optimization is no longer a quarterly or monthly reporting cycle; it is an almost immediate response to market, capital and risk factor changes as they happen. “Increasingly, we hear that clients are trying to obtain an array of risk metrics more in real time, released multiple times during the day, not just as an end-of-day or over the weekend,” said Tabb.

Capital markets firms already have a great deal of sophistication in dealing with data, but the velocity and variety – not just the volume – present challenges such as:

  • Aggregating risk exposures to interactively analyze, explore and drill down to business unit, desk, portfolio, instrument or horizon.
  • Offering up-to-the-minute assessments of risk exposures for large, complex portfolios of financial instruments and rapidly analyzing (in near-real time) incremental value at risk (VaR), counterparty exposures and liquidity measures.
  • Dynamically and interactively stress testing to anticipate the impact of extreme events on portfolio values.
  • Analyzing unstructured data, such as those millions of tweets potentially yielding clues to validate existing trading strategies and launch new ones.
  • Continuous surveillance to identify and prevent rogue trading, internal and external fraud, regulatory violations like money laundering, and market crashes.

What’s the solution to finding “the devil [that] is in the detail we can’t see”, as Miranda Mizen of TABB put it in an October 2012 presentation? The answer is to apply analytics that delivers the velocity and volume needed to deliver real-time visibility and impact from big data. Two example technologies are event-stream processing (ESP) and in-memory data visualization.

While capital markets firms use ESP to support trading processes, pairing ESP with other high-performance, in-memory analytical techniques can deliver high value in areas like risk and liquidity management, rogue trading, fraud and compliance. By performing deeper analyses on data captured in-stream to reveal unseen patterns, sentiments and relationships, and then injecting the results back into the business in real time, firms can act faster with greater foresight. This helps them to dynamically revalue portfolios and manage limits during the trading day, identify suspicious patterns to reduce trading and fraud losses, and also capture new business from institutional clients.

Better risk management and continuous surveillance

For risk management, firms can combine ESP and high-performance risk analytics for on-demand, intraday valuations of large portfolios of complex financial instruments for market, counterparty credit and liquidity risks. In-memory analytics provides interactive analyses of consolidated risk exposure and stress testing for instant response and visualization at many levels, including business group, portfolio, desk, instruments and time horizons. While traders can manage their own positions and portfolios on a near-real-time basis, it is the aggregation of the firm’s position that gives the greatest transparency. The two solutions together can address firms’ needs for continuous limits monitoring, real-time risk aggregation and dynamic portfolio valuations. The bottom line? Better management of market risk, liquidity and counterparty credit risk during the trading day rather than waiting for the overnight reports.

For continuous surveillance, ESP can be combined with fraud management solutions to detect and prevent rogue trading activities and abuse of banking regulations like money laundering. Together, these solutions can provide the continuous compliance processes that senior executives and management boards now require.

ESP acts as the prevention engine, listening to all trading activity in real time while also analyzing data from watch lists, business rules, scores, and suspicious trading patterns. The prevention engine continually feeds the detection engine with relevant trades and trade patterns in real time. This creates an aggregated database of data which acts as a reference point for future simulations and continually improves the accuracy of the system. The detection engine applies a number of sophisticated high-performance analysis techniques to the database and provides detection alerts to appropriate staff for further investigation, or sends the alert back into ESP to stop the trading (or other) processes from continuing. These same techniques can also be used to detect and prevent other forms of operational risks like external fraud and cybercrime.

While data visualization products are also used in capital markets firms today, it is the coupling of the visualization engine with an in-memory analytical engine that supercharges the level of insight from big data sources. Large data sets (perhaps billions of rows) can be opened in seconds and explored easily and visually by business analysts using analytical and charting capabilities like box plots, correlations, binning and distributions, word clouds and network diagrams (for unstructured data), along with on-the-fly forecasting, auto-charting, “what does it mean?” pop-ups, and drag-and-drop capabilities. Identifying correlated patterns in institutional customer transactions can help with many initiatives, such as more targeted sales pitches for additional business, scanning through millions of trades to quickly identify data issues causing abnormally high exposures, or back-testing trading strategies and market signals.

The bottom line? With ESP and in-memory data visualization combined with other high-performance analytics, capital markets firms and global banks can make better decisions faster to capture more revenue, better manage risks and protect themselves from internal and external fraud.

NOTE: Originally published in the Tabb Forum.