Using text analytics to augment intelligence gathering

By Tom Sabo, Principal Solutions Architect, SAS

To prevent crime, fraud or terrorism, it’s important to use every tool available. In a big data world, that includes the mass of data from social media, email and other communications methods. The challenge is to analyze that data to uncover patterns that indicate a crime might occur.

The US Congress has taken notice of the power of analytics in the realm of fraud prevention through the proposed Fraud Reduction and Data Analytics Act. Why analytics? In short, it works.

Social media is providing the IC with huge amounts of critical, relevant information, including specific events and occurrences, relationships between people and organizations and even recruitment tactics.

The increased application of analytics as a fraud prevention control has resulted in a 284 percent return on investment for the Centers for Medicare and Medicaid Services’ Fraud Prevention System and holds further promise for reducing the overall 4 percent federal improper payment rate that occurred in FY 2014. This 4 percent translates to $125 billion in improper payments. By implementing analytics, federal oversight offices can analyze both structured and unstructured data and stem the tide of fraud and improper payments.

The value of analytics is apparent within the defense, intelligence and law enforcement communities as well. The Department of Homeland Security, in its National Critical Infrastructure Security and Resilience Research and Development Plan, points to a need for an increased use of data science and analytics. Included in these priorities is a need to, “Develop integrated and scalable risk assessment and management approaches; develop integrated and proactive capabilities, technologies and methods to support secure and resilient infrastructure; [and] harness the power of data sciences to create unified, integrated situational awareness and to understand consequences of action.”

This approach maps to the tools used by the intelligence community (IC), which aggressively seeks new, competitive advantages in its global battle against terrorist groups. Using text analytics on the vast array of social media can provide a new weapon in the arsenal.

Military leaders are successfully turning to social media data, along with other information shared by coalition forces, to identify enemy locations and generate early warnings and real-time alerts to improve situational awareness. Social media is providing the IC with huge amounts of critical, relevant information, including specific events and occurrences, relationships between people and organizations and even recruitment tactics. The fast pace and sheer amount of chatter occurring on Twitter, blogs and news sites can be a wellspring of valuable information. But it’s a tremendous challenge.

Learn how to integrate law enforcement data to predict and prevent crime

Due to the size and speed of social media, it is impossible for intelligence analysts to manually decipher and make sense of it all. IC agencies need analytics to consume this information and put it into a framework that allows the analyst to determine what information is pertinent, unique and relevant to the specific mission. The analytics framework does the heavy lifting by providing insights in the context of the mission, filtering out spurious data so that an analyst is only reviewing the most relevant information.

Text analytics on social media data is only one part of the equation. It’s also important for IT analysts to integrate social media data with other, more traditional data sources.

For example, social media might reveal that people in a certain region are talking about a particular construction site. Even though these individuals live in different locations, the chatter indicates that a large structure is being built. This may be an early indication that something is on the drawing board. Analysts can then use specialized visualization techniques – or mapping technologies such as overhead imagery – to verify that the structure is, in fact, being built. This knowledge provides intelligence officials with an early indicator that something is happening and informs them of potential enemy activity.

Text analytics enables high-powered computing to complement human intelligence with a repeatable and highly effective approach that generates better intelligence. With the ongoing threats to national security, social media and text analytics are helping military and intelligence organizations monitor and interdict terrorist activity. With more data coming online every day, this aspect of intelligence will only continue to grow.



Get More Insights


Want more Insights from SAS? Subscribe to our Insights newsletter. Or check back often to get more insights on the topics you care about, including analytics, big data, data management, marketing, and risk & fraud.

Back to Top