In any crisis, communities need food, water and medicine to survive. At the United Nations Global Pulse (UNGP), in addition to the aforementioned essentials, we also view information as aid; information can save lives, livelihoods and resources. Our work is focused on understanding how raw, unscripted comments and trends found in social media and other digital data sources could help identify regions that may be in trouble financially, economically or socially. The ultimate goal is to help the public sector act more quickly on those indicators that show negative changes in human well-being. I like that my job helps address the world’s most wicked problems – with data.
My SAS colleague blogged about our work, and I want to give you even more insight into the changes that can result from tracking this new kind of data. For background, the UN Global Pulse initiative began in 2009 as a reaction to the global financial crisis; decision-makers needed more up-to-date information on how communities were faring and being impacted by the economic crisis than was available with traditional statistics. Global Pulse functions as an R&D lab, currently monitoring social media to detect trends in four cities in Indonesia – a developing country that has just crossed the line to being more than 50 percent urban. More and more people are moving to the urban environments and, they are buying cell phones and talking on Twitter. It is a prime area to investigate, and we are hoping that our work there will be a prototype for ideas that can applied globally.
But enough set-up – what do we actually want to accomplish? Why are we exploring social media?
These days, people tell social networks things they wouldn’t even tell their doctors. By analyzing volumes of comments on social media, we can, for example, predict a spike in unemployment in a given geography. Granted, this data reflects some very complex phenomena and human behaviors that have social, political and environmental dimensions – so I must note that causality is elusive. However, think about these examples of social chatter and how the information contained in them could aid the society at large.
|Societal Category||Trends found in social media data||Decision –> Improved policy|
|EMPLOYMENT||“I’m a business major and haven’t been able to find a job. I may need to move to Jakarta to find one.”||Public sector organizations focused on skills training and job placement in Jakarta begin offering more services.|
|PROGRAMS FOR THE POOR||“I stood in line for 2 hours trying to get rice today and then found out I was unqualified for the government subsidy program.”||Government may need to revisit qualification criteria and/or take other measures to help people in need get available assistance.|
|HEALTH EPIDEMIC||“Seems like everyone is sick with the flu.”||Health organizations can get a head start on anticipating flu season and preparing vaccines.|
|EFFECTIVENESS OF PROGRAMMES AND SERVICES & PROGRESS RELATED TO MILLENIUM DEVELOPMENT GOALS (MDG)||“Where is the doctor at this 24/7 clinic? It’s 3pm on Thurs.”||The UN and other agencies get an idea of how well certain programmes are working; they can target resources to specific areas to increase probability of meeting a goal.|
These are just a few ways this kind of data can affect critical decisions if properly analyzed. However, let’s not discount the value of traditional statistics, which are precise and thorough. The unstructured data found in social networks is new; it’s complimentary. There’s a tradeoff between accuracy and speed. Unstructured data is less accurate but faster, sometimes in orders of magnitude faster. But the question remains: How can you use it?
How do you effectively incorporate those fast insights into decision making? You have to determine whether a comment or set of comments is an anomaly or a trend. What is the outcome you’re trying to achieve?
Consider the weather app you inevitably use daily on your mobile device or PC. You can tell whether it’s sunny or snowing in any given geography. Of course, the accuracy of the forecast for tomorrow versus three days from now will different greatly. This data is fast but it’s less precise. The outcome continues to change due to the speed of data.
Weather apps rely on sensor data. Social chatter is also sensor data. The Global Pulse initiative will ideally help public-sector decision makers get a sense of the current economic climate anywhere in the world. If they monitor signs of stress, they can then dig deeper into issues, verify them and take action – using the information as aid for people in trouble.
What other decisions could be made if governments had this insight? I look forward to your comments. Meantime, watch for another blog post on how Global Pulse is growing its database beyond social media.