Big Data Analytical Advances From Academia, Business Are Enhancing Exploration Of Our Universe
Statisticians have combined state-of-the-art analytical techniques from the academic and business worlds to tackle the big data challenges confronting astrophysicists and astronomers as they explore the mysteries of our universe, Lars K.S. Daldorff and Siavoush Mohammadi today told a session at the 2015 Joint Statistical Meetings (JSM 2015) in Seattle.
These technical advances – called automatic explorative analysis of data – have the potential to greatly aid scientists seeking to understand our universe. Researchers using big data will also benefit from these advances, said Daldorff and Mohammadi while presenting a contributed paper titled Novel Application of Statistical Tools for Big Data Analyses of Solar Physics at JSM 2015.
Daldorff is an atmospheric, oceanic and space sciences research fellow in the College of Engineering at the University of Michigan and a consultant for NASA’s Goddard Space Flight Center. Mohammadi is a consultant with Infotrek, a Swedish business intelligence and data warehousing company.
The new analytical techniques Daldorff and Mohammadi described are being used in a study of giant magnetic loops generated by our solar system’s sun. When physicists use large supercomputers to simulate the sun, their research produces massive amounts of data. But the phenomenon of interest is usually located at a specific point in time and space, essentially creating a proverbial needle-in-a-haystack situation for the researcher.
The large quantity of data has forced physicists to reduce data amounts, which they do by looking at small portions of the data at the time, making the process long and slow before true insight is found.
But what if you could scan the entire haystack at once to find the proverbial needle? That’s the question Daldorff and Mohammadi sought to answer when they looked to commercial analytics solutions to explore, categorize and display the large amount of solar research project data from the plasma simulations Daldorff had conducted for NASA.
There are still many open questions surrounding solar magnetic loops associated with solar spots, which contribute to a considerable increase in X-ray and ultraviolent radiation from the outer solar atmosphere and into the upper atmosphere of the Earth. The phenomena can be seen in this video clip released by the NASA Heliophysics Science Division as part of the Solar Dynamics Observatory project.
The astrophysicist community speculates that when these powerful arches are created, a phenomenon called magnetic reconnection occurs. It’s this moment in the data that researchers like Daldorff and Mohammadi want to identify both spatially and in time – or both where and when.
The duo uses statistical methods that frequently are used in data warehouses and by analysts at companies to study human behavior (e.g., customers) or scientific data, in this case coronal loops. These are analytical methods that combine computational power and statistics to turn information into insight. These standardized methods, widely used in the business world, suddenly find use for a completely different type of data.
As for analytical tools, they have been using SAS® Visual Analytics – a big data discovery, interactive exploration and reporting tool that works in-memory. Many of the analytical methods employed by SAS Visual Analytics are standardized methods and used for data analytics in numerous industries. The methods for identifying points of interest, finding relevant data relationships, performing analysis, visualizations and creating reports are the same, regardless if it’s used on business or scientific data, Daldorff and Mohammadi told session attendees.
This automatic exploration of large data sets using statistics and modern analytical methods can greatly reduce the time it takes to extract insight from big data – not just for heliophysics research, but all data-intensive research subjects. It removes a major, manual repetitive step and automates it so that subject experts can focus on the research topic instead of processing data manually.
“Our hope is these results can help with solar magnetic loops research at NASA and at the same time, our work will show the effectiveness of explorative analysis of data in other data-intensive fields. There are numerous possibilities for this new application that could potentially help various types of researchers – in academia, business and science – obtain quicker insights and results from their research’s big data,” said the duo.
JSM 2015 is being held Aug. 8-13 at the Washington Convention Center in Seattle. More than 6,000 statisticians – representing academia, business and industry, as well as national, state and local governments – from numerous countries are attending North America’s largest statistical science gathering.
About JSM 2015
JSM, which has been held annually since 1974, is being conducted jointly this year by the American Statistical Association, International Biometric Society (ENAR and WNAR), Institute of Mathematical Statistics, Statistical Society of Canada, International Chinese Statistical Association, International Indian Statistical Association, Korean International Statistical Society, International Society for Bayesian Analysis, Royal Statistical Society and International Statistical Institute. JSM activities include oral presentations, panel sessions, poster presentations, continuing professional development courses, an exhibit hall, a career-placement service, society and section business meetings, committee meetings, social activities and networking opportunities. Click here for more information about JSM 2015.
About the American Statistical Association
The ASA is the world’s largest community of statisticians and the second-oldest continuously operating professional society in the United States. Its members serve in industry, government and academia in more than 90 countries, advancing research and promoting sound statistical practice to inform public policy and improve human welfare. For additional information, please visit the ASA website at amstat.org.
SAS is the leader in analytics. Through innovative analytics, business intelligence and data management software and services, SAS helps customers at more than 83,000 sites make better decisions faster. Since 1976, SAS has been giving customers around the world THE POWER TO KNOW®.