Big data requires a new breed of analyst
By Jay Liebowitz, author and Orkand Endowed Chair in Management and Technology
Everyone is talking about big data. Big data is a big deal, especially in data-intensive industries such as cybersecurity, finance, health care, marketing, transportation and energy. And many of us are already familiar with the three V's of big data – volume, velocity and variety. But the key question is: How do we extract big knowledge from big data?
The answer to this question is partly through analytics, which is a growing field within various sectors. Some people look at data analytics in terms of educating future "data scientists," while others are exploring business analytics through educating a new kind of "business analyst."
Regardless of what you call them, the new breed of analytics specialists need to have a combination of skills, including statistical techniques, applied mathematical methods, advanced machine learning algorithms, data visualization, and business and communication skills.
We often can find analysts who have the technical savvy, but they lack the business and communication skills to explain to managers and executives how results from analytics can inform organizational decision making.
The new breed of analysts must also possess the experience and instinct for knowing whether the analytics results have gone awry or whether they seem to have produced some valuable results. Recent MIT conferences on big data have talked about this value of intuition and making smarter decisions.
The Partnership for Public Service and IBM's October 2012 joint report titled From Data to Decisions II talks about the importance of tapping a mix of people with different backgrounds and strengths. The report highlights the importance of an analytics team approach that takes into account background and experience. This approach should lead to a credible set of analytics results that focus on the goals and objectives for that enterprise. Having a multidisciplinary perspective should provide great value to the analytics team.
New academic programs for analytics
Universities and colleges are offering new degrees and programs in analytics. The Master of Science in Analytics program at North Carolina State University has been a model for most of these analytics programs and has recently doubled in size due to its popularity. Other programs include:
- The University of Maryland University College – online master's in analytics (plans are underway to start this summer).
- Stevens Institute of Technology – master's in business intelligence and analytics.
- New York University – master's in business analytics.
- Ohio State University – advanced analytics center, formed with IBM in November 2012.
'Instinct' a factor
It has already been shown that business analytics can inform decision making (see the 2011 study conducted by Bloomberg and SAS). But how can we ensure that the next generation of analysts is prepared to extract the "big knowledge" from the "big data"? Certainly, having some background in knowledge discovery techniques as part of the analytics team may provide some helpful way to look for hidden patterns and relationships in the data and text. But how do we know that these relationships are worthy of further analysis and exploration? How does "instinct" factor into the results?
It's a Catch-22: You want analysts with technical and organizational experience, but new analysts who have the most up-to-date technical training often haven't had enough experience to best understand the organizational setting. Thus, the team approach with a mix of background and experience will help address this concern in terms of building "instinct" into the analysis.
Other ways to instill instinct into organizational decision making through analytics are:
- Rotate analysts' assignments within the organization so that they can better understand its functional components.
- Assign senior-junior analyst mentoring so that the younger analysts can learn from the experiences of others.
- Don't take an answer for granted – bounce the analytics results off of others in the organization to check the validity and reasonableness of the results.
Five tips for finding – and keeping – great analysts
- Don’t rely on human resources or search firms to find top analytical talent. Rely on your internal network of analysts and look to resource groups where analysts gather.
- Create a defined job path to keep your talent. If people can't rise above the job title of analyst or statistician, they will take their talents elsewhere.
- Hire communicators. You don't want to hire the rocket scientist who can't explain what he or she is doing to the layperson.
- Build relationships with universities. Good analytics candidates come from many disciplines, including econometrics, statistics, finance, marketing and agronomy.
- Coach, coach and coach some more. Since it is tough to find the "perfect" analyst (even if you control the search), training is important. Many strong technical individuals with the potential to be great communicators are a bit introverted and need help learning to be assertive.
Get a full explanation of each point in this blog post.