A global study commissioned by data science and analytics company Alteryx, Inc. and IDC claims approximately 54 million data workers around the world face common challenges associated with understanding the complexity, diversity and scale of their organizations’ data.
The Alteryx-commissioned study The State of Data Science and Analytics, uncovered inefficiencies, ineffectiveness and wasted time as many organizations turn to data as the lifeblood of their digital transformation. The report says there is a greater need for data literacy and streamlined processes as businesses undergo digital transformation.
Eighty percent of organizations now leverage data across multiple organizational processes, but despite increases in innovation, data workers still waste 44 percent of their time each week because they are unsuccessful in their activities. Data workers spend more than 40 percent of their time searching for and preparing data instead of gleaning insights and, on average, use four to seven different tools to perform data activities, adding to the complexity of the data and analytics process.
The term ‘data worker’ spans beyond the 54 million identified in this study, but the findings are indicative of the challenges specific to those engaging in significant data work in their day-to-day jobs.
Some key findings from the report include:
On average, data workers leverage more than six data sources, 40 million rows of data and seven different outputs along their analytic journey.
The top frustrations cited by data workers in the survey are indicative of root causes that are responsible for inefficiencies and ineffectiveness. For example, more than 30 percent of data workers say they spend too much time in data preparation, a task that can often be automated.
Eighty-eight percent of data workers, approximately 47 million people worldwide, use spreadsheets in their data activities. Spreadsheet functions are often used as a proxy for data preparation, analytics and data application development tools but are error-prone and expose the organization to compliance and trust issues.
IDC Director of Data Integration Stewart Bond says leaders need to consolidate platforms and increase collaboration across channels to prevent data silos.
“Data is at the core of digital transformation, but until organization leaders address these inefficiencies to improve effectiveness, their digital transformation initiatives can only get so far. Consolidating platforms and looking for tools that address the needs of any data worker, whether a trained data scientist or an analyst in the line of business, can help reduce the friction that many organizations experience on their path to becoming data-driven,” says Bond.
The survey found that data workers are unsuccessful for a variety of reasons, including lack of collaboration, knowledge gaps and resistance to change. Participants reported the lack of creative and analytic thinking, analytic and statistical skills, and data preparation skills as the highest-ranked skills gaps responsible for productivity issues, indicative of the pervasive talent gap that exists between data scientists and data workers in the line of business.
Alteryx Chief Data and Analytics Officer Alan Jacobson says to overcome these issues, many organizations are hiring Chief Data and Analytics Officers to streamline analytic processes, build a culture of analytics and encourage data literacy across the enterprise as part of their broader digital transformation strategy.
“Collecting data alone won’t digitally transform a business and the answer is not as easy as hiring a leader, a few data scientists or over-investing in disparate technologies. The key is to empower all users, many of whom are currently stuck in spreadsheets, to analyze data effectively to drive real, business-changing results. As the data landscape becomes more complex, this survey exposes the tip of the iceberg when it comes to the sheer volume of workers needing to conduct analysis on a daily basis and the untapped potential for them to drive meaningful business impact,” says Jacobson.
The State of Data Science and Analytics is based on a comprehensive survey of more than 800 individuals performing data functions across geographies, industries, company sizes and departments.
Alteryx says it offers an end-to-end analytics platform that empowers data analysts and scientists to break data barriers and deliver insights. To access the full IDC Infobrief, download here.