Talend Senior Vice President & GM Asia Pacific Stu Garrow argues that technology can help data engineers stay relevant in the Age of AI.
By Stu Garrow
As the next big transformative technologies to take the tech market by storm, it is no surprise that industries are worried about artificial intelligence (AI) and machine learning (ML) taking over the workplace.
By some estimations, AI could replace up to 40% of jobs in the next 15 years. IDC forecasts an explosion of spending on AI systems in the Asia-Pacific region, reporting an increase of almost 80 percent this year compared to 2018 and growth at a compound annual growth rate (CAGR) of 50 percent between 2018 and 2022. Many who support AI and automation, believe that this technology could fundamentally change the workplace.
Paranoia about being displaced by machines is growing, and based on the numbers, it may be warranted. But the key to staying relevant in the workplace is to use technological advances in your favour. If given the right tools, employees can utilise technology to do the grunt work for them so they can add human value — something AI can’t do.
Although Singapore is considered to be lagging behind Indonesia and Thailand in terms of AI adoption, the public and private sectors are working to improve the country’s adoption rates. AI Singapore (AISG) – a national initiative driven by an intra-governmental partnership that includes the National Research Foundation, Smart Nation and Digital Government Office (SNDGO), Economic Development Board (EDB), Infocomm Media Development Authority (IMDA), SGINnovate and the Integrated Health Information Systems (IHiS) – is pushing forward with efforts to enhance Singapore’s capabilities.
Despite this strong commitment to pushing AI adoption, AISG’s director notes that the ongoing uncertainty around where and how to start with the technology is a challenge that the organisation will need to overcome with education.
The Big Opportunity
AI is a massive industry, with potential in every field from autonomous cars to human resources. It is estimated by PwC that AI could add up to USD15.7 trillion to the global economy by 2030. Many of those who believe AI will impact jobs also believe that AI can create new opportunities or enhance existing professions.
One strategy to ensure job security in the midst of this changing landscape is to choose a job that is in high demand. However, a recent study of adult Singaporeans found that science, technology, engineering and mathematics (STEM) careers are losing out to business fields as only between nine and 18 percent of respondents are interested in pursuing these fields compared to 30 percent who were interested in business. KPMG notes that the skills shortage in technology is at its highest level since 2008.
For example, as AI and machine learning rely heavily on data to operate, the need for data engineers has never been more evident than it is today. The role of the data engineer is to help the company make the best use of data to accomplish business objectives, especially as it relates to AI. Data engineers can prepare companies for technological innovation and automate projects enabling companies to bring more data-driven projects into production. Essentially serving as the experts in designing, building, and maintaining the data-based systems in support of an organization’s analytical and transactional operations. Today, data engineers need to be more productive than ever before while working in a constantly-changing data environment.
When thinking about how AI could actually improve the workplace without displacing humans, Bayer Digital Farming comes to mind. Using Talend for its data lake, Bayer Digital Farming developed a WEEDSCOUT App for Farmers. The app uses machine learning and artificial intelligence to match photos of weeds in a Bayer database with weed photos farmers send in. Accessible all over the world, the photo database resides on a private cloud stored on Amazon Web Services. It gives the grower the opportunity to more precisely predict the impact of his or her actions such as choice of seed variety, application rate of crop protection products, or harvest timing.
Through this use of AI, not only does the farmer benefit, but data engineers at Bayer can use and store this information for future recall and analysis that would not previously have been available. Adding value to their jobs as well as others in the organization.
It’s safe to say that AI will dramatically reshape the role of the data engineer and force them to acquire new skills in order to stay relevant. Companies that can utilise the existing data engineers and pair their skills with the right software will come out ahead.
Adapting most successfully to the coming era will allow data engineers to enjoy an abundance of work opportunities, but the process will require a different mindset than many software developers have today.