At the Gartner Data & Analytics Summit in Sydney, Director Analyst Peter Krensky has unveiled the top trends that are shaping the future of data science and machine learning (DSML). As the industry continues to grow and adapt to the increasing significance of data in artificial intelligence (AI), the focus is now shifting towards generative AI investments.
Peter Krensky emphasized that DSML is no longer solely centered around predictive models but is evolving into a more democratized, dynamic, and data-centric discipline, largely driven by the enthusiasm surrounding generative AI. He also acknowledged that while potential risks are emerging, there are many new capabilities and use cases for data scientists and organizations to explore.
Here are the top trends highlighted by Gartner:
Trend 1: Cloud Data Ecosystems Data ecosystems are rapidly moving from self-contained software or blended deployments to full cloud-native solutions. Gartner predicts that by 2024, 50 percent of new system deployments in the cloud will be based on cohesive cloud data ecosystems rather than manually integrated point solutions. Organizations are advised to assess data ecosystems based on their ability to resolve distributed data challenges and integrate with data sources beyond their immediate environment.
Trend 2: Edge AI The demand for Edge AI is on the rise as it enables real-time data processing at the point of creation, leading to immediate insights, detection of new patterns, and compliance with stringent data privacy requirements. Gartner forecasts that by 2025, over 55 percent of data analysis by deep neural networks will occur at the point of capture in an edge system, compared to less than 10 percent in 2021. Organizations are encouraged to identify applications and AI training required to leverage edge environments near IoT endpoints.
Trend 3: Responsible AI Responsible AI aims to ensure AI’s positive impact on society while addressing ethical considerations. It encompasses making the right business and ethical choices during AI adoption, including value, risk, trust, transparency, and accountability. Gartner predicts that by 2025, 1 percent of AI vendors will concentrate the majority of pretrained AI models, making responsible AI a societal concern. Organizations are advised to adopt a risk-proportional approach and seek assurances from vendors regarding risk management and compliance obligations.
Trend 4: Data-Centric AI Data-centric AI signifies a shift from model and code-centric approaches to a focus on data to build better AI systems. Solutions like AI-specific data management, synthetic data, and data labeling technologies aim to overcome data challenges, such as accessibility, volume, privacy, security, complexity, and scope. The use of generative AI to create synthetic data is rapidly growing, with Gartner predicting that by 2024, 60 percent of data for AI will be synthetic. This will enable the effective training of machine learning models by simulating reality and future scenarios, thereby derisking AI.
Trend 5: Accelerated AI Investment AI investment is set to accelerate further, with organizations and industries keen to grow through AI technologies and AI-based businesses. Gartner predicts that by the end of 2026, more than $10 billion will have been invested in AI startups relying on foundation models — large AI models trained on massive amounts of data.
A Gartner poll of over 2,500 executive leaders revealed that 45 percent increased their AI investments due to recent hype around ChatGPT, and 70 percent are currently exploring generative AI, with 19 percent in pilot or production mode.
The future of data science and machine learning looks promising as these trends pave the way for innovation, new applications, and responsible AI adoption across various industries. Organizations are urged to stay ahead by embracing these trends to maximize the potential of AI and data-driven decision-making.
