Data engineering is shifting towards intelligent automation using AI agents to make data pipelines self-healing.
At 3M, Manjunatha G shared a practical path to integrate AI agents into data engineering workflows for adaptive and resilient pipelines.
The use of AI, particularly large language models with system prompts, enables dynamic detection of schema changes, volume tracking, and business logic adjustments without manual intervention.
Small code changes driven by AI logic can significantly improve operational efficiencies in data pipelines, making them less fragile and more aligned with business needs.