Position Overview
The Data & AI Engineer role is responsible for building scalable data pipelines, developing AI agents, and delivering end-to-end data and AI applications on the enterprise data platform, with a primary focus on Databricks.
This role bridges data engineering, AI engineering, and data science to deliver production-grade solutions that power analytics, automation, and intelligent applications across Suntory Beverage & Food’s global operations.
As part of the GHQ AAA team, you are expected to contribute to the organisation’s enterprise AI architecture by delivering reusable, scalable, and high-performance data and AI solutions.
Key Responsibilities
Data Pipeline Development
- Design, develop, and maintain scalable data pipelines using Databricks (Delta Lake, PySpark, workflows).
- Build robust ETL/ELT processes to ingest, transform, and serve data from multiple enterprise sources (SAP, external data, APIs).
- Ensure data quality, reliability, and performance optimisation across pipelines.
- Implement data models aligned with analytics and AI use cases (e.g., feature-ready datasets).
- Collaborate with data governance teams to ensure compliance with enterprise data standards.
AI Agent Development
- Design and develop AI agents and GenAI solutions (e.g., knowledge assistants, automation agents) using Databricks and Azure AI capabilities.
- Implement Retrieval-Augmented Generation (RAG), prompt engineering, and orchestration logic for enterprise AI use cases.
- Integrate agents with enterprise data sources, vector databases, and APIs.
- Collaborate with business teams to identify and deliver AI-driven automation and productivity use cases.
- Ensure scalability, performance, and responsible AI practices in agent deployment.
Databricks App Development
- Develop end-to-end data and AI applications using Databricks (e.g., notebooks, dashboards, apps, APIs).
- Build interactive analytics or AI-driven applications for business users (e.g., demand planning tools, decision support apps).
- Expose data and AI services via APIs (e.g., Databricks Model Serving, API integration).
- Work with front-end or BI teams (e.g., Power BI, apps) to integrate backend logic into user-facing solutions.
- Optimise application performance and ensure scalability for enterprise usage.
Data Science Development
- Support development of machine learning models (forecasting, optimisation, classification).
- Perform feature engineering and data preparation for modelling.
- Collaborate with Data Scientists to productionise models within Databricks environment.
- Integrate trained models into pipelines and applications for real-time or batch inference.
- Ensure alignment between modelling outputs and business requirements.
Experiences & Skills Requirements
- Bachelor’s or Master’s degree in Computer Science, Data Engineering, Data Science, or related field.
- 1–3 years of experience in data engineering, AI engineering, or related roles.
- Strong hands-on experience with Databricks (Delta Lake, PySpark, workflows, model serving).
- Proficiency in Python and SQL for data processing and application development.
- Experience building data pipelines and ETL processes in large-scale environments.
- Practical experience in AI/GenAI development (LLMs, RAG, prompt engineering).
- Familiarity with API development and integration for data and AI services.
- Understanding of machine learning concepts and model deployment workflows.
- Experience with MLOps / DataOps practices (CI/CD, pipeline automation, monitoring) is preferred.
- Knowledge of Azure ecosystem (ADLS, ADF, Azure AI, API Management) is a plus.
- Strong problem-solving skills and ability to work across cross-functional teams.
- Experience in FMCG, supply chain, or commercial analytics is an advantage.