Senior ML/AI Engineer - A26409

We believe in positively impacting the lives of people around us and the environment we live in through the use of technology. Hence, we are committed to providing a conducive environment for all employees to realise their full potential, who in turn have the opportunity to continuously drive innovation.

Co-Development Business Unit is hiring for Senior ML/AI Engineer

You will work in Singapore Government Agencies

This is a fixed term contract role. The engagement is 2 years.

Internal Code: A26409

What will you do?

The ML/AI Engineer sits within the DSAD Platform and Architecture Team and is responsible for productionising, operating and improving machine-learning and AI workloads on the SWDA data platform using AWS and Databricks. The role combines ML/AI engineering with practical data engineering, working closely with data scientists, data engineers, platform engineers, product teams and governance stakeholders to build reliable data and feature pipelines, curate production-ready datasets, and establish reusable MLOps/AIOps capabilities that support secure, observable and cost-effective AI services from data ingestion and experimentation through deployment, monitoring, retraining and retirement.

MLOps/AIOps Platform Engineering: Design and implement reusable platform components and golden paths for experiment tracking, model registry, pipeline orchestration, deployment, monitoring, feedback and retraining using Databricks, MLflow and approved AWS services.

• Production Engineering: Convert notebooks and prototypes into modular, tested, containerised and maintainable batch jobs, APIs, model-serving endpoints or agentic services, with clear interfaces and operational ownership.
• Feature Engineering Pipeline: Design, build and maintain reliable feature engineering pipelines on Databricks and AWS for ingesting, transforming, validating and serving structured and unstructured data used by ML/AI workloads.
• Performance and Cost Engineering: Optimise Spark and SQL jobs, file layout, partitioning, cluster configuration and workload scheduling to improve reliability, runtime and AWS/Databricks cost efficiency.
• CI/CD/CT Automation: Build automated workflows for testing and promoting code, data, features, prompts, models and infrastructure across environments, including quality, security, evaluation and human approval gates.
• Reproducibility and Governance: Implement source, data, model, prompt and environment versioning; capture lineage, evaluation evidence, approvals and deployment history; and provide rollback and retirement mechanisms.
• Monitoring and Reliability: Implement dashboards and alerts for system health, latency, throughput, errors, resource utilisation, cost, data quality, drift and model performance; maintain runbooks and support incident response and root-cause analysis.
• Model Evaluation and Experimentation: Develop repeatable offline and online evaluation, regression testing, champion-challenger comparison and A/B testing patterns, with agreed release and rollback criteria.
• Security and Responsible AI: Apply least-privilege IAM, encryption, secrets management, network and environment controls, vulnerability scanning, audit logging, sensitive-data controls and safeguards for untrusted AI inputs and tool use.
• Recommendation and Search Support: Support model maintenance and retraining, package and dependency updates, new data ingestion, experimentation, filters, logic changes and performance reporting for SWDA recommendation and search workloads.
• Optimisation and Continuous Improvement: Improve scalability, resilience, deployment frequency and cost efficiency; reduce manual steps and duplicated patterns; and contribute to the team’s MLOps maturity roadmap.
• Documentation and Capability Transfer: Produce architecture decisions, model cards, release evidence, operational runbooks and user guidance, and coach DSAD teams so supported solutions can be operated sustainably.

Requirements

What are we looking for?

• Strong Python and software-engineering capability, including APIs, packaging, automated testing, Git, code review, debugging and modular design.
• Hands-on experience with Databricks and MLflow for experiment tracking, model registry, workflow orchestration or model serving.
• Strong hands-on experience with Spark/PySpark and SQL, including distributed data processing, query optimisation, partitioning, joins, incremental processing and troubleshooting data pipelines.
• Experience designing and implementing lakehouse or data-lake pipelines using Databricks, Delta Lake and AWS S3, including Bronze, Silver and Gold data-layer patterns.
• Experience with data modelling, schema design, data contracts, metadata, lineage, data-quality validation and reconciliation for production analytics and ML datasets.
• Experience orchestrating batch or streaming workflows using Databricks Workflows, AWS Glue, Step Functions, EventBridge, Apache Airflow or an equivalent enterprise orchestration tool.
• Hands-on AWS data and cloud engineering experience with a relevant mix of S3, Glue, Lake Formation, IAM, KMS, ECR, ECS/EKS, Lambda, Step Functions, EventBridge, CloudWatch, API Gateway, Secrets Manager, Redshift, MSK/Kinesis or equivalent approved services.
• Experience with Docker, infrastructure as code such as Terraform, and CI/CD platforms such as GitLab CI, GitHub Actions, Jenkins or Harness.
• Understanding of ML lifecycle controls covering data validation, feature engineering, training, evaluation, deployment, monitoring, drift, retraining and retirement.
• Ability to implement reproducibility and traceability across code, datasets, features, parameters, environments, prompts and model artefacts.
• Experience implementing system and model observability, alerts, runbooks, incident diagnosis and rollback or recovery procedures.
• Working knowledge of cloud security, least-privilege access, encryption, secrets management, vulnerability management and audit logging.
• Experience with Databricks Unity Catalog, Delta Live Tables or Lakeflow, feature engineering/feature stores, change data capture, event streaming using MSK/Kinesis/Kafka, and cross-account or governed data-sharing patterns.
• Experience productionising generative-AI or agentic systems, including orchestration, state, prompt/version management, evaluation, controlled tool use, guardrails and human approval.
• Experience with Kubernetes, OpenTelemetry, Prometheus, Grafana, model drift tooling, A/B testing or champion-challenger deployment.
• Experience supporting high-volume, low-latency or public-facing AI services and optimising cloud inference and training costs.
• Experience in Singapore public-sector environments, including VAPT, IM8-aligned controls, data classification, system assurance or responsible-AI governance.
• Strong engineering judgement and a pragmatic mindset, with the ability to distinguish a supportable production service from a demonstration.
• Structured problem-solving across data, model, application and infrastructure layers, including calm and evidence-based incident handling.
• Ability to work across data engineering and ML engineering boundaries, translating model requirements into dependable datasets, features, interfaces and service-level expectations.
• Clear written and verbal communication, including the ability to explain architecture, evaluation evidence, risk, cost and operational trade-offs to technical and management audiences.
• Collaborative working style with data scientists, data engineers, platform engineers, product teams, cybersecurity and governance stakeholders.
• Ability to work independently, document decisions, manage ambiguity and deliver through incremental, testable outcomes.
• Commitment to knowledge transfer, reusable engineering patterns and sustainable in-house capability rather than bespoke solutions.

Reference Practices

This role is aligned to the Singapore Government MLOps Playbook’s emphasis on secure infrastructure, automated delivery, testing, monitoring, feedback, retraining and progressive MLOps maturity. It also reflects agentic-AI engineering practices around orchestration, state, evaluation, observability, controlled tool use and human approval.

References: MLOps Playbook — https://playbooks.aip.gov.sg/mlops-playbook/ | “Harnessing the harness” — https://medium.com/dsaid-govtech/harnessing-the-harness-983e6eaa605c

Benefits

What do we offer in return?

Competitive Compensation: Market competitive salary and variable performance bonus aligned to your skills, impact, and contribution.