Freelance Senior AI Platform Expert - Azure Databricks

Role

We are looking for an experienced freelancer to advise and support our internal platform team in designing, building and improving an AI platform on Azure Databricks. The role is internal only, with no client-facing responsibilities.

The freelancer will act as a senior technical reference for the team, helping shape the architecture, establish engineering standards, review technical implementations and support the team in building and operating a reliable, scalable and secure platform. The ideal candidate combines strong hands-on technical expertise with the ability to mentor engineers and transfer knowledge.

The freelancer will support the team across the design, implementation and continuous improvement of the platform, including:

• Defining and reviewing the target architecture on Azure Databricks, covering workspaces, environments (dev/test/prod), Unity Catalog, security, networking and integration with Azure services.
• Advising on CI/CD/CT pipelines for code, data and models, including automated testing, promotion across environments, model registry workflows and continuous training triggered by schedules, drift or new data.
• Reviewing architecture decisions, code, Terraform configurations and pipeline implementations, providing actionable recommendations and identifying areas for improvement.
• Defining reusable MLOps standards, templates and engineering best practices.
• Supporting platform reliability, scalability, governance, security and operational excellence.
• Contributing hands-on when needed to unblock the team, troubleshoot technical challenges or develop reference implementations.
• Coaching team members, facilitating design and code reviews, and transferring knowledge to help the team become increasingly autonomous.

Requirements

• Extensive experience in data engineering, ML engineering, platform engineering or MLOps, ideally with at least 8 years of relevant experience.
• Strong practical experience with Azure Databricks in production environments, ideally at least 3 years.
• Solid knowledge of MLOps and the Databricks ecosystem, including MLflow, model registry, Model Serving, Feature Engineering, Databricks Workflows and monitoring.
• Experience designing and implementing CI/CD/CT pipelines, ideally using GitLab, and infrastructure-as-code practices.
• Hands-on experience with Terraform and cloud infrastructure, ideally using the Databricks and Azure providers.
• Solid knowledge of Azure services, including ADLS Gen2, Key Vault, Entra ID and cloud networking (VNet injection, private endpoints).
• Strong Python and PySpark skills, alongside software engineering best practices such as testing, packaging and code quality.
• Experience with Unity Catalog, including data governance, lineage and access control.
• Proven ability to mentor engineers, conduct design and code reviews, and establish technical standards.
• Experience with GenAI/LLMOps on Databricks, including technologies such as Vector Search, Mosaic AI Agent Framework and AI Gateway, is a strong plus.
• Experience implementing secure, private-network cloud environments and working with sensitive data is highly valued.