Senior ML Infrastructure Engineer

Join us at EIT:

At the Ellison Institute of Technology (EIT), we’re on a mission to translate scientific discovery into real world impact. We bring together visionary scientists, technologists, policy makers, and entrepreneurs to tackle humanity’s greatest challenges in four transformative areas:

This is ambitious work - work that demands curiosity, courage, and a relentless drive to make a difference. At EIT, you’ll join a community built on excellence, innovation, tenacity, trust, and collaboration, where bold ideas become real-world breakthroughs. Together, we push boundaries, embrace complexity, and create solutions to scale ideas for lab to society. Explore more at www.eit.org

Your Role:

Join our SciComp team to build the cloud and compute foundation that enables scientific breakthroughs. Deliver reliable, secure platforms and self-service guardrails that accelerate experimentation and turn ideas into results - faster, at scale, and with confidence.

Your Responsibilities:

• Build, operate, and continuously optimise our high-performance GPU training and inference clusters, focusing on robust, high-availability scheduling, isolation, and automated lifecycle management.
• Drive systems design and implementation for high-throughput data paths, optimising I/O, caching, and data locality across compute and storage (including our current Lustre implementation).
• Proactively benchmark, profile, and resolve performance bottlenecks across the compute, network, and orchestration layers to maximise efficiency for distributed training and inference.
• Establish comprehensive observability, resilience, and automated security controls to ensure compliance and robust operation of sensitive research environments.
• Partner with Research, Data, and Applied teams to forecast capacity and cost for GPU and storage needs, setting quotas and streamlining ML experimentation pipelines.

Requirements

Essential Skills, Qualifications & Experience:

• Proven experience leading the design, build, and operation of high-performance ML compute clusters at scale
• A proactive, autonomous approach to systems design and the proven ability and desire to ideate, co-create and implement optimal solutions
• Exposure to migrating or transforming ML infrastructure from traditional schedulers to modern, containerised systems
• Expertise with high-throughput storage systems for ML/HPC workloads
• Expert-level understanding of GPU architecture, high-speed networking for distributed training, and performance profiling to resolve bottlenecks
• A solid grasp of IaC and CI/CD practices (e.g., Terraform, Argo CD)

Benefits

We offer the following salary and benefits:

• Competitive salary (dependent on experience) + travel allowance + bonus
• Enhanced holiday. Our annual leave allowance is 25 days plus 8 bank holidays and an additional 3 days between Christmas and New Year. You will also have the opportunity to purchase an additional 5 days annual leave in January and July.
• Pension - Employer contribution 7.5%, minimum employee contribution 5%
• Life Assurance.
• Income Protection
• Private Medical Insurance as standard for you, your partner and any dependents. Including hospital Cash Plan
• Employee discounts
• Electric car scheme
• Nursery Salary Sacrifice scheme
• Cycle to Work Scheme
• Family Planning
• Neurodiversity support including advise and assessments
• Coaching & Therapy services

Working together - what it involves:

You must have the right to work permanently in the UK with a willingness to travel as necessary. In certain cases, we can consider sponsorship, and this will be assessed on a case-by-case basis.

You will live in, or within easy commuting distance of, Oxford/London (or be willing to relocate)

The SciComp team work to a hybrid working pattern of 3 days in the office, 2x in our Oxford Office and 1x in our London office. You must be able to commit to this should you be successfully appointed.