Senior Data Engineer

About the Role

As a Senior Data Engineer joining our new Marine Insurance initiative, you will play a central role in shaping the systems that power Kpler’s global market intelligence platform. You will lead the end-to-end design and delivery of complex data pipelines and cloud infrastructure, ingesting, processing, and serving large-scale data to our global customers. Acting as a technical leader, you will set data architecture directions and mentor peer engineers, ensuring our platform reliably scales as our datasets and product offerings expand.

Key Responsibilities

Design and own end-to-end data pipelines: Build, deploy, and operate production-grade data pipelines and infrastructure from initial architecture through deployment, monitoring, and maintenance.

Deploy containerized AWS services: Develop and containerize applications within AWS using Python, Docker, Kubernetes, Airflow, and Infrastructure as Code (Terraform).

Ensure data integrity and event streaming: Architect robust PostgreSQL database structures and event-streaming flows (Kafka/Flink) to guarantee seamless data flow and integrity.

Set technical direction and standards: Lead complex pipeline design, peer reviews, and logging, monitoring, and alerting frameworks to maintain pipeline resilience.

Partner cross-functionally and mentor peers: Collaborate closely with engineers, product managers, and data scientists while mentoring engineers to elevate team craft and technical development.

Evaluate emerging data tools: Research and implement modern data infrastructure improvements and frameworks to continuously elevate Kpler's platform capabilities.

Experience & Background

What you'll need (Must-haves)

Data engineering experience at scale: 5+ years of experience designing, operating, and owning production-grade data pipelines and infrastructure.

Core tech stack depth: Strong Python skills with hands-on experience in Docker, Kubernetes, AWS, Airflow, and relational databases like PostgreSQL.

Streaming, IaC, and observability: Practical experience with event streaming (Kafka/Flink), Infrastructure as Code (Terraform), CI/CD practices, and production logging/monitoring systems.

Technical leadership & AI coding tools: Demonstrated ability to lead architectural discussions, mentor engineers, and leverage AI-assisted coding tools (Copilot, Cursor, Claude) with strong technical judgment.

Methodologies & communication: Experience working in Agile environments with strong collaboration skills and fluency in written and spoken English.

Nice-to-haves

Large-scale analytics platforms: Experience building large-scale data processing systems or analytics platforms.

Cross-team initiatives: Track record of leading technical initiatives across multiple teams.

Domain knowledge: Familiarity with commodities, shipping, energy, or marine insurance market data.