Lead Perception Engineer

About the Role

You will work on building and optimizing 3D perception systems that enable robots to understand and interact with complex real-world environments. The goal is to develop robust perception pipelines that work reliably across both simulation and real-world construction sites, ensuring accurate scene understanding, localization, and decision-making.

Key Responsibilities

3D Perception Development & Scene Understanding

• Design, implement, and deploy realtime 3D perception pipelines leveraging LiDAR, IMU, Stereo, and RGB cameras.
• Develop algorithms for spatial and temporal data interpretation to enable high-fidelity semantic scene understanding.
• Optimize ego-motion estimation and localization modules to ensure seamless integration with downstream planning and control tasks.

Deep Learning

• Train and integrate deep learning models required for semantic world understanding and surface finish classification for quality control
• Collect and curate high-quality datasets (real and synthetic) and automate training pipelines and experiment tracking.
• Benchmark and optimize perception models for edge devices to ensure real-time performance in resource-constrained environments

Sensor Fusion & Localization

• Design and implement sensor fusion strategies combining visual, inertial, and spatial data
• Architect and implement sophisticated sensor fusion strategies that integrate classical methods with deep learning approaches to convert noisy, asynchronous data from heterogeneous sensors into a unified environment representation.

Calibration

• Develop and automate robust extrinsic and intrinsic calibration procedures (Camera, LiDAR, IMU) using target-based and/or targetless methods.
• Design and implement algorithms for online calibration drift detection and self-healing to maintain system integrity during long-term deployments.
• Establish rigorous quantitative metrics to objectively evaluate and certify calibration quality.

Collaboration

• Partner with Navigation, Manipulation, and Cloud teams to ensure perception outputs are optimized for downstream path planning, grasping, and remote fleet monitoring.
• Define, track system-level and own system level KPIs and perception metrics to identify regressions across software iterations.

Requirements

• 5+ YoE
• Strong fundamentals in computer vision and 3D perception
• Proficiency in C++ (Python is a plus)
• Familiarity with PyTorch, TensorRT
• Basic understanding of Localisation and SLAM
• Ability to work with real-world data and debug complex systems

Nice to Have

• Experience with Nvidia Deepstream, GStreamer, Holoscan
• Experience with ROS/ROS2
• Hands-on experience with LiDAR, RGB-D cameras, or IMUs
• Familiarity with OpenCV, PCL, or similar libraries
• Experience working on robotics or vision-based projects