Machine Learning Engineer
- Salary
- $220K - $300K
- Level
- Mid
Not stated in the posting: work type, visa.
Open. First seen 26 October 2026.
About the role
About the company
Our client is a technology company.
The role
Raydar is recruiting for this role on behalf of our client. Design, build and run evaluation systems that measure how machine learning models perform as they change over time. Work across defining metrics, building data pipelines and automated judging, and simulating test environments, then turn the findings into concrete model improvements.
What you'll do
- Define evaluation metrics and success criteria for model behavior that shifts over time, and keep them current as methods evolve.
- Build evaluation pipelines covering data handling, labeling, versioning, reruns and automated judging.
- Read model traces and results to find real failure points and propose fixes.
- Create simulated users and environments to test models at scale.
- Own evaluation work from the initial question through pipeline, experiments and written findings.
Requirements
What we're looking for
- Experience designing, building and running complex evaluations or simulators for reinforcement learning post-training, such as LLM-as-judge setups, RL environments or simulations.
- Peer-reviewed research at leading machine learning venues, or independently released open-source projects of similar caliber.
- Production-quality Python, including testing, maintenance and a regular release cadence.
- Working knowledge of PyTorch.
Bonus points
- Hands-on time creating artificial test agents or mock user populations for assessing or tuning language models.
- Advanced study in a numerate discipline such as computing or applied maths is a plus.
- Track record of contributing to open-source projects in the machine learning research community.
- Experience with personalization, memory or identity modeling for language models.
- Experience with data pipeline tooling and SQL.
Benefits
Compensation and benefits
- Salary: $220K - $300K
- Equity
- Health, dental and vision coverage
- Retirement plan
- Flexible time off
Location and work model
- New York, NY, United States
- On-site, 5 days in-office per week
- Full-time
What this role asks for
Particular to this role
SQL
Rare in Raydar's other 299 open postings.
Standard for data science / ML roles
LLM · Machine Learning · PyTorch · Python
Common in data science / ML postings at any company.
Compared with Raydar's other open jobs.
Other openings for this role
3 open postings for this role in New York, first posted 23 October 2026. The company is likely hiring for more than one position. None has been taken down and posted again.
- 26 October 2026 – open (this posting)
- 23 October 2026 – open
- 23 October 2026 – open
Salary context
75 other open postings titled Machine Learning Engineer state a salary: median USD 151,500 to USD 220,000 a year.
See Machine Learning Engineer salaries in New York →
Similar jobs
- Machine Learning and Python SME at Avalore, LLC
Shares 5 skills: PyTorch, LLM, Machine Learning… · same seniority · same country - Machine Learning Data Scientist - Consumer Forecasting at OpenAI
Shares 5 skills: PyTorch, LLM, Machine Learning… · same country - Machine Learning R&D at Qualcomm
Shares 4 skills: PyTorch, LLM, Machine Learning… · same seniority · same country - Senior Machine Learning Engineer I at Axon
Shares 4 skills: PyTorch, LLM, Machine Learning… · same country - Machine Learning Engineer III, Shield at Box
Shares 4 skills: PyTorch, LLM, Machine Learning… · same country - Software Engineer at Raydar
Also at this company
More jobs like this
Is this your posting? Email info@careerholo.com to have it removed.