True Talent Solutions Partners

Research Engineer, Evals

San Francisco
Salary
$250,000 - $290,000 annually
Level
Mid
Work type
On-site · San Francisco
Visa
Sponsors visas

Open. First seen 5 November 2026.

Apply on the company's site →

About the role

About the Role

Build evaluation systems for a web data extraction product that turns URLs into clean, structured data for AI applications. You will define how output quality is measured and connect evaluation results to model and product decisions.

What You'll Do

  • Design rigorous metrics for extraction quality across diverse websites, formats, and edge cases.
  • Build scalable evaluation pipelines and harnesses.
  • Generate and curate trusted datasets for measuring extraction quality.
  • Close the feedback loop between evaluation results and model or product decisions.
  • Make evaluation results clear and actionable for the broader team.

What We're Looking For

  • Typically 4 to 10+ years of relevant experience, including at least 2 years focused on evaluations.
  • Experience building evaluation systems for LLM-based structured extraction from messy, real-world web pages or documents, including field-level accuracy measurement at scale.
  • Strong Python skills and experience with machine learning, LLMs, data pipelines, evaluation frameworks, web scraping, NLP, and statistical analysis.
  • Ability to design evaluation metrics and systems from the ground up, apply statistical rigor, and build gold-label or reference-extraction datasets.
  • Experience combining research methodology with engineering, and using evaluation findings to inform real model or product decisions.

Compensation & Benefits

Base salary: $250,000 to $290,000 USD annually. Visa sponsorship is available.

Location

Hybrid, with the primary location in San Francisco, California. Toronto, Canada is also a work location. The role requires working on-site five days a week in San Francisco or Toronto.

What this role asks for

Particular to this role

NLP

Rare in True Talent Solutions Partners's other 124 open postings.

Standard for software engineering roles

Python

Common in software engineering postings at any company.

Compared with True Talent Solutions Partners's other open jobs.

The team: Client Roles

This role is one of 125 open in Client Roles. 125 were posted in the last 30 days, including 7 lead roles: the team is hiring the people who run it too.

WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS

Python (66 of 125), LLM (50 of 125), TypeScript (44 of 125), React (39 of 125), Machine Learning (30 of 125), JavaScript (28 of 125), AWS (25 of 125)

OTHER ROLES ON THIS TEAM

From the team and dates on True Talent Solutions Partners's own job board.

Posting history

Reposted 4 times since 30 October 2026: a posting for this role in San Francisco was taken down and the role was posted again.

Similar jobs

More jobs like this

SKILLSPython 47,541Machine Learning 20,110LLM 18,849NLP 2,461
PLACESSan Francisco 7,720United States 74,582
COMPANYJobs at True Talent Solutions Partners 125

Is this your posting? Email info@careerholo.com to have it removed.