True Talent Solutions Partners

Software Engineer, RL Environments

San Francisco
Salary
$260,000 - $290,000 annually
Level
Mid
Work type
On-site · San Francisco
Visa
Sponsors visas

Open. First seen 3 November 2026.

Apply on the company's site →

About the role

About the Role

Build datasets, evaluation rubrics, and reward signals that help improve large language models. You will work with AI research teams to test data-collection approaches, investigate model failure modes, and develop measurements that inform training runs.

What You'll Do

  • Design data slices that reveal meaningful model failure modes across areas such as finance, software, and enterprise workflows.
  • Build and refine evaluation rubrics and reward signals for RLHF and RLVR pipelines.
  • Model annotator behavior and run experiments to improve model capabilities.
  • Develop quantitative methods to measure dataset quality, diversity, and downstream impact.
  • Create and manage real-world and synthetic data pipelines.
  • Work with research teams to turn training objectives into concrete data and evaluation specifications.

What We're Looking For

  • One to four years of hands-on experience designing machine learning data-collection strategies and evaluation rubrics.
  • Familiarity with large language model training and evaluation methods.
  • Experience creating data slices to reveal model failure modes across domains such as finance, code, or enterprise workflows.
  • Ability to design lightweight experiments, work through ambiguous results, and draw actionable insights.
  • A practical, builder-oriented approach and interest in how data quality and structure shape model behavior.

Compensation & Benefits

Base salary is $260,000 to $290,000 annually. Visa sponsorship is available.

Location

On-site in San Francisco, California.

The team: Client Roles

This role is one of 126 open in Client Roles. 126 were posted in the last 30 days, including 7 lead roles: the team is hiring the people who run it too.

WHAT THE TEAM WORKS WITH, ACROSS ITS POSTINGS

Python (63 of 126), LLM (49 of 126), TypeScript (45 of 126), React (40 of 126), Machine Learning (31 of 126), JavaScript (27 of 126), AWS (26 of 126)

OTHER ROLES ON THIS TEAM

From the team and dates on True Talent Solutions Partners's own job board.

Posting history

Reposted 2 times since 25 October 2026: a posting for this role in San Francisco was taken down and the role was posted again.

See Software Engineer salaries in San Francisco →

Similar jobs

More jobs like this

SKILLSMachine Learning 19,840LLM 18,602
PLACESSan Francisco 7,694United States 73,396
COMPANYJobs at True Talent Solutions Partners 126

Is this your posting? Email info@careerholo.com to have it removed.