Destava

Product Manager - AI Evaluations Tooling

Relativity is hiring for Product Manager - AI Evaluations Tooling in Illinois. Hybrid, Mid level, $115k–$173k a year stated. First seen November 13, 2026, posted 2 times; open as of November 19, 2026. Jobs taken straight from company career pages — no aggregator in between.

Save to applyFree forever. No commitment, no card required.

Company
Relativity
Where
Illinois
Arrangement
Hybrid
Level
Mid
Pay stated
$115k–$173k a year
First seen
November 13, 2026
Postings
2
Listed via
Workday
Open for
6 days, as of November 19, 2026

Applying to this one?

See how well you fit, before you apply

Save it to your board. Add your résumé once, and Destava shows where you are strong, what is missing and how to pitch yourself, then tailors your résumé to this posting.

  1. Found it
  2. Save it
  3. See your fit

Free forever. No commitment, no card required.

Or apply on Relativity's own site

About this job

Relativity seeks a Product Manager to lead strategy and roadmap for its AI evaluation platform. This position is part of the Evaluations Tooling team, which develops a shared platform for company-wide AI performance measurement and enhancement. The role offers remote or hybrid work in Illinois.

Responsibilities

  • Direct the product vision, strategy, and roadmap for the Evaluations Platform, covering offline evaluation pipeline, online evaluation and production monitoring, eval discovery and reuse, SME authoring and approval, and agentic evaluation.
  • Perform ongoing discovery with applied scientists, application engineering teams, and product PMs to grasp their AI quality experiences, designing the platform to let teams measure and improve performance independently without involving an applied scientist for every query.
  • Coordinate the development of a top-tier agentic testing system, including trajectory evaluation, tool-call correctness, intermediate-state rubrics, long-running judge orchestration, and multi-step flow simulation. Ensure rapid and safe deployment of new agentic features, turning the platform into a
  • Collaborate with legal subject-matter experts to define AI evaluation tasks, correct outcome criteria, challenging cases, and judgment expression for encoding, scoring, and reuse across products. Construct the SME authoring and approval workflow around expert practices.
  • Manage team-facing quality dashboards so every AI product team can view their performance, compare against quality standards, and see changes since the last release without engineering support.
  • Conduct build-versus-buy assessments for tracing infrastructure and commercial eval tools, with recurring reviews as the vendor market evolves.
  • Collaborate with engineering and applied science on judge calibration to ensure LLM-as-judge reliability for deployment gating.
  • Communicate technical tradeoffs clearly to audiences from applied scientists and attorneys to executive leadership.

Requirements

  • At least 4 years in product management at a tech firm, including 2+ years on technical, platform, infrastructure, or data/ML products.
  • Strong grasp of software development lifecycle and agile methodologies.
  • Proven ability to conduct user discovery with technical audiences and convert findings into roadmap choices.
  • Superior written and verbal communication, explaining complex technical ideas to technical and non-technical groups.

Nice to have

  • Familiarity with LLM-based products and practical knowledge of their evaluation methods (rubrics, LLM-as-judge, offline vs. online). Direct experience with agentic systems is beneficial but not mandatory.
  • Background as a platform, developer-tools, or data/ML PM serving technical clients, or a demonstrated capacity to quickly master a technical field and gain trust with engineers and scientists.
  • Ease in integrating non-engineering domain experts into technical processes and converting their judgment into actionable system inputs.
  • Sharp analytical skills: sets metrics, tests hypotheses, and pivots based on evidence.

Benefits

  • Attractive base pay, yearly performance bonus, and long-term incentive opportunities.

Read more and apply on Relativity's site

Questions about this job

When was this Product Manager - AI Evaluations Tooling job at Relativity posted?

November 13, 2026

Destava first saw it on November 13, 2026, on Relativity's own careers page (Workday). Relativity has listed this job at this place 2 times in the last 90 days, counting the ones taken down.

Is Relativity still accepting applications for it?

Open as of November 19, 2026

It was open on Relativity's own site when Destava last read it, on November 19, 2026, 6 days after we first saw it. When the company takes it down, this page says the job has closed.

How do I apply for Product Manager - AI Evaluations Tooling at Relativity?

On Relativity's own site

On Relativity's own site, at kcura.wd1.myworkdayjobs.com: open the posting. Destava does not take the application. Save the job to your Destava board to track it, see how well you fit and tailor your résumé to it.

Is this Relativity job posting legit?

Listed on Relativity's own careers page

Destava read it on Relativity's own careers page at kcura.wd1.myworkdayjobs.com, run on Workday, not on a job board. First seen November 13, 2026, posted 2 times. Apply on the company's own site, and never pay to apply for a job.

What does this job pay?

$115k–$173k a year

The posting states $115k–$173k a year.

Is this job remote?

Hybrid

It is hybrid: part of the week on site in Illinois, the rest from home.

With Destava: Save to apply · Tailor my résumé · Write a cover letter