Product Manager - AI Evaluations Tooling
Relativity is hiring for Product Manager - AI Evaluations Tooling in Illinois. Hybrid, Mid level, $115k–$173k a year stated. First seen November 13, 2026, posted 2 times; open as of November 19, 2026. Jobs taken straight from company career pages — no aggregator in between.
Save to applyFree forever. No commitment, no card required.
- Company
- Relativity
- Where
- Illinois
- Arrangement
- Hybrid
- Level
- Mid
- Pay stated
- $115k–$173k a year
- First seen
- November 13, 2026
- Postings
- 2
- Listed via
- Workday
- Open for
- 6 days, as of November 19, 2026
Applying to this one?
See how well you fit, before you apply
Save it to your board. Add your résumé once, and Destava shows where you are strong, what is missing and how to pitch yourself, then tailors your résumé to this posting.
- Found it
- Save it
- See your fit
Free forever. No commitment, no card required.
About this job
Relativity seeks a Product Manager to lead strategy and roadmap for its AI evaluation platform. This position is part of the Evaluations Tooling team, which develops a shared platform for company-wide AI performance measurement and enhancement. The role offers remote or hybrid work in Illinois.
Responsibilities
- Direct the product vision, strategy, and roadmap for the Evaluations Platform, covering offline evaluation pipeline, online evaluation and production monitoring, eval discovery and reuse, SME authoring and approval, and agentic evaluation.
- Perform ongoing discovery with applied scientists, application engineering teams, and product PMs to grasp their AI quality experiences, designing the platform to let teams measure and improve performance independently without involving an applied scientist for every query.
- Coordinate the development of a top-tier agentic testing system, including trajectory evaluation, tool-call correctness, intermediate-state rubrics, long-running judge orchestration, and multi-step flow simulation. Ensure rapid and safe deployment of new agentic features, turning the platform into a
- Collaborate with legal subject-matter experts to define AI evaluation tasks, correct outcome criteria, challenging cases, and judgment expression for encoding, scoring, and reuse across products. Construct the SME authoring and approval workflow around expert practices.
- Manage team-facing quality dashboards so every AI product team can view their performance, compare against quality standards, and see changes since the last release without engineering support.
- Conduct build-versus-buy assessments for tracing infrastructure and commercial eval tools, with recurring reviews as the vendor market evolves.
- Collaborate with engineering and applied science on judge calibration to ensure LLM-as-judge reliability for deployment gating.
- Communicate technical tradeoffs clearly to audiences from applied scientists and attorneys to executive leadership.
Requirements
- At least 4 years in product management at a tech firm, including 2+ years on technical, platform, infrastructure, or data/ML products.
- Strong grasp of software development lifecycle and agile methodologies.
- Proven ability to conduct user discovery with technical audiences and convert findings into roadmap choices.
- Superior written and verbal communication, explaining complex technical ideas to technical and non-technical groups.
Nice to have
- Familiarity with LLM-based products and practical knowledge of their evaluation methods (rubrics, LLM-as-judge, offline vs. online). Direct experience with agentic systems is beneficial but not mandatory.
- Background as a platform, developer-tools, or data/ML PM serving technical clients, or a demonstrated capacity to quickly master a technical field and gain trust with engineers and scientists.
- Ease in integrating non-engineering domain experts into technical processes and converting their judgment into actionable system inputs.
- Sharp analytical skills: sets metrics, tests hypotheses, and pivots based on evidence.
Benefits
- Attractive base pay, yearly performance bonus, and long-term incentive opportunities.
Read more and apply on Relativity's site
Questions about this job
When was this Product Manager - AI Evaluations Tooling job at Relativity posted?
November 13, 2026
Destava first saw it on November 13, 2026, on Relativity's own careers page (Workday). Relativity has listed this job at this place 2 times in the last 90 days, counting the ones taken down.
Is Relativity still accepting applications for it?
Open as of November 19, 2026
It was open on Relativity's own site when Destava last read it, on November 19, 2026, 6 days after we first saw it. When the company takes it down, this page says the job has closed.
How do I apply for Product Manager - AI Evaluations Tooling at Relativity?
On Relativity's own site
On Relativity's own site, at kcura.wd1.myworkdayjobs.com: open the posting. Destava does not take the application. Save the job to your Destava board to track it, see how well you fit and tailor your résumé to it.
Is this Relativity job posting legit?
Listed on Relativity's own careers page
Destava read it on Relativity's own careers page at kcura.wd1.myworkdayjobs.com, run on Workday, not on a job board. First seen November 13, 2026, posted 2 times. Apply on the company's own site, and never pay to apply for a job.
What does this job pay?
$115k–$173k a year
The posting states $115k–$173k a year.
Is this job remote?
Hybrid
It is hybrid: part of the week on site in Illinois, the rest from home.
More jobs like this
A search agent finds the jobs this list misses: the ones that mean the same without the same words. Have Destava watch for product manager - ai evaluations tooling jobs in Illinois, new matches every morning on your own account.
With Destava: Save to apply · Tailor my résumé · Write a cover letter