Destava

AI Agent Evaluation Engineer

Cresta is hiring for AI Agent Evaluation Engineer in Canada (Remote); United States (Remote). Remote, Mid level. First seen November 13, 2026; open as of November 18, 2026. Jobs taken straight from company career pages — no aggregator in between.

Save to applyFree forever. No commitment, no card required.

Company
Cresta
Where
Canada (Remote); United States (Remote)
Arrangement
Remote
Level
Mid
First seen
November 13, 2026
Listed via
Greenhouse
Open for
5 days, as of November 18, 2026

Applying to this one?

See how well you fit, before you apply

Save it to your board. Add your résumé once, and Destava shows where you are strong, what is missing and how to pitch yourself, then tailors your résumé to this posting.

  1. Found it
  2. Save it
  3. See your fit

Free forever. No commitment, no card required.

Or apply on Cresta's own site

What the posting says

What You'll Do : Evaluation Design & Quality Definition Define and maintain the evaluation rubric for the agent across three axes: adherence (did it follow its design, call the right tools, respect guardrails), outcome (did the conversation achieve its goal), and conversation experience (latency, turn-taking, tone, repetition). Write the annotation guidelines and failure taxonomy the team uses to talk about failures precisely; run calibration sessions and measure inter-rater agreement. Classify and prioritize failure modes by severity and frequency so engineering and product work on what…

Read more and apply on Cresta's site

Questions about this job

When was this AI Agent Evaluation Engineer job at Cresta posted?

November 13, 2026

Destava first saw it on November 13, 2026, on Cresta's own careers page (Greenhouse).

Is Cresta still accepting applications for it?

Open as of November 18, 2026

It was open on Cresta's own site when Destava last read it, on November 18, 2026, 5 days after we first saw it. When the company takes it down, this page says the job has closed.

How do I apply for AI Agent Evaluation Engineer at Cresta?

On Cresta's own site

On Cresta's own site, at job-boards.greenhouse.io: open the posting. Destava does not take the application. Save the job to your Destava board to track it, see how well you fit and tailor your résumé to it.

Is this Cresta job posting legit?

Listed on Cresta's own careers page

Destava read it on Cresta's own careers page at job-boards.greenhouse.io, run on Greenhouse, not on a job board. First seen November 13, 2026. Apply on the company's own site, and never pay to apply for a job.

What other jobs is Cresta hiring for?

89 open postings

Cresta has 89 open postings on its own site: see all jobs at Cresta.

How many software quality assurance analysts and testers jobs are open?

10,064 open

10,064 open, 2,744 of them new this week, at 2,842 companies: see them all.

With Destava: Save to apply · Tailor my résumé · Write a cover letter

This page includes information from the O*NET 31.0 Database by the U.S. Department of Labor, Employment and Training Administration (USDOL/ETA). Used under the CC BY 4.0 license. O*NET® is a trademark of USDOL/ETA.