probrSee if there's a fit

Evaluation & post-training for models already live in production.

Custom eval datasets●Benchmarking●SFT●RLHF●RLVR●Failure analysis●Proven lift●

We find where yourmodel fails inproduction,then fix it.

Custom evaluation datasets, benchmarking, and post-training for teams already running AI in production.

See if there's a fit→

2026

↓ Scroll

The gap

↘
Public benchmarkYour own dataSilent failureProduction driftEdge casesLong-tail inputsRegression risk

Most teams can tell you how their model scores on a public benchmark. Very few can tell you where it fails on their own data, in production. That gap is where things quietly break, and it's where we work.

What we do

Two services / one loop

01 / EVALUATION

Evaluation

We turn your production logs into a labelled evaluation set for your task.

You keep the dataset, the harness, and per-slice scores — re-runnable on any model or checkpoint.

  • —Failure taxonomy from your own traffic, not a generic rubric
  • —Human-labelled gold set plus judge agreement rate
  • —Per-slice scores: edge cases, long tail, regressions
  • —Versioned harness you own and run in CI
Deliverable: dataset + harness + score report
02 / POST-TRAINING

Post-training

We train against the failing slices the eval found.

Every run is re-scored on the same benchmark, with a regression suite so one fix doesn't break another slice.

  • —SFT sets built from your failing cases, not synthetic filler
  • —RLHF preference pairs with documented annotation guidelines
  • —RLVR with verifiable rewards where outputs can be checked
  • —Before/after scores per slice, plus regression checks
Deliverable: checkpoints + training data + score delta
The wedge

Three steps, in order

01.

Bring us your
raw data.

Logs, transcripts, tickets, labels — whatever production actually produces. We handle intake, PII scrubbing, and sampling.

1–2 weeks
02.

We build a custom eval
and benchmark for
your use case.

Failure taxonomy, held-out splits, human-labelled gold set, and a calibrated judge — scored against the behaviour your business depends on.

3–5 weeks
03.

We post-train to close
the gaps we find,
then prove the lift.

SFT, RLHF, and RLVR against the failing slices, then the same benchmark re-run. Same benchmark, before and after — the delta is the deliverable.

6–12 weeks

Start with the eval.
If the numbers say the model is fine,
we tell you that too.

See if there's a fit→

Track record

100+

projects delivered since 2019

~1,000

consultations since 2019

~10

clients served, including enterprises

Teams we've worked with

desireOS
GENIUS
Heiko
Mercor logoMercor
Abundant
Clinpax logoClinpax
See what our clients have to say

Testimonials

Architected a multi-agent AI Scribe for clinical notes, billing codes, and letters, plus a knowledge-graph clinical Q&A co-pilot benchmarked against leading commercial scribes.
Clinpax logo

Clinpax

Project outcome

Probr AI brings depth and breadth of knowledge in AI and LLMs that is exceptional; honest about what would work, what wouldn't, and where to focus effort.
H

Heiko

Client

Excellent research work, always informative and open to innovation; they built our AI sales bot from raw data, properly aligned for product sales.
D

DesireOS

Client

Who's behind it

Led by a team of research scientists with a PhD in agentic systems, LLM post-training, and evaluation, paired with commercial leads embedded in the global AI ecosystems.

Domains we work across

↘
Selected domain

Healthtech