Project management for AI product teams

The board that knows when your model got worse.

Backlog, sprints and bugs, with your evals built in. When quality, cost or latency slips, Evalban files the bug, blocks the merge and tells Slack, with the failing examples attached.

Free for teams of up to 5. We're onboarding AI product teams as design partners now.

Listing classifier · Macro F15 runs · baseline 0.880

EVB-133· Filed automatically from run 5

Eval regression: Listing classifier Macro F1 0.871 (baseline 0.880)

S2 MajorAI failure3 failing examplesfails 7 of 20 re-runs

Evalban in #ml-alerts
Listing classifier regressed on ft-1027. Filed EVB-133 and blocked the merge of PR #42.

From a bad eval run to a fixed bug, without anyone copying numbers into a ticket.

Most trackers know when code ships. Evalban also knows whether the model behind it still works.

Send evals from CI
One line in your eval script or GitHub Action posts the score, the prompt or model version, and any failing examples.
client.log_run("listing-classifier", 0.871, version="ft-1027")
A drop files the bug
Below baseline, Evalban opens an AI failure bug with the inputs and outputs that failed, and assigns it to whoever owns the work.
0.871 < 0.880 → EVB-133
Done means it passes
The issue can't move to Done, and a merged pull request won't close it, until a new run is back above baseline.
Definition of done: linked evals pass ✕
The team hears first
Slack gets the regression, the blocked merge and any S1 bug the moment they happen, before a customer notices.
#ml-alerts · 10:42 AM

Everything a small AI team runs on, in one place.

Replace the tracker, the eval spreadsheet and the Slack thread where someone asks "what changed on Tuesday?"

A board built for AI work

Backlog, sprints, Scrum or Kanban, and a workload view that shows who's over capacity before the sprint slips.

Eval scores on every card that has a model behind it
A plain markdown wiki for specs and model cards

Bugs for things that fail sometimes

AI failures rarely reproduce every time. Evalban records the input, the output and the trace, then measures how often it fails.

fails 7 of 20 runs4.2% of users → S2

Every version, with what it costs

Each prompt and model version keeps its score, cost per call and latency side by side. Roll back to the last good one and it's recorded.

prompt v6 · 0.84 · $0.0021 · 840 msgpt-big · $0.0094 · over budget

Connected to where you work

Pull requests that mention an issue move it to review, merges move it to Done, and Slack gets the alerts that matter.

GitHub, Slack, and a GitHub Action for evals
Python and TypeScript SDKs, plus a plain HTTP API

Five minutes from sign-up to your first caught regression.

Create an API key, add one call where your evals already run, and set a baseline.

eval.py

from evalban import Evalban

client = Evalban(api_key=os.environ["EVALBAN_API_KEY"],
                 url="https://api.evalban.com")

res = client.log_run("listing-classifier", 0.871,
    version="ft-1027",
    failures=failing_examples,
    cost_per_call_usd=0.0004, latency_p95_ms=168)

# {"gate": "regressed", "bug_key": "EVB-133", ...}

.github/workflows/evals.yml

# after your eval step
- uses: evalban/eval-action@v1
  with:
    api-key: ${{ secrets.EVALBAN_API_KEY }}
    experiment: listing-classifier
    results-file: eval-results.json
    fail-on-regression: true

# The job summary shows the score vs baseline,
# and the build fails if the model got worse.

Simple per-person pricing.

Start free. Pay when the whole team is on it.

Free

For small teams trying Evalban on a real project. Up to 5 people.

Get early access

Most teams

Team

$12 per person / month

For AI product teams shipping every week, with evals, GitHub and Slack.

Get early access

Business

$22 per person / month

For larger teams that need more control over access, security and support.

Talk to us

Startups get 50% off. Prices in US dollars, billed monthly.

Catch the next regression before your users do.

We'll reply within a few days to set up your workspace.