Sample data. These figures are not a customer bill.Learn more
parezenparezen
Northwind AI
Scale · Sample · cloud
Home
  • Get startedSoon
  • Audits
  • Overview
Analyse
  • Spend
  • Cost per task
  • Agents
  • Model comparisonSoon
  • Budgets
  • AlertsSoon
  • Reports
  • PR cost checksSoon
  • AdvisorsSoon
  • Spend analysis
Optimise
  • RecommendationsSoon
  • Routing policiesSoon
  • Shadow testsSoon
  • CachingSoon
  • TrimmingSoon
  • BatchSoon
  • Waste detectorSoon
Gateway
  • SetupSoon
  • ProvidersSoon
  • ModelsSoon
  • API keysSoon
  • DeploymentsSoon
  • PlaygroundSoon
  • LogsSoon
Savings & billing
  • Savings history
  • Savings ledger
  • Bill match
  • Billing
  • Invoices
Settings
  • Data & privacySoon
  • OrganisationSoon
  • ProjectsSoon
  • TeamSoon
  • RolesSoon
  • SSOSoon
  • Audit logSoon
Plan: Scale · Sample ↗
DocsHelp

Sample data. Not a customer bill.

Northwind AI/Shadow tests/Shadow test

Shadow test

Planned

Score distribution, cost delta and failing examples if logging is on.

Passanthropic/claude-haiku-4.5 vs anthropic/claude-sonnet-4.5Document extract
View recommendation
Score
0.90
Bar 0.86 · 10% sample
Estimated cost delta
▼ -66.7%
Candidate $0.02 · baseline $0.07
Latency delta
▲ 0.0%
680 ms vs 680 ms p50

Score distribution

Samples relative to the customer's threshold.

Green bars clear the 0.86 bar. Amber bars do not.

Failing examples

Only the tail under the bar. Prompts stay hidden unless content logging is on.

  • Score 0.77

    Extract the invoice total. The supplier block repeats the same address twice and the line items run past the usual page.

  • Score 0.80

    The statement date is ambiguous between the header and the footer. Context includes the full prior-month ledger.

  • Score 0.82

    Currency is named in the preamble and again in every row. The repeated instruction asks for USD and for the source code.