Skip to main content
FitForJev

Blog · 9 min

Jev fails nine ways. You handle each in code.

Nine documented failure modes and who handles each: code, Jev, LLM, human. You answer six questions to check your read.

jaggednessverdictsuse_codehybrid

Jev documents its failures. You design around them. The 1.13 jaggedness list is at docs.typesafe.ai/model-jaggedness and defines the other half of the API contract. Each item names work you own in code. What the model cannot do, you handle.

01

Literal reading

Jev follows literal conditions and cannot infer intent. Phrase each condition to stand alone.

02

Math & arithmetic

Jev fails counting and summing. You keep math in code.

03

Date & time

Jev cannot do deltas or ordering. You parse and compare in code.

04

Multi-hop indirection

Jev struggles with links from A through B to C. You split them into atomic direct checks.

05

Context rot

Irrelevant state lowers accuracy. You send only the state the question needs.

06

Adversarial content

Jev 1.13 trusts state by default. You guard injection in code.

07

Contradictory criteria

You write instructions and criteria that agree.

08

Broken invariants

P(yes) and P(no) do not sum to 1 across Nouls. You check each side on its own.

09

Prose / code generation

Jev cannot write text or code. You route generation to an LLM.

The Jev Test flowchart triages in three checks: prose needed, code-sufficient, judgments atomic. Then you choose from five verdicts. native means pure judgment. hybrid splits judgment to Jev and prose to the LLM. About 60 percent of enterprise cases land here. weak means a small salvageable core. not_jev means the wrong tool. use_code means no AI is needed. need_info covers inputs that need more detail. The badge names the verdict. The numbers beside it give the score band in small monospace type.

The Jev test · 6 questions

Score is 0 of 6.
  1. "Route support tickets to billing, tech, or sales."

  2. "Review a PR and write comments explaining issues."

  3. "Classify clothing items from product photos."

  4. "Validate that a payload matches the JSON schema."

  5. "Write end-to-end marketing campaign copy."

  6. "Make our customer support better."

Each verdict divides work four ways (SPEC section 5.4). You keep exact checks, math, and dates in code: under 1ms, $0, no fuzzy judgment. You keep regex, thresholds, and state prep there too. You give Jev atomic Choice, Score, and Noul gating. That path takes about 100ms and costs $0.042 per 1M tokens, with no prose or multi-hop reasoning. You give frontier LLMs long-form prose and synthesis. That path takes 2 to 8 seconds and costs $5 to $15 per 1M tokens. You keep LLMs out of tight latency loops. You review confidence edges from 0.3 to 0.7. That review takes hours to days. It costs more than any other path, so you spend none of it on repetitive triage. With broad input such as "manage our sales pipeline", you isolate the Jev-native core: lead Score, intent Choice, and budget and churn Nouls. Route the rest away.

You archive the evaluator when TypeSafe ships official translation from problems to typed questions or keeps waitlist access closed past 90 days. You also archive it when this page turns into a generic prompt directory. Until then, paste the fuzzy version and answer the typed questions.

Try it on your own problem.

Paste the fuzzy version and get a verdict: Jev answers the typed questions, your code computes the fit.

Evaluate your problem