Building Complex AI Workflows with Jev

On 20 synthetic expense claims, a single Jev question applying an entire reimbursement policy matched 17 expected decisions. A graph using the same model for narrow judgments, with policy rules and calculations in code, matched all 20.

The three differences involved a spending total, a deadline, and a claim just above a monetary limit. They were decisions that ordinary software could calculate exactly.

This is a small experiment, deliberately constructed to exercise policy boundaries. It does not establish production accuracy or superiority over another model. It does demonstrate something more useful than a feature list: the capability of a model in one call does not define the capability of a system built around it.

My earlier exploration of Jev included capability probes and a prototype that translated natural-language travel requests into structured search parameters. For this article, I ran 75 fresh API calls against jev-1.13.0, using only synthetic inputs. I retained the requests, responses, expected outcomes, token usage, and timings.

The question I wanted to investigate was whether a task that appears to require extended reasoning can be expressed as a graph of smaller judgments, with no free-text generation along the way.

Jev makes that an interesting question because its interface is built around bounded answers. Choice selects an option; Noul answers a yes-or-no question with a probability; Score evaluates against ordered descriptions. In the live tests, I exercised all three, including four questions in a request and a separate 32-question fan-out. Those calls returned the requested typed answers. The fan-out was a mechanics check using repeated questions, not evidence of 32 distinct reasoning abilities. TypeSafe primitive definitions

These interfaces encourage a useful discipline: specify the judgment precisely enough that another piece of software can consume it.

The expense experiment

The expense experiment made that discipline concrete. The policy distinguished meals, travel, equipment, and other expenses. It required a receipt, set a 30-day submission window, and applied category-specific spending limits. A matching receipt supported a claim; a clearly mismatched receipt caused rejection; insufficient evidence required review. Claims over a limit needed manager approval.

In the single-question version, Jev received the full policy and selected the final action. In the graph version, code first checked the submission dates and receipt availability. Jev then classified the expense. A second Jev call received that category and compared the claim with the receipt description. Code applied the remaining rules and calculated the total.

That second call depended on the first. It was a small sequential graph, rather than a collection of unrelated questions asked together.

Results and policy

The results were:

• Single policy question: 17 of 20 expected decisions, using 20 model requests and 12,551 input tokens.
• Decision graph: 20 of 20 expected decisions, using 30 model requests and 11,537 input tokens. Four cases exited before inference; two stopped after classification.
• Observed median model wait per case: 636 milliseconds for the single question and 602 milliseconds for the graph. The graph figure sums sequential request times and assigns zero model wait to cases resolved entirely in code. It excludes application overhead.

The small timing difference is not a speed claim. There was one run per case and approach, with execution order alternated. The improvement that matters in this experiment is the removal of three avoidable decision errors.

A hotel claim contained amounts of 180 and 145 against a limit of 300. A meal claim totalled 50.20 against a limit of 50. A travel claim was submitted after 31 days against a 30-day window. The single-question version approved all three. The graph sent the first two for manager review and rejected the late claim.

I did not inspect hidden reasoning, so I cannot prove which internal step failed. I can say that the graph placed those calculations under deterministic control and produced the expected outcomes.

The final policy node is short enough to inspect directly. Dates and missing receipts have already been checked before this function runs. The decimal version below was replayed against all 16 recorded paths that reached classification and matched their expected results.

from decimal import Decimal

LIMITS = {"meal": Decimal("50"), "travel": Decimal("300"),
          "equipment": Decimal("200")}

def apply_policy(claim, category, evidence):
    if category not in LIMITS:
        return "review"
    if evidence == "mismatch":
        return "reject"
    if evidence != "supports":
        return "review"
    total = sum(Decimal(str(x)) for x in claim["line_amounts"])
    if total > LIMITS[category] and not claim["manager_approved"]:
        return "manager_review"
    return "approve"

Where decomposition works

The architectural implication is broader than reimbursement. A complex task can be implemented with Jev and code when its semantic work can be decomposed into sufficiently reliable, bounded judgments, and the graph preserves the information needed to combine them correctly.

“System 1” and “System 2” are useful shorthand here for small judgments and extended reasoning. I am using them as architectural descriptions, not claims about human cognition or model internals.

The qualification is in the decomposition. Every semantic node needs an answer space that covers the relevant outcomes, enough evidence to answer, and acceptable measured performance. Edges must carry sufficient state. Code must handle dependencies, iteration where needed, and stopping conditions. The complete graph must meet a latency and cost budget.

The absence of free-text generation makes a bounded-output design more plausible. It does not guarantee a practical decomposition. A difficult scheduling problem can return only a schedule identifier and still require substantial search. A diagnosis can return one label and still demand evidence the system does not have. Some tasks need a solver, a richer reasoning model, additional information, or a person.

Decomposition also moves part of the reasoning into the engineer who designs the graph. In this experiment, I specified policy precedence, category limits, and which evidence mattered. That is valuable engineering work, and part of what the result measures.

Questions and confidence

The smaller classification tests exposed why this work cannot be skipped. Across 12 synthetic support messages, the initial topic question matched 11 expected labels. A separate technical-issue question also matched 11. Both differed from my intended answer on this message:

The export error was fixed. Now I only need the duplicate charge refunded.

The model identified both billing and technical issues. My expected label was billing only. The original questions did not clearly say that resolved issues should be excluded, so treating this simply as a model failure would miss the design error.

I rewrote the questions to ask about currently unresolved problems. The original case then matched the intended outcome, as did five additional authored examples. This was a targeted repair followed by a small follow-up check, not an independent benchmark. It showed how an apparently minor temporal distinction changes the task.

A second probe was more revealing. I asked Jev to classify “Please change the office wallpaper to green” using only refund, replacement, and technical support as options. It chose technical support with 0.86 confidence. Adding an “other or unclear” option changed the answer to that option, with 0.85 confidence.

A confident preference among available answers cannot establish that the answer set is adequate.

TypeSafe documents Choice and Score confidence as a statistic derived from the returned probability distribution. It should not be read automatically as a measured probability of correctness for a particular workflow. TypeSafe confidence documentation

In the expense experiment, the incorrectly approved 31-day claim carried confidence of 0.73. A rule that automatically accepted every answer above 0.7 would have accepted that mistake. The wallpaper example would have cleared 0.8. These observations support testing thresholds against labelled outcomes, including missing-category cases. They do not establish a universally better threshold.

Calibration is an empirical property: probability estimates must correspond to observed outcomes. A field named “confidence” does not establish it. Guo and colleagues on calibration

Choosing the right component

These results give me a practical way to decide where to use Jev.

Use it for bounded semantic judgments with inspectable criteria. Topic classification, detecting an explicitly defined condition, and comparing a claim with a supplied description fit the interfaces exercised here. Write the question so a reviewer can identify the evidence supporting the answer. Include meaningful outcomes for missing information or an unsuitable input.

Use a graph when the larger decision has separable semantic and deterministic parts. The expense workflow is a tested example. A similar design is worth evaluating for document triage or service routing where code can own thresholds, permissions, and policy precedence. Those are candidate applications, not additional capabilities validated by this experiment. Dependent judgments need separate stages; independent judgments may share a request. A wide fan-out does not remove a genuine dependency.

Prefer ordinary code when the inputs already determine the answer exactly. Dates, totals, identifiers, explicit permissions, and policy limits should not become probabilistic questions merely because a model is available. The three expense-case differences show the practical value of this boundary. TypeSafe’s published guidance likewise identifies arithmetic and date comparison as weaknesses to avoid assigning directly to Jev 1.13. Jev 1.13 limitations

Choose another component when the task needs open-ended synthesis or search. The interface tested here selected supplied outcomes and returned scores; it did not draft explanations, invent plans, or search an unrestricted space of solutions. I would use a generative model for prose and novel proposals, and an appropriate solver for exact optimisation. Jev could still evaluate bounded parts of those workflows, but that combination would need its own tests.

Do not use confidence as authorisation or as the sole security boundary. A model can help interpret a request. Permission to execute it must come from trusted application state. One deliberately instruction-bearing expense description produced the expected result in this run; that is far too little evidence to establish resistance to adversarial inputs. A sensitive workflow needs adversarial evaluation and controls outside the model.

Avoid a graph whose complexity costs more than it saves. Every additional stage introduces orchestration, latency, intermediate state, and another possible error. If preserving meaning requires a sprawling taxonomy or dozens of fragile transitions, a larger reasoning model may be the more maintainable choice. Keep conventional classifiers in the comparison when labelled data and a stable task make them viable.

Typed output alone is also an insufficient reason to switch. General-purpose model APIs offer schema-constrained outputs; Claude’s documentation is one example. A fair evaluation would give competing models the same decomposition and downstream rules. This experiment compared two ways of using Jev, not Jev against frontier models or trained classifiers. Claude structured outputs

Economics and whole-graph evaluation

The economics deserve the same precision. At TypeSafe’s published input rate of $0.042 per million tokens, the graph’s 11,537 input tokens imply roughly $0.000485 in model charges for the 20 expense cases. Output tokens are uncharged under that pricing. This is a calculation from recorded usage, not an invoice measurement, and excludes the surrounding system. Jev pricing

At that scale, review effort, engineering maintenance, and the consequences of mistakes can dominate token cost. I would choose an architecture using the proportion of cases safely automated, the errors left among those cases, end-to-end latency, and total operating cost.

I would also test the graph as a unit. A correct category does not guarantee a correct reimbursement decision. An incorrect early category can change which evidence the next node examines. A failed service call can leave the workflow without a usable answer. Production readiness requires tests for those paths, not just successful node outputs.

The most defensible claim from this work is that a modest decision graph extended what I could reliably accomplish with Jev on a defined test set. It did so by narrowing the semantic questions and giving exact computation and policy execution to code.

That is a useful design pattern to carry into the next project: make the judgments small enough to evaluate, make the policy explicit enough to inspect, and measure the outcome of the whole graph.