Typed Probabilistic Decisions in Java with Decision Models and Spring AI
Learn to use decision models in Java with Spring AI: typed yes/no, choice, and score answers with probabilities, plus guardrails for your chat models.
Large language models made it easy to handle tasks that used to require a specially trained machine learning model. You can prompt a chat model to classify a text, check whether an email is spam, pick the right data source to answer a question, or route a support ticket to the right team. No training required!
But is a chat model the right tool for the job? In all these scenarios, you don't need a model to write anything. You need a decision.
In this article, I'll show you how to use decision models in your Java applications with Spring AI, relying on the new DecisionModel API I designed for the 2.1 release and implemented together with Christian Tzolov (Spring AI Lead), with integrations for Ollama, OpenAI, and System One (including Jev). We'll build the customer support desk of an online musical instrument store, and use decision models to classify tickets, detect the customer's sentiment, and add guardrails to a chat model.
The Downsides of Using Chat Models for Decisions
Using a chat model for these scenarios is convenient, but it comes with some downsides:
- Latency and cost. Latency can be quite high, and you end up using far more tokens than the task needs.
- Prompt design. Prompts require careful design to get the desired output and to minimize the risk of hallucinations and prompt injection.
- Output. Chat models generate text, whereas we need something structured. Most inference services now support structured outputs, and Spring AI makes it easy to convert an answer into a Java type. Still, the model is generating strings, and you don't get a probability your application can act on.
Decision Models
On September 15th, TypeSafe AI announced Jev and started a new movement: models specialized in generating typed probabilistic decisions. We call them decision models.
Instead of generating text, a decision model answers typed questions about an input: yes or no, one of several options, or a level on a scale. Each answer comes with its probabilities, and your application decides what to do with them. We don't need to worry about the output format either, since it's defined in advance and documented in the System One API, announced together with Jev.
Decision models are also much faster than chat models. And since they select among the outcomes you define rather than generating text, they can't make up an answer outside those outcomes. That also limits what a prompt injection can achieve, though carefully crafted inputs can still influence the decision. A decision model can also simply be wrong, so think about what happens when it is.
Decision models are a good fit for classification, such as sorting customer messages by topic and sending each one to the right team. They also work well as guardrails, for example, to keep off-topic messages away from a chat model or to catch a reply that doesn't answer the user's question. And they help with everyday business logic, such as finding out whether a customer reports a broken product, or how satisfied the customer is.
The DecisionModel API in Spring AI
In Spring AI 2.1, we introduced the DecisionModel API, a portable abstraction for working with decision models. You send a DecisionPrompt made of an input and one or more named questions, and you get back a DecisionResponse with a typed answer for each question.
The input can be a free text, a set of named fields, or a conversation made of the same Message types you already use with a ChatModel. Then, you pick the type of question based on what your code does next:
YesNoQuestionwhen your code takes one of two paths. TheYesNoAnswercontains the probability of "yes".ChoiceQuestionwhen your code takes one of several paths. TheChoiceAnswercontains the selected option and the probability of each option.ScoreQuestionwhen you need a level on an ordered scale, such as a priority. TheScoreAnswercontains the most likely level, the probability of each level, and the expected level as a decimal number.
Notice what's missing: there's no prompt to fine-tune, no output format to describe, and no response to parse. It's time to see the example in action. Let's go!
Quick Start
Let's build the customer support desk of an online musical instrument store. We'll use Ollama, so that everything runs locally on your machine.
1. Project Setup
For this example, I'll use Spring Boot 4.2, Spring AI 2.1, Gradle, and Java 25. Here's how the dependencies section of your build.gradle file should look.
dependencies {
implementation 'org.springframework.boot:spring-boot-starter-webmvc'
implementation 'org.springframework.ai:spring-ai-starter-model-ollama'
testAndDevelopmentOnly 'io.arconia:arconia-dev-services-ollama'
testImplementation 'org.springframework.boot:spring-boot-starter-webmvc-test'
testRuntimeOnly 'org.junit.platform:junit-platform-launcher'
}
dependencyManagement {
imports {
mavenBom 'io.arconia:arconia-bom:0.32.0'
mavenBom 'org.springframework.ai:spring-ai-bom:2.1.0-RC2'
}
}
The Spring AI Ollama starter autoconfigures a DecisionModel bean for you, so there's nothing else to wire.
For the best performance, I recommend installing Ollama directly on your machine (version 0.40.4 or later, since the decision endpoint needs a recent version).
2. Configuring the Decision Model
We'll use d1-omni:600m, a small decision model available in Ollama. Let's configure it in the application.yml file, and let Spring AI pull it at startup if it's not on your machine yet. We don't need a chat model for now, so we exclude it from the initialization, together with the embedding model.
spring:
ai:
ollama:
init:
pull-model-strategy: when_missing
chat:
include: false
embedding:
include: false
decision:
model: d1-omni:600m
3. Asking a Yes/No Question
The first thing our support desk needs to know is whether a message is something it should handle at all. That's a job we might have given to a chat model (and I've done exactly that in the past), but it's really a yes/no question.
Let's create a TicketController class and use the auto-configured DecisionModel to ask it.
@RestController
@RequestMapping("/tickets")
class TicketController {
private final DecisionModel decisionModel;
TicketController(DecisionModel decisionModel) {
this.decisionModel = decisionModel;
}
@GetMapping("/on-topic")
YesNoAnswer onTopic(String text) {
return decisionModel.call(text, Map.of("onTopic", YesNoQuestion.builder()
.text("Is this message something the customer support of an online musical instrument store can help with?")
.whenYes("It asks about buying or owning an instrument or an accessory, or about orders, shipping, returns, billing, or the customer's account")
.whenNo("It is about anything else, such as cooking, travel, homework, music history, or general knowledge")
.build()))
.getYesNo("onTopic");
}
}
Each question has a name, which we use to retrieve its answer from the response. The optional whenYes and whenNo criteria define what counts as a yes and what counts as a no. Since this is a small model, they make a big difference.
4. Running the Application
Let's run the application.
./gradlew bootRun
Or, if you prefer using the Arconia CLI:
arconia dev
If no native Ollama connection is detected on your machine, Arconia will spin up an Ollama service for you, using Testcontainers. In that case, make sure you have a container runtime such as Podman or Docker available.
5. Calling the application
Let's verify the application is working as expected. I'll use httpie to send a few requests. First, a message the support desk should handle.
http :8080/tickets/on-topic text=="My order 48213 was supposed to arrive last week and the tracking page hasn't changed since Monday." -b
{
"probability": 0.95,
"yes": true
}
Next, a message that has nothing to do with our store. For example, a question a friend of mine asked me recently.
http :8080/tickets/on-topic text=="What's your recipe for lasagna?" -b
{
"probability": 0.01,
"yes": false
}
Here's a more interesting one. This message is about music, but it's still not something our customer support can help with.
http :8080/tickets/on-topic text=="Who composed the Moonlight Sonata?" -b
{
"probability": 0.32,
"yes": false
}
6. Choosing a Threshold
The model doesn't make the decision for you. It gives you a probability, and your application decides how sure it needs to be. The yes field in the answers above comes from isYes(), which uses a threshold of 0.5. You can pass your own, based on what a wrong decision would cost you.
@GetMapping("/on-topic/threshold")
boolean onTopicThreshold(String text, @RequestParam(defaultValue = "0.5") double threshold) {
return decisionModel.call(text, Map.of("onTopic", YesNoQuestion.builder()
.text("Is this message something the customer support of an online musical instrument store can help with?")
.whenYes("It asks about buying or owning an instrument or an accessory, or about orders, shipping, returns, billing, or the customer's account")
.whenNo("It is about anything else, such as cooking, travel, homework, music history, or general knowledge")
.build()))
.getYesNo("onTopic").isYes(threshold);
}
The model assigns a probability of 0.84 to the following message. With the default threshold, the support desk would handle it. With a threshold of 0.9, it wouldn't.
http :8080/tickets/on-topic/threshold text=="Which digital piano is best for a beginner?" threshold==0.9 -b
false
Keep in mind that probabilities are not guarantees, and you can't compare them across models or across questions with a different wording. Choose your thresholds by testing them on real examples from your application, and test them again whenever you change the model or the question.
Beyond Yes or No
Not every decision has two possible outcomes. Let's extend our support desk to cover the other types of question.
Choice Questions: Classifying a Support Ticket
Sorting support tickets is a classic classification scenario. Exactly one team should handle each ticket, which makes it a good fit for a ChoiceQuestion.
@GetMapping("/category")
ChoiceAnswer category(String text) {
return decisionModel.call(text, Map.of("category", ChoiceQuestion.builder()
.text("Which team should handle this support ticket?")
.option("billing", "Payments, invoices, refunds already issued, and duplicate charges")
.option("shipping", "Late, lost, or damaged deliveries, and changes to a delivery address")
.option("returns", "Sending an instrument back, or asking for a replacement or a refund")
.option("technical", "An instrument or piece of gear that doesn't work, or help with setting it up")
.option("account", "The customer's account on the store's website: logging in, resetting its password, and managing personal details")
.option("other", "None of the above")
.build()))
.getChoice("category");
Each option has a name and a description of the situation it covers. Notice the other option: without it, the model has to pick a team even for a ticket that doesn't belong to any of them.
Let's try it with a ticket that could belong to more than one team.
http :8080/tickets/category text=="The neck of my new guitar is cracked. Can I get my money back?" -b
{
"confidence": 0.91,
"probabilities": {
"account": 0.0003,
"billing": 0.009,
"other": 0.03,
"returns": 0.93,
"shipping": 0.0003,
"technical": 0.03
},
"probability": 0.93,
"rankedOptions": ["returns", "other", "technical", "billing", "shipping", "account"],
"value": "returns"
}
Besides the selected option, you get the probability of each option and the options ranked from the most to the least likely. That tells you how clear the case is. Here, the model is quite sure. When the top two options are close, you might want a person to handle the ticket instead.
Score Questions: Detecting the Customer's Sentiment
How does the customer feel? We could treat this as a choice, but sentiments have a natural order, going from the most negative to the most positive. That's what a ScoreQuestion is for.
@GetMapping("/sentiment")
ScoreAnswer sentiment(String text) {
return decisionModel.call(text, Map.of("sentiment", ScoreQuestion.builder()
.text("How does the customer feel?")
.level("angry")
.level("frustrated")
.level("neutral")
.level("happy")
.build()))
.getScore("sentiment");
}
Levels are numbered by position, starting from 0. So angry is level 0 and happy is level 3.
http :8080/tickets/sentiment text=="This is the second time my delivery is late. I'm disappointed, please look into it." -b
{
"confidence": 0.78,
"nearestLevel": 1,
"probabilities": [0.09, 0.78, 0.13, 0.0007],
"roundedLevel": 1,
"value": 1.04
}
The nearestLevel is the level the model finds most likely, in this case frustrated, and it's the one to act on. The value is the average level weighted by the probabilities, which is useful for tracking trends over time or for sorting tickets.
Multiple Questions: Triaging a Support Ticket
So far, we've asked one question at a time. In a real support desk, you'd want to know all of this at once when a new ticket comes in. A single call to the model can answer several questions of different types.
A ticket is also more than a free text: it has a subject, a body, and a customer. So, instead of a text, we'll use a set of named fields as the input.
@GetMapping("/triage")
TriageResult triage(String subject, String body, @RequestParam(defaultValue = "individual") String customer) {
var ticket = new LinkedHashMap<String, Object>();
ticket.put("subject", subject);
ticket.put("body", body);
ticket.put("customer", customer);
var categoryQuestion = ChoiceQuestion.builder()
.text("Which team should handle this support ticket?")
.option("billing", "Payments, invoices, refunds already issued, and duplicate charges")
.option("shipping", "Late, lost, or damaged deliveries, and changes to a delivery address")
.option("returns", "Sending an instrument back, or asking for a replacement or a refund")
.option("technical", "An instrument or piece of gear that doesn't work, or help with setting it up")
.option("account", "The customer's account on the store's website: logging in, resetting its password, and managing personal details")
.option("other", "None of the above")
.build();
var sentimentQuestion = ScoreQuestion.builder()
.text("How does the customer feel?")
.level("angry")
.level("frustrated")
.level("neutral")
.level("happy")
.build();
var urgencyQuestion = ScoreQuestion.builder()
.text("How urgently does this ticket need a reply, given its content and the type of customer?")
.level("low", "A general question that can wait a few days")
.level("normal", "A problem that should be handled within a day")
.level("high", "The customer is blocked right now, especially when they are a music school")
.build();
var response = decisionModel.call(DecisionPrompt.builder()
.fields(ticket)
.question("category", categoryQuestion)
.question("sentiment", sentimentQuestion)
.question("urgency", urgencyQuestion)
.build());
var category = response.getChoice("category").getValue();
var sentiment = sentimentQuestion.getLevelLabels().get(response.getScore("sentiment").getNearestLevel());
var urgency = urgencyQuestion.getLevelLabels().get(response.getScore("urgency").getNearestLevel());
var queue = "angry".equals(sentiment) ? "escalated" : category;
return new TriageResult(queue, category, sentiment, urgency);
}
record TriageResult(String queue, String category, String sentiment, String urgency) {}
Here's the part I like the most: the model answers the questions, and the application makes the decision in plain Java. Tickets from angry customers are escalated, and everything else goes to the team in charge of the category. No prompt can change that rule.
LinkedHashMap for the fields. The same goes for the questions: Map.of() has no order, so add the questions one by one to the builder when you ask more than one.Let's try it with an unhappy music school.
http :8080/tickets/triage \
subject=="Duplicate charge, cancelling" \
body=="You charged our music school twice for order 48213 and nobody replies. We are closing our account and moving to another supplier." \
customer==music-school -b
{
"category": "billing",
"queue": "escalated",
"sentiment": "angry",
"urgency": "high"
}
And now with a customer who just needs some help.
http :8080/tickets/triage \
subject=="Amplifier cuts out" \
body=="My new guitar amplifier keeps cutting out every few minutes. How can I fix it?" -b
{
"category": "technical",
"queue": "technical",
"sentiment": "frustrated",
"urgency": "normal"
}
Guardrails for Chat Models
Decision models don't replace chat models. They work great together! Our store also needs an assistant that answers customer questions, and that's a job for a chat model. We can use a decision model to guard it, checking both what goes into the chat model and what comes out of it.
In Spring AI, the natural place for that logic is the Advisor API, which lets you intercept a ChatClient call before and after it reaches the model.
Configuring the Chat Client
First, let's add a chat model to the application.yml file. I'll use granite4.2:3b, an Apache 2.0 open-source model from the IBM Granite family. Remember to remove the chat.include: false property we added earlier, so that Spring AI can pull the model at startup.
spring:
ai:
ollama:
init:
pull-model-strategy: when_missing
embedding:
include: false
chat:
model: granite4.2:3b
temperature: 0.0
decision:
model: d1-omni:600m
Next, let's create a SupportChatController class. The assistant can only rely on the store policy to answer questions. We register two advisors, which we'll implement in a moment.
@RestController
class SupportChatController {
private static final String SYSTEM_PROMPT = """
You are the customer support assistant of an online musical instrument store.
Answer the question using only the store policy below.
If the policy doesn't cover the question, say that you don't have that information.
Store policy:
- Standard shipping takes 3 to 5 business days and is free for orders over 50 euros.
- Instruments and accessories can be returned within 30 days of delivery, in their original packaging.
- Refunds are issued to the original payment method within 10 business days.
""";
private final ChatClient chatClient;
SupportChatController(ChatClient.Builder chatClientBuilder, DecisionModel decisionModel) {
this.chatClient = chatClientBuilder
.defaultSystem(SYSTEM_PROMPT)
.defaultAdvisors(
new DecisionInputGuardrailAdvisor(decisionModel),
new DecisionOutputGuardrailAdvisor(decisionModel))
.build();
}
@PostMapping("/support/chat")
String chat(@RequestBody String question) {
return chatClient.prompt()
.user(question)
.call()
.content();
}
}
Input Guardrail: Blocking Off-Topic Messages and Sensitive Data
Why spend tokens on a lasagna recipe? And why should a credit card number ever reach a chat model? The input guardrail asks both questions about the user message, in a single call to the decision model, before the chat model gets involved. If the message fails either check, the chat model isn't called at all, and the user gets a default response instead.
@Override
public ChatClientResponse adviseCall(ChatClientRequest chatClientRequest, CallAdvisorChain chain) {
String userMessage = chatClientRequest.prompt().getUserMessage().getText();
if (!StringUtils.hasText(userMessage)) {
return block(chatClientRequest, OFF_TOPIC_RESPONSE);
}
DecisionResponse decision = decisionModel.call(DecisionPrompt.builder()
.text(userMessage)
.question("sensitiveData", YesNoQuestion.builder()
.text("Does this message contain sensitive data that must not be shared in a chat?")
.whenYes("It contains a credit card number, a CVV, a social security number, or a bank account number")
.whenNo("It contains none of those. Order numbers and instrument names are not sensitive")
.build())
.question("onTopic", YesNoQuestion.builder()
.text("Is this message something the customer support of an online musical instrument store can help with?")
.whenYes("It asks about buying or owning an instrument or an accessory, or about orders, shipping, returns, billing, or the customer's account")
.whenNo("It is about anything else, such as cooking, travel, homework, music history, or general knowledge")
.build())
.build());
if (decision.isRefused("sensitiveData") || decision.getYesNo("sensitiveData").isYes()) {
return block(chatClientRequest, SENSITIVE_DATA_RESPONSE);
}
if (decision.isRefused("onTopic") || !decision.getYesNo("onTopic").isYes()) {
return block(chatClientRequest, OFF_TOPIC_RESPONSE);
}
return chain.nextCall(chatClientRequest);
}
Calling chain.nextCall() passes the request on to the next advisor and, eventually, to the chat model. Otherwise, the block() method returns a ChatClientResponse with the default response, without calling the model.
Some providers can refuse to answer a specific question. Ollama doesn't, but other integrations might. A guardrail should fail safely, so a refusal blocks the message too.
Let's see it in action. Since we need to send the question in the request body, we'll use the --raw option of httpie.
http --raw "What's the recipe for lasagna?" :8080/support/chat -b
I can only help with questions about our store, your orders, and our instruments.
http --raw "Please refund order 48213 to my credit card 4111 1111 1111 1111, CVV 123." :8080/support/chat -b
For your safety, please don't share payment details or personal ID numbers in this chat. Remove them and ask your question again.
Output Guardrail: Was the Question Answered?
A reply can also be the problem. What if the store policy doesn't cover what the customer is asking? We told the chat model to say it doesn't have that information, but that's a frustrating place to leave a customer. Let's hand the question off to a human agent instead.
The output guardrail runs after the chat model. It passes the conversation to the decision model, as a List<Message>, including the reply from the chat model. If the question wasn't answered, it replaces the reply.
@Override
public ChatClientResponse adviseCall(ChatClientRequest chatClientRequest, CallAdvisorChain chain) {
ChatClientResponse chatClientResponse = chain.nextCall(chatClientRequest);
ChatResponse chatResponse = chatClientResponse.chatResponse();
if (chatResponse == null || chatResponse.getResult() == null) {
return chatClientResponse;
}
List<Message> conversation = new ArrayList<>(chatClientRequest.prompt().getInstructions().stream()
.filter(message -> message.getMessageType() != MessageType.SYSTEM)
.toList());
conversation.add(chatResponse.getResult().getOutput());
DecisionResponse decision = decisionModel.call(DecisionPrompt.builder()
.conversation(conversation)
.question("answered", YesNoQuestion.builder()
.text("Does the last assistant message answer the user's question?")
.whenYes("It gives the information the user asked for")
.whenNo("It dodges the question, refuses to answer, says the information isn't available, or talks about something else")
.build())
.build());
if (decision.isRefused("answered") || !decision.getYesNo("answered").isYes(ANSWERED_THRESHOLD)) {
return chatClientResponse.mutate()
.chatResponse(ChatResponse.builder()
.generations(List.of(new Generation(AssistantMessage.builder().content(HANDOFF_RESPONSE).build())))
.build())
.build();
}
return chatClientResponse;
}
Two details are worth a closer look, and both come from testing this guardrail on sample conversations.
First, the code leaves out the system prompt. You should only send what the decision needs, and that was certainly the case here: when I included the system prompt, the probability of "I don't have that information." being an answer went from 0.02 to 0.54. Weird...
Second, the threshold is 0.35 rather than the default 0.5. In my sample conversations, all the replies admitting they couldn't help scored 0.27 or lower, whereas all the correct answers scored 0.43 or higher. It's a value that works for this model, this question, and these conversations, not a recommendation. In fact, an earlier version of this example, built around a different kind of store (an Italian grocery store, in case you were wondering), needed a different threshold.
The order of the advisors matters as well. The input guardrail has order 0 and the output guardrail has order 1, so the input guardrail runs first and the output guardrail gets the last word.
Let's verify it works. The store policy covers the following question, so the customer gets the answer from the chat model.
http --raw "How long do I have to return an instrument?" :8080/support/chat -b
You have 30 days from the date of delivery to return an instrument (and accessories), provided they are returned in their original packaging.
The next question is about our store, so it passes the input guardrail. But the store policy doesn't cover it, so the chat model can't answer, and the output guardrail hands it off.
http --raw "Do you ship to Iceland?" :8080/support/chat -b
I'm sorry, I couldn't find an answer to your question. I'm passing it on to one of our support agents, who will get back to you.
Conclusion
In this article, I showed you how to use decision models in a Java application with Spring AI, using the new DecisionModel API available in the 2.1 release.
Using a customer support desk as an example, I covered the three types of question (yes/no, choice, and score), how to ask several questions at once, and how to act on the probabilities with thresholds. Finally, I showed you how to combine decision models and chat models by building input and output guardrails with the Advisor API.
The main benefit of this approach is that many tasks we've been handing to chat models become typed method calls: you ask a question, you get an answer with a probability, and your application decides what to do with it, in plain Java.
In an upcoming article, I'll talk about how to use decision models to improve your agentic workflows when using retrieval augmented generation (RAG) and tool calling. Stay tuned!
Are you using Spring AI? Have you tried decision models yet? I'd love to hear from you! Share your experience on Bluesky or LinkedIn.
First version published on November 6th, 2026.
Cover picture from Pexels.