Understanding Confidence Scores in AI

How Sure is an AI About Its Answer?

A confidence score is a metric that quantifies the certainty of an AI model in its own response. It's a crucial feature for building trust and reliability, as it helps users gauge whether an answer is a confident assertion or a speculative guess.

However, not all AI systems generate this score in the same way. The architecture of the AI determines whether the confidence score is an organic byproduct of its process or something that must be intentionally calculated.

Confidence Scoring by AI Type

We can broadly categorize AI systems based on how they handle confidence.

1. Agentic RAG: The "Organic" Confidence Score

Retrieval-Augmented Generation (RAG) models first search a knowledge base for relevant information and then use that information to generate an answer. This multi-step process has built-in checkpoints for measuring confidence.

• Retrieval Confidence: The initial search (often a cosine similarity search) returns documents with a relevance score. A high score means the system is confident the retrieved information is relevant.
• Generation Confidence: When the LLM generates the answer, it has an internal probability for each word (token) it produces. A sequence of high-probability words indicates the model is confident in its formulation.

2. Reasoning Agents & Fine-Tuned LLMs: The "Fabricated" Score

Reasoning agents and standard fine-tuned LLMs generate a response directly from the prompt without an explicit retrieval step. Consequently, a confidence score is not an automatic byproduct and must be fabricated.

How to Fabricate a Confidence Score

There are two primary methods to generate a confidence score for these models. The right choice depends on your requirements for determinism, cost, and latency.

Method 1: Self-Consistency (Multi-Shot, temperature > 0)

This method is powerful for complex reasoning tasks where you want to ensure the answer is robust.

The approach is to check if the model arrives at the same conclusion through different "reasoning paths":

1. Set Temperature > 0: Change temperature to a moderate value (e.g., 0.5) to enable randomness.
2. Query Multiple Times: Ask the Agent the same question N times (e.g., 5 or 10 times).
3. Perform a Majority Vote: Count the occurrences of each unique answer. The frequency of the most common answer becomes the confidence score.

Example: You ask a classification agent a question 5 times and get 4 "A"s and 1 "B". Your confidence score for answer "A" is 80%.

This method is only effective when temperature is above 0. At temperature=0, the model's output is designed to be deterministic, making this check unnecessary and misleading.

Method 2: Token Probabilities (Single-Shot, temperature = 0)

This is the most efficient method and is ideal for deterministic systems, like a classifier running at temperature=0. It reads the model's certainty directly from a single API call.

How it works: When an LLM chooses a word (token) for its answer, it assigns it a probability. This probability is a direct measure of its confidence in that choice.

1. Enable Logprobs: In your API call, set the parameter to return log probabilities (e.g., logprobs: True for OpenAI's API).
2. Extract Probability: From the API response, find the log probability of the token(s) that represent your final answer (e.g., the category "Sales").
3. Convert to a Score: Convert the log probability into a percentage (using the formula e^logprob * 100) to get a human-readable confidence score.

Example: The model classifies a text as "Support". You check the logprobs and see the "Support" token had a probability of 0.92. Your confidence score is 92%. This method is extremely fast as it only requires one API call.

---

Summary: Key Takeaways

Understanding how an AI's confidence score is calculated is fundamental to interpreting its outputs correctly.

• RAG systems offer an organic, multi-layered measure of certainty from their retrieval and generation steps.
• For non-RAG agents, a confidence score must be fabricated. There are two primary methods:
  • Self-Consistency: Best for complex reasoning. It requires multiple API calls with temperature > 0 to check if different reasoning paths lead to the same answer.
  • Token Probabilities: Best for deterministic tasks (like classification at temperature=0). It's a fast, single-shot method that reads the model's direct certainty in its chosen answer token.
• Choosing the right method is critical. Use Token Probabilities for deterministic needs and Self-Consistency when you need to validate a robust reasoning process.

For LangGraph:

from typing import TypedDict, Annotated
from langchain_core.messages import BaseMessage, HumanMessage
from langgraph.graph.message import add_messages

# Define your state with a slot for confidence score
class AgentState(TypedDict):
    messages: Annotated[list[BaseMessage], add_messages]
    result: str
    confidence: float  # Stores the confidence score

# Example agent node function
def my_agent_node(state: AgentState) -> AgentState:
    # Your agent logic here: process input, call tools, etc.
    agent_result = "Sample agent output"  # Replace with your actual result
    # Compute your confidence score (this could be from an LLM-as-judge, classifier, etc.)
    confidence_score = 0.85  # Replace with your actual confidence logic

    # Update the state to include both result and confidence
    new_state = {
        "messages": state["messages"],
        "result": agent_result,
        "confidence": confidence_score
    }
    return new_state
# This pattern allows you to route based on confidence, retry logic, or expose confidence to the user

from langgraph.prebuilt import create_react_agent

def check_weather(location: str) -> str:
    return f"It's always sunny in {location}"

graph = create_react_agent(
    "anthropic:claude-3-7-sonnet-latest",
    tools=[check_weather],
    prompt="You are a helpful assistant",
    ai_evaluator=True,
    evaluator_model="anthropic:claude-3-7-sonnet-latest",
    evaluate_threshold=0.7,
)
# In this case, the confidence score (`eval_score`) is automatically stored in the state for each tool call if the evaluator is enabled.