D2 · The Responses API and Model Selection

This is the heaviest domain on the OAI-API mock alongside agentic systems – roughly 11 of 60 items (18%). It tests whether you can drive the Responses API fluently: build a request, read the output items, keep conversation state, stream, run work in the background, get structured JSON, call functions, set reasoning effort, pass files, manage long context with compaction, count tokens, and pick the cheapest model that meets the bar. The Responses API is the primary interface for new work; Chat Completions is legacy.

What you need to know

The Responses API takes an input (a string or a list of typed items) and returns a response whose output is an ordered list of items – messages, function_calls, reasoning, and tool results. State is either stateless (you resend history) or server-held (store: true plus previous_response_id). You choose a reasoning effort per request and follow the rule use the lowest effort that gets the result. You get machine-readable output with structured outputs (JSON Schema), extend the model with function calling and hosted tools, run slow work in background mode, and keep long conversations inside the context window with compaction. Model selection is a four-way choice – Astra, Sol, Terra, Luna – driven by task difficulty, volume and cost.

Learning objectives

By the end of this page you should be able to:

1. Construct a Responses API request and parse the output item list.
2. Manage conversation state statelessly and with previous_response_id.
3. Use streaming and background mode appropriately.
4. Produce structured outputs and handle function calling end to end.
5. Set reasoning effort by the lowest-effort-that-works rule.
6. Select a model from Astra / Sol / Terra / Luna using price, context and difficulty.

2.1 The request and response shape

A minimal call names a model and an input and reads output_text.

from openai import OpenAI
client = OpenAI()

resp = client.responses.create(
    model="gpt-5.6-terra",
    input="Summarise this ticket in one sentence: ...",
)
print(resp.output_text)  # convenience accessor for the text

import OpenAI from 'openai';
const client = new OpenAI();

const resp = await client.responses.create({
  model: 'gpt-5.6-terra',
  input: 'Summarise this ticket in one sentence: ...',
});
console.log(resp.output_text);

The full response carries an ordered output array. Do not assume item zero is your text – reasoning items, tool calls and messages can interleave.

resp.output = [
  { type: "reasoning", ... },            # may appear for reasoning models
  { type: "function_call", name, args }, # if the model called a tool
  { type: "message", role: "assistant",
    content: [ { type: "output_text", text: "..." } ] }
]

2.2 Conversation state

You have two ways to keep a multi-turn conversation, and the choice has cost and privacy consequences.

first = client.responses.create(
    model="gpt-5.6-terra", input="My name is Sam.", store=True,
)
second = client.responses.create(
    model="gpt-5.6-terra",
    input="What did I say my name was?",
    previous_response_id=first.id,
)

2.3 Streaming and background mode

                  latency need
                        │
       ┌────────────────┼─────────────────┐
       ▼                ▼                 ▼
 User is waiting   Long task, user    Fire-and-forget,
 for tokens        can wait a bit      hours-long
       │                │                 │
   stream=True     stream, or         background=True
   (token-by-      background +        + poll or webhook
    token UI)      webhook             for completion

• Streaming (stream=True) emits server-sent events so a UI shows tokens as they arrive – use it whenever a human waits.
• Background mode (background=True) starts the response and returns immediately; you poll the response ID or receive a webhook when it finishes. Use it for long reasoning or agentic runs that outlive an HTTP request.

2.4 Structured outputs

When another system consumes the output, force a schema rather than parsing prose.

schema = {
    "type": "object",
    "properties": {
        "queue": {"type": "string", "enum": ["billing", "tech", "sales"]},
        "priority": {"type": "integer", "minimum": 1, "maximum": 4},
    },
    "required": ["queue", "priority"],
    "additionalProperties": False,
}

resp = client.responses.create(
    model="gpt-5.6-luna",
    input="Route this ticket: 'I was charged twice this month.'",
    text={"format": {"type": "json_schema", "name": "routing",
                     "schema": schema, "strict": True}},
)

strict: True guarantees the output validates against the schema – you no longer defensively parse. This is the right pattern for classification, extraction and any machine-consumed result.

2.5 Function calling

Function calling lets the model request that your code run, then continue with the result. The loop is: model emits a function_call → you execute it → you return a function_call_output referencing the call_id.

tools = [{
    "type": "function",
    "name": "get_weather",
    "description": "Get current weather for a city.",
    "parameters": {
        "type": "object",
        "properties": {"city": {"type": "string"}},
        "required": ["city"], "additionalProperties": False,
    },
}]

resp = client.responses.create(
    model="gpt-5.6-terra", input="What's the weather in Oslo?", tools=tools,
)

call = next(o for o in resp.output if o.type == "function_call")
result = get_weather(**json.loads(call.arguments))  # your code runs

final = client.responses.create(
    model="gpt-5.6-terra",
    previous_response_id=resp.id,
    input=[{"type": "function_call_output",
            "call_id": call.call_id, "output": json.dumps(result)}],
)

2.6 Reasoning effort and the lowest-effort rule

The GPT-5.6 family exposes reasoning efforts from none to max; Astra adds xhigh. Higher effort means more internal reasoning tokens – better on hard problems, but slower and more expensive (you pay for reasoning tokens as output).

The rule from the docs: use the lowest reasoning effort that gets the result. There is no exact mapping from old GPT-5.5 efforts to GPT-5.6 – re-tune per task with your eval, do not assume.

2.7 File inputs, compaction and token counting

• File inputs: pass PDFs and images as input items (file uploads or base64/URL for images) so the model reads them directly; you do not paste raw text you already have as a file.
• Compaction: long conversations eventually approach the 1.05M-token window. Compaction summarises earlier turns so the thread continues without losing the plot; the Agents API does this automatically, and you can do it yourself in a Responses loop by replacing old turns with a summary.
• Token counting: every response reports usage (input_tokens, output_tokens, total_tokens). Use it to track cost and to know how close you are to the context limit.

Context budget (1.05M window)
┌───────────────────────────────────────────────────────────┐
│ system + tools │ retrieved context │ history │ headroom     │
└───────────────────────────────────────────────────────────┘
   as history grows → compact old turns → keep headroom for output

2.8 The model line-up (September 2026)

All four latest models: text + image input, text output, multilingual, vision, 1.05M context, 128K max output, served through the Responses API.

Previous generation still available: gpt-5.5 ($5 / $30) and gpt-5.5-pro ($30 / $180).

Selection guidance, from the docs: Astra for the hardest end-to-end work (sustained reasoning, judgment, multi-tool); Sol for complex, open-ended, high-value work; Terra the pragmatic all-rounder and natural replacement for GPT-5.5 workloads; Luna for clear, repeatable, high-volume tasks (extraction, classification, transformation, structured summaries).

Decision framework

Use the LADDER for model-and-effort selection: start low, climb only when an eval proves you must.

The habit the course prizes: change one thing, then re-measure. Jumping straight to Astra at max is the anti-pattern this framework exists to prevent.

Common mistakes

Scenario challenge

Scenario. Your team ships a customer-support assistant. It must (a) answer FAQs instantly in a chat widget, (b) look up the customer’s current plan from an internal billing service, (c) occasionally handle a gnarly multi-account billing dispute that needs careful reasoning, and (d) return a structured {intent, plan, next_action} object to the surrounding app. A junior engineer proposes: “Use gpt-6-astra at max for everything, resend the full transcript each turn, and parse the JSON out of the reply text.”

Expert reasoning trace.

1. Split the workload by difficulty. Most turns are FAQs and lookups – well-defined, high-volume work that gpt-5.6-luna or gpt-5.6-terra handles at low/medium effort. Only the rare multi-account dispute needs escalation. One-model-fits-all at max burns roughly 50× the input price for the common path.
2. Live plan data is a function call, not model knowledge. The billing lookup must be a function_call to the internal service; recalling it or fine-tuning it in would be stale and unverifiable.
3. Structured output, not prose parsing. The app consumes {intent, plan, next_action}, so use text.format with a JSON Schema and strict: true. Parsing prose is fragile and the junior’s plan would break the first time the model adds a friendly sentence.
4. State: don’t resend everything. For a chat widget, store: true + previous_response_id keeps input tokens down; for long threads, compact older turns.
5. Latency: stream the FAQ answers so the widget feels instant; use high effort only on the dispute path, ideally in background mode if the user can wait.
6. Escalation is eval-driven. Route the dispute path to Sol (or Astra at high) only after an eval shows Terra fails it – LADDER, not default.

Exam-correct decision: route common turns to Luna/Terra at low effort with streaming, use function calling for the billing lookup, force structured output with a schema, keep state server-side, and escalate the rare hard case up the LADDER on eval evidence. Not Astra-at-max-for-everything, not prose JSON parsing, not full-transcript resends.

Assessment traps

Practice questions

Each item states how many responses to select. Commit before revealing.

Key takeaways

• The Responses API is the primary interface for new work; iterate the output item list rather than assuming index zero.
• Keep state statelessly (resend) or server-side (store + previous_response_id); compact long threads.
• Stream for waiting users; use background mode for work that outlives a request.
• Force machine-consumed results with structured outputs and strict: true.
• Access live and private data with function calling, not fine-tuning or pasted dumps.
• Use the lowest reasoning effort that works; there is no exact GPT-5.5 → GPT-5.6 mapping.
• Climb the LADDER for model choice – Luna → Terra → dial effort → Sol → Astra – re-measuring with an eval after each change.