The Difference Between chat_completion and responses, and the Value of Migrating

Let us look at why codex-cli did this—perhaps beginning with issues in the official Codex repository.

And distinguish the main differences and the value of migrating: what benefits, if any, would it bring to XnneHangLab?

The Difference Between Anthropic Messages and OpenAI’s Streaming Tool Responses

And let us recall a question I left in an earlier ramble:

1
Can we investigate why the Anthropic protocol supports alternating tool tokens and chat tokens, while OpenAI can only do tools first and chat afterward, yet prompt injection seems able to make it announce the tool it will execute?
2

3
Does it have the complete tool schema when it announces this? In which LLM call does that announcement happen?

This may not be precise enough. The scenario was this:

Once we added ToolCall to a desktop-companion setting, a ToolCall not only brought an extra LLM call, but often took a while to execute. The tool result then had to be sent back to the LLM for a reply. Perceptually, that added five or six seconds of response latency, and the model appeared unusually silent during it.

At the time, I added a policy-level prompt-injection plugin: pre_tool_preview.

It roughly tells the model: if you are about to execute a tool, verbally announce what you are going to do first.

Afterward, the model did indeed give short previews such as Let me take a screenshot and see what you are doing? and Let me look through your recent diary entries before doing things.

The overall effect was good. It softened the deathlike silence that had been there before.

Why Does My pre_tool_preview Work? What Are Its Limits?

I have partly forgotten the exact implementation and need to verify it again. I also need a token-level streaming token demonstration: how the tokens are ordered, whether the preview occurs before or after tool tokens, why it can preview, then execute, then answer, and how many LLM Tool Calls lie in between.

How many times can it preview? Does it support staged previews in a long tool-call chain?

If I migrate to Responses, will this feature carry over? Will Responses change the returned token stream?

My roommate also says Responses supports long connections. Is that true? Could it remove the first-token delay caused by handshakes in desktop-companion conversations?

And why did I once say Anthropic supports talking while working while OpenAI does not? What is different about their streaming token flows?

How Can an Agent Talk While Working in Engineering Terms?

We know that pre_tool_preview is probably only the most basic approach.

In engineering practice, some Agents support long ToolCall chains, such as Obsidian-yolo.

This seems to involve high-frequency ToolCalls and feedback. How are their prompts written? How do they get the model to support long chained exchanges? It appears to contradict my earliest understanding of chat_completion, because there seem to be no repeated handshakes in the middle.

When do repeated handshakes mostly happen?

Those are roughly the questions. Next, let us ask Teacher Fable or Teacher Opus. Please do not answer only from your own knowledge; many of these details need evidence. Ideally, use issues from the codex or newapi repositories and documentation from Anthropic and OpenAI.

A Summary and Another Complaint About OpenAI

Okay, then this part is clear.

OpenAI simply did not want to maintain two sets of logic in Codex (chat_completion and responses), so it unilaterally removed the chat interface for more agile development and faster iteration. It cut off the people who do not directly pay OpenAI—the users of third-party model providers—and was criticized by the community for a long time.

But it also seems internally contradictory. It removed chat_completion, then recently merged the ChatGPT and Codex clients into an architecture like Claude Desktop.

Earlier, we discussed Learning from Bad Architecture: What Claude Desktop’s Five Scheduling Systems Reveal About Separating Cowork and Code. This architecture makes maintenance costs rise geometrically; separate maintenance is much better than merging everything together. Recently, during host adaptation for memU, both Claude Desktop and Codex Desktop showed terrible adaptability because of the strange sandboxes and low shell permissions of their Cowork and Chat modes.

The first move looks like a pursuit of technical cleanliness, and the next turns the architecture into a pile of mess. (Perhaps the code is not all maintained in the official Codex repository; maybe the core is maintained separately, like Claude Code, with the desktop client separate.)

It is probably all driven by commercialization, yet I do not understand the pursuit or the decisions.

OpenAI is always like this: it likes doing two things at once instead of doing one thing well. While updating GPT models, it splits off codex models, then o1, o2, and o3; later it throws away Codex and focuses only on GPT. The naming is also chaotic, as if it names things whenever it feels like it. Versions 5.1, 5.2, 5.3, and 5.4 update at a frantic pace, but after all of that most people felt 5.2 was best. 5.4 speaks like a machine and has high information redundancy; it is messy enough that an ADHD reader can finish a long paragraph without absorbing a sentence.

Claude, by contrast, has consistently pursued mathematics, coding, and logic. Its naming and iteration are rigorous and romantic, drawn from Greek mythology. And when a name changes, it is a real major update—for example, Fable.

At present, GPT has only 5.6 sol and terra that I find usable. Below 5.6, there is confusion, disorder, and a lack of trust.

I do not understand OpenAI. I do not understand how it can play such a strong hand so badly. I do not understand Codex either. Perhaps its own employees do not, which is why everyone recommends connecting 5.6 to Claude Code for the best experience XD.

What to Watch: There Is No Migration Value Yet, Only Migration Risk

What we care about more is the value of migrating.

But it appears newapi’s native Responses support and translation are currently poor, with much technical debt. It only makes sure replies can get through without error; almost none of the real features are preserved. Instead, quite a few things are stripped away.

Most people currently cannot reach a true Responses API. In other words, continuing to use chat_completion is best for us. newapi’s support for it has matured for more than three years; there is no need to gamble on switching to responses.

Complaining Again

I think I understand.

It turns the very strange data structure of choices[0].message.content into a white box that looks like something Pydantic can validate. It also replaces the many type unknown errors I used to get while writing streaming code with native support.

When I first encountered the OpenAI library, I wondered how a PyPI package used by so many people could be full of pyright errors and require so many type:ignores. Its type enforcement was a complete mess. Looking at only a short code fragment, you often could not know what it returned because every possible kind of content was wrapped in the shell of content.

How can such a large company design data structures so carelessly?

When I supported FastMCP, the biggest obstacle came from OpenAI’s chat_completion. Its ToolCall handling was exhausting to read. Later, I removed FastMCP entirely, and it felt wonderful.

But is responses really designed well? Is its data structure actually cleanly validated? I do not believe it. I have too much distrust of OpenAI. Its past roughness, unreasonableness, and chaotic disorder make it feel like an amateur troupe.

The Confusion Around Talking While Working

I need an answer to the question above.

Why did Claude once tell me that only Anthropic Messages could truly meet my desktop companion’s need to talk while working, while OpenAI could only execute all tools and reply afterward? In practice, after injecting the pre_tool_preview plugin, I saw it give a preview before executing a tool.

I have forgotten the implementation details and may need to confirm them: XnneHangLab.

Opus

That earlier statement was half right and half wrong. Anthropic does natively talk while working, but OpenAI is not “unable to do it”—it is “unable to express multiple segments.” Your plugin did not break through the protocol; it used a field combination the protocol already permits but models do not normally choose.

Anthropic: Talking While Working Is the Default in the Official Example

The official streaming documentation gives this tool-use example verbatim:

1
content_block_start  index:0  {"type":"text"}
2
content_block_delta  index:0  "Okay" "," " let" "'s" " check" " the" " weather" ...
3
content_block_stop   index:0
4
content_block_start  index:1  {"type":"tool_use","name":"get_weather"}
5
content_block_delta  index:1  {"type":"input_json_delta","partial_json":"{\"location\":"}
6
content_block_stop   index:1
7
message_delta                 {"stop_reason":"tool_use"}

The model itself says “Okay, let me check the weather in San Francisco” first, then emits tool_use—without prompt injection. What you gained through a plugin is the first documented example on Anthropic’s side.

Its content is also an ordered array of blocks, where index is the final array position. The web-search example is even more extreme: text(0) → server_tool_use(1) → tool_result(2) → text(3) → …, continuing to index:17. Text and tools genuinely alternate in multiple segments within one message.

OpenAI: It Can Speak Once, but Cannot Express Multiple Segments

An assistant message in chat completions has content—a single string—and a tool_calls array as two parallel fields. Therefore:

• One preview plus a batch of tool calls is completely valid. Your plugin working is not a hack.
• Speak → work → speak → work cannot be expressed. There is only one content string, with nowhere to place a second segment.

There is a subtler point: ordering information is lost when the message is saved back. During streaming, you really receive delta.content before delta.tool_calls—autoregressive generation means generation order is arrival order. But once assembled, it becomes two parallel fields. In the next turn, the model sees {content: "Let me take a screenshot", tool_calls: [...]}. The structure itself no longer says “I spoke first, then acted”; it can only infer that semantically. Anthropic’s index preserves it structurally.

Your Implementation: Confirmed from the Repository

Question	Answer
Where is the injection?	type = "policy", injection_position = "before_tools", priority = 10. It goes through PolicyPlugin.get_prompt_segments() → SystemPromptBuilder.build(), into the system prompt before tool definitions.
When is it injected?	Once when creating the agent through AgentFactory.create_core_with_profile(), not as a hook every turn.
Does the preview have the full tool schema?	Yes. The prompt is in the system prompt, while tools are passed in full through stream_with_tools(tools=tools_schema) in the same API request.
Which LLM call makes the preview?	The same call that decides to call the tool, not an extra call. Preview tokens and tool_call tokens come from the same completion.
Are preview tokens before or after tool tokens?	Before. In the core.py loop, delta.content arrives first, accumulates into text_buf, and is yielded directly to the user; delta.tool_calls arrives later and accumulates in tool_calls_buf.
How many previews can it make?	One per round because there is only one content. With max_rounds = 6, it can make up to six segments.
Are staged previews in a long chain supported?	Yes, at one segment per round, not multiple segments within a single round.

The key assembly code is exactly the “one message carrying both a preview and tool_calls” combination:

1
assistant_payload = {
2
    "role": "assistant",
3
    "content": text_buf or None,   # ← preview text goes here
4
    "tool_calls": [...],
5
}

So you were already using a valid combination allowed by the protocol. Models simply do not do it by default; your prompt calls that behavior forth.

Where “OpenAI Cannot” Is Wrong and Right

	Anthropic Messages	OpenAI Chat Completions
Speak once, then act, within one round	✅ Native default	✅ Possible, but needs prompt guidance
Multiple segments within one round	✅ (server-tool example reaches index 17)	❌ Only one content
Ordering preserved structurally	✅ index	❌ Two parallel fields
Staged previews across rounds	✅	✅ (your six rounds)

The right part: Anthropic is native behavior and supports multiple segments within a round; OpenAI has neither. The wrong part: “OpenAI can only execute all tools and reply afterward.” No: one spoken preview per round is protocol-legal, and your plugin proves it.

The real gap is small in client-tool scenarios. When Anthropic encounters a client tool, it also ends with stop_reason: "tool_use", returns to the client, and produces one segment per round. The source of true multiple alternations is server tools: tools run on the server, control never returns to the client, and one message can proceed text → tool → result → text. Chat completions has no server tools, so it can never do this. That is exactly the gap Responses fills.

Additional Questions

I want to understand two things.

1. Do Anthropic Messages and OpenAI both strictly alternate in sequence? In other words, will no ToolCall token be sent before all preview message tokens are complete?
2. Why does the OpenAI protocol not support alternation? Is this a limitation of model output? Do most models have it, or is the protocol simply badly designed?

Opus

a: Yes. Both are strictly serial; “alternation” was the wrong word from the start. b: This is purely a protocol problem, not a model limitation—and the evidence is in OpenAI’s own SDK types. Also, the behavior in your example may be caused by your prompt rather than the protocol; see the final section.

a. Both Are Strictly Serial; There Is No Token-Level Interleaving

Anthropic: the official streaming documentation defines the flow strictly:

A series of content blocks, each of which has a content_block_start, one or more content_block_delta events, and a content_block_stop event.

That means block 0 must finish with content_block_stop before block 1 can start with content_block_start. In the official example, the final text_delta from “Okay, let’s check the weather…” arrives, then content_block_stop index:0, and only then does content_block_start index:1 begin the tool_use. There is no mixing of text_delta(0) and input_json_delta(1) in the stream.

OpenAI is similarly autoregressive: delta.content arrives first, followed by delta.tool_calls, also in sequence.

So the accurate statement is: both providers concatenate [text-token segment][tool-token segment] in sequence; the difference is only how many segments can be joined.

	Shapes it can join
Anthropic	[text][tool][text][tool]… (the server-tool example reaches index:17)
OpenAI chat	[text][tool]—only this one pair

“Alternation” should be renamed “segmentation.” What your original question meant by “alternating tool tokens and chat tokens” was segmentation ability, not mixed token streaming. Neither provider does the latter, and neither should; a client cannot render a half-sentence with an inserted tool call.

b. It Is the Protocol, Not the Model—the Proof Is in OpenAI’s Own Type Definitions

Evidence one: the same model can create multiple segments when the protocol changes. gpt-5.x under Responses can legally produce an output array like message → function_call → message → function_call; the same model under chat completions cannot. The model is unchanged, and capability changes with the protocol. That alone excludes model-output limitations.

Evidence two: the asymmetry in openai-python is obvious.

1
# Output (what the model sends you) — chat_completion_message.py
2
class ChatCompletionMessage(BaseModel):
3
    content: Optional[str] = None          # ← one string
4

5
# Input (what you send back) — chat_completion_assistant_message_param.py
6
ContentArrayOfContentPart: TypeAlias = Union[
7
    ChatCompletionContentPartTextParam,
8
    ChatCompletionContentPartRefusalParam,  # ← only text and refusal
9
]
10
content: Union[str, Iterable[ContentArrayOfContentPart], None]

Two things are true at once:

1. Assistant content can already be an array on the input side, while output remains a string. The array shape exists in the protocol but was never extended to model output.
2. Even that array contains only text and refusal, not a tool_call part. So even manually constructing it cannot represent an ordered “text → tool → text” sequence; tool_calls always remains another parallel field.

That is the root problem: it is not that no array exists, but that the array has no tools. The protocol contains nowhere that can preserve ordering information.

Evidence three: chat completions is the odd one out among major providers. Gemini’s Content.parts is also a mixed array, and its documentation explicitly warns:

the model may return a mix of functionCall, toolCall, and toolResponse parts in a single turn. Don’t assume the functionCall will always be the last item in the parts array.

That is effectively saying that text can occur after a functionCall. Anthropic has ordered blocks; Gemini has ordered parts; Responses has ordered items. Only chat completions uses two parallel fields.

Is It Really “Garbage Design”?

Not garbage, but unreconstructed historical baggage. content: str made perfect sense in the GPT-3.5 era, when a turn could only produce text. When function calling was added in 2023, adding a parallel tool_calls field was backward compatible; changing content from a string to an ordered array was not. It would have broken every client that read choices[0].message.content.

That locked it in. Multimodal input later received an array because it was a new field and new shape, while output remained a string forever. This is why Responses required a new endpoint rather than adding parameters to chat completions: the change could not be compatible, so the protocol had to change. It also explains why Codex had to remove support rather than softly preserving it.

Finally: Your Example May Be Your Own Prompt’s Fault

You said users lose awareness after the first preview. But across rounds, the multiple-segment restriction does not apply. The second assistant message has its own content field and can legally preview again:

1
round 1: {content: "Let me read the diary.", tool_calls: [read_diary]}
2
round 2: {content: "The diary is empty; I will check memory.", tool_calls: [read_memory]}   ← fully legal
3
round 3: {content: "I found…"}

Why did this not happen? Look at rule five of your own pre_tool_preview injection:

“If this is the first tool call in the current round, and the user can perceive the wait, prioritize this upfront explanation.”

The phrase “first tool call in the current round” may have been interpreted by the model as “only preview once in a user turn.” The behavior you observed strongly matches that prompt.

Try a control experiment first: change it to “Before every tool call, give an upfront explanation; if it repeats the previous one, use different wording.” Then see whether a second preview appears. If it does, your long-chain silence came from the prompt, not the protocol—a vastly easier problem to fix.

(And a correction to the previous section: saying “once per round, up to six segments” was correct; max_rounds = 6 gives you six-segment budget. What actually consumed one segment and then stopped was likely this prompt rule.)

Damn!

Oh, I finally understand. I could see only one preview all this time because of my own prompt!

And in a tool loop, each OpenAI protocol tool-call segment is independent and can carry a preview message?

This matters greatly to me. My later engineering goal is to eliminate silence in long tool chains for the desktop companion. I had planned to break one tool chain into multiple steps, but now it seems I may not need to!

Is that right?

So I only need to improve the prompt and add some guardrails? I do not need a complicated implementation?

Opus

The overall direction is right, but do not celebrate yet—three points need calibration, and the second one is my responsibility.

Correction One: It Is “One per Round,” Not “One per Tool”

These two lines in your core.py explain it:

1
max_rounds = 6                                    # 503
2
*(_exec_tool(tc, ...) for tc in ordered_tool_calls)   # 606, parallel execution

In one round—one LLM call—the model can emit N tool_calls, which you execute in parallel. That round has only one content field, so N tools share one preview.

The precise statement is: a preview has the granularity of “one per LLM call,” not “one per tool.”

For one-to-one previews, pass parallel_tool_calls: false to the API; it is a standard chat-completions parameter and openai-python types it. But that trades latency for awareness. If three tools each take five seconds, parallel execution takes five seconds while serial execution takes fifteen. Usually that is not worth it: one preview saying “I will check the diary and memory at the same time” is enough.

Correction Two: My Diagnosis Was a Hypothesis, Not a Conclusion

In the previous section, I said the prompt might be at fault. That was a high-confidence inference from reading the code, but I did not run your agent. Do not treat it as a verified fact—run the control experiment first.

And even after changing the prompt, there is a second obstacle: after getting a tool_result, models themselves tend to jump straight into the next tool call. How malleable that behavior is depends on the model. The ones you use on the desktop companion—DeepSeek and Grok-like models—may be more stubborn than frontier models.

So set realistic expectations: improving the prompt will probably help substantially, but may not consistently produce a preview every round. Prepare for partial success rather than treating one edit as a complete solution.

Correction Three: Previews Do Not Remove Silence During Tool Execution

This is easiest to overlook. A preview covers the instant from the LLM deciding to call a tool until the tool starts, not the three to five seconds while the tool runs.

The real result is therefore:

1
Before: [say one thing] ────30 seconds of silence──── [answer]
2
After:  [say one thing]─5s silence─[say one thing]─5s silence─[say one thing]─5s silence─[answer]

It splits one long silence into several shorter silences; it does not remove silence. The perceived improvement is huge, but a tool that itself takes ten seconds remains silent for ten seconds. Only UI-level progress feedback—such as a spinner or “Reading…”—can cover that; the protocol cannot.

Do You Still Need to Split It into Steps?

If the goal is only to remove silence, no. Your original plan to break it down would use engineering to imitate a capability the protocol already has.

But notice one hard limit: max_rounds = 6. Once a chain exceeds six rounds, your own system truncates it. If you want long tool chains to remain perceptible, raise this number first; no matter how good previews are, the current budget is only six segments.

A Concrete Prompt Shape

Do not only write “preview before calling.” Make each round’s sentence both close the previous step and preview the next. That is the form you actually want:

Before every tool call, say one sentence. If this is not the first tool call in the current round, that sentence must do two things: use half a sentence to state the previous result, then explain the next action. Do not repeat the previous wording.

The output becomes the ideal form you described:

1
Let me read the recent diary.                  ← preview only
2
[Read Diary] → empty
3
The diary is empty; I will check memory.       ← closure + preview
4
[Read Memory] → ...
5
I found...                                     ← closure

This is more natural than previewing every step and uses fewer tokens: users receive a coherent narrative rather than six repeated “Let me check…” lines.