Coze: The Low-Code Ceiling

Coze is an Agent-building platform launched by ByteDance; in 2024, it open-sourced its core engine as Coze Studio. Its design goal is clear: enable nontechnical users to build AI bots through a visual interface and quickly deploy them to platforms such as WeChat, Feishu, and Discord.

Coze’s Workflow editor uses drag-and-drop nodes to orchestrate processes. Together with its built-in plugin marketplace and knowledge bases, it really can produce a functional customer-service bot or knowledge Q&A assistant within minutes. This is where Coze genuinely excels.

But it runs into a wall in our scenario. First, Coze Workflow has an explicit limitation: it does not support an end node with streaming output within a workflow. That prevents us from segmenting and dispatching text to TTS at the token level, breaking the entire speech-synthesis stage. Second, although Coze’s plugin system supports third-party APIs, its lifecycle follows a request-response model and has no hooks such as on_before_turn or on_after_turn. There is nowhere in Coze’s architecture to attach our proactive OCR trigger or post-call memory graph construction. Finally, Coze’s open-source community edition and commercial edition differ in functionality. Features such as custom voices are available only in the commercial edition, whereas we need a fully controllable TTS pipeline.

In short, Coze was designed for “chatbots,” not “companion Agents.”

Dify: The Limits of Workflow Expressiveness

Dify is an excellent LLMOps platform whose core value lies in visual workflow orchestration and all-in-one RAG management. By dragging and dropping nodes, you can quickly build an Agent application with knowledge-base retrieval, model switching, and conditional branches. Dify’s Agent Node even supports custom strategy plugins such as ReAct, CoT, and ToT, while its Plugin Trigger mechanism can subscribe to external events that start a workflow.

At first glance, Dify’s Plugin Trigger seems capable of meeting our need for “proactive OCR triggers”: subscribe to an external OCR event and use it to start the workflow. The problem appears downstream.

Dify’s Agent Node follows a three-stage execution model: initialization, iterative loop, and final response. Each iteration is a complete LLM call. Its output is a structured JSON response rather than the token stream we need. Although Dify’s Chat API supports streaming over SSE, that stream operates at the node level: you can see the result after a node finishes, but you cannot access the LLM’s output token by token as it is generated and segment it in real time.

The more consequential limitation concerns memory. Dify’s Agent Node uses TokenBufferMemory to manage context through a sliding token window. That works well for cost control, but we need semantic retrieval from an external knowledge graph: use the current conversation to recall relevant memory fragments from Neo4j, then inject them at a specific point in the user prompt. This bidirectional memory flow—retrieval before the call and graph construction afterward—falls outside the design scope of Dify’s Agent Node.

What is Dify well suited to? Enterprise RAG applications, customer-service workflows, and content-generation pipelines—scenarios that are not sensitive to streaming granularity and whose memory requirements involve “recall within a window” rather than “cross-session graph relationships.”

LangChain: The Cost of Abstraction

LangChain was an early mover in the Agent framework space. Its greatest contribution was defining the concept of a “Chain”: a composable pipeline connecting a Prompt, an LLM, an Output Parser, and Tools. In 2023 and 2024, this abstraction dramatically lowered the barrier to building LLM applications.

LangChain’s central strengths are composability and its ecosystem. It offers the broadest collection of Integrations, with official adapters for hundreds of third-party tools and vector databases; a mature Document Loader / Text Splitter / Retriever stack for RAG; and LangSmith for Tracing and Eval. If you are building a conventional retrieval-augmented Q&A or document-analysis application, LangChain may still be the fastest place to start.

Yet LangChain’s problem lies precisely in its core abstraction.

A Chain is linear. A standard LangChain Agent works like this: receive input → construct a Prompt → call the LLM → parse its output → execute a tool if needed → call the LLM again → produce the final output. It is a one-way pipeline. Our scenario, however, requires the token stream to be handled by several consumers simultaneously while the LLM is generating: a sentence segmenter, an action extractor, a TTS filter, and a frontend display processor. Tool calls can also interrupt the text stream, insert tool-status labels, and then resume streaming after execution finishes.

This is not a chain. It is a stream with loops, branches, interruptions, and resumptions.

More concretely, LangChain’s AgentExecutor, which was marked deprecated by the end of 2025, blocks the entire chain while a tool call is executing. You have no opportunity to stream status information to the frontend during that execution. When our AgentCore encounters a tool call, it first yields a structured ToolCallEvent(status="running") event so the frontend can immediately display the tool’s status. It then executes all tool calls concurrently, yields ToolCallEvent(status="completed") when they finish, and resumes streaming generation. Throughout the process, text tokens and tool events flow out of the same AsyncIterator as distinct types for the frontend to consume separately.

LangChain’s memory abstractions, such as ConversationBufferMemory and ConversationSummaryMemory, were likewise designed for simpler scenarios. They assume memory is “some compressed form of the conversation history” that can simply be placed in the Prompt. Our memory is an independent service with its own search and write APIs. It must be queried asynchronously before a conversation begins and written to asynchronously after it ends. LangChain provides no native lifecycle hooks for this kind of external asynchronous memory service.

Finally, LangChain has too many abstraction layers. A tool call passes through Tool → ToolKit → AgentExecutor → OutputParser and several layers of wrapping. During debugging, you often need to follow five or six nested Traces in LangSmith just to locate the problem. In a real-time streaming scenario like ours, where optimization happens at the millisecond level, every abstraction layer adds latency and debugging overhead.

LangGraph: The Closest Match, Yet Still Misaligned

LangGraph is the LangChain team’s answer to the limitations of its own linear architecture. It introduces a directed graph, StateGraph, for orchestrating Agent workflows and supports complex control flow such as loops, branches, conditional edges, and human approval. Since LangChain and LangGraph both reached their 1.0 milestones in October 2025, LangChain’s create_react_agent() has in fact run on top of the LangGraph engine.

Several LangGraph design choices are genuinely worth learning from. Its concept of State—a shared state object passed between graph nodes—offers more flexible data transfer than a LangChain Chain. Time-Travel Debugging lets you return to any node in a graph execution and replay it, which is extremely useful when debugging complex workflows. Its Human-in-the-Loop mechanism lets you insert human approval on any edge in the graph.

But there is a fundamental mismatch between LangGraph’s graph model and our requirements: granularity.

A LangGraph Node represents “one complete operation”: one LLM call, one tool execution, or one decision. Nodes pass State to one another through Edges. This design is ideal for orchestrating multistep workflows such as “retrieve documents, summarize them, generate a report, and send it after human approval.”

Our core requirement, however, is not “orchestration between steps” but “token-level processing within a single LLM call.” As the LLM streams its output, every token must pass through a four-layer decorator pipeline: sentence_divider → actions_extractor → tts_filter → display_processor. This is not “one graph node finishes and passes its result to the next.” It is “the output stream is split and processed in real time inside a single node.” LangGraph’s State Graph cannot express behavior at this granularity.

Moreover, our Hook system needs “lifecycle hooks for a conversational turn,” not “hooks for graph-execution nodes.” on_before_turn retrieves memories before the entire graph begins, on_after_turn writes memories after the graph finishes, and on_after_playback is not triggered until frontend playback ends, because some post-processing must wait for TTS playback to finish. These three hooks span three entirely different timescales: the LLM call, tool execution, and frontend playback. LangGraph’s graph is designed around the “LLM + Tool” execution flow; it has no concept of “frontend playback complete.”

LangGraph also has a practical problem at scale: performance. As the number of graph nodes and edges grows, execution slows, memory usage rises, and debugging becomes harder. Its tight coupling to the LangChain ecosystem also means that if you want to use a lighter-weight LLM client, such as the openai SDK directly, you must write a great deal of glue code to adapt it.

Harness Engineering: The Right Direction, but We Do Not Need a Control Plane

The hottest concept of 2026 is Harness Engineering: “Models provide raw intelligence; harnesses make that intelligence useful.” Gartner predicts that by the end of 2026, 40% of enterprise applications will include AI Agents, while flaws in the harness layer rather than inadequate model reasoning will account for 65% of Agent project failures.

Microsoft Agent Framework (MAF) reached 1.0 GA in April 2026, unifying AutoGen and Semantic Kernel while providing production-grade capabilities such as Shell access, human approval flows, and cross-session context management. The industry is also standardizing around two protocols: MCP (Model Context Protocol) for vertical interaction between Agents and Tools, and A2A (Agent-to-Agent) for horizontal delegation.

The five layers emphasized by Harness Engineering—tool orchestration, validation loops, context and memory, guardrails, and observability—align closely with our own practice. Yet these frameworks typically assume either a “single-turn request-response” or “multistep workflow” model, with security, compliance, and observability as their central concerns. They are control planes designed for scenarios such as enterprise SRE, customer-service automation, and code generation.

What we need is not a control plane, but a “presentation layer that lets an Agent accompany you like a person”: real-time speech, facial expressions, proactive conversation, and emotional memory. None of these requirements appear on the Roadmap of any Harness Engineering framework.

Lifecycle Hooks (Hook System)

We defined three hook points that cover the full lifecycle of a conversational turn:

1
class HookPlugin(ABC):
2
    async def on_before_turn(self, user_text, ctx) -> str | None:
3
        """Before the call: retrieve memories and inject context"""
4
    async def on_after_turn(self, user_text, assistant_text, ctx) -> None:
5
        """After the call: write memories and update the graph"""
6
    async def on_after_playback(self, user_text, assistant_text, ctx) -> None:
7
        """After playback: post-process once TTS playback has finished"""

MemoryPlugin is a typical implementation. In on_before_turn, it performs a semantic search against the Memory Bench service and injects the recalled memory fragments into the context. In on_after_turn, it asynchronously writes the current conversation to mem0 for memory extraction while updating the Neo4j knowledge graph.

MoodChatPlugin implements a different pattern. Rather than responding passively, it schedules conversations proactively according to an emotion score: it talks more when happy and stays quiet when feeling low. In gaming-companion mode, it also uses OCR change detection to decide whether to speak first.

One key design decision is that hook return values are concatenated and injected into the [memory context] block of the user prompt rather than the system prompt. This avoids treating transient information as stable fact.

Streaming Tool-Calling Loop

AgentCore.run_turn() implements a multiround streaming tool-calling loop with a maximum of six iterations per turn. Its return type is AsyncIterator[str | ToolCallEvent]: text tokens flow out as strings, while tool calls flow out as structured events.

1
max_rounds = 6  # A safety guardrail, not a technical limit
2

3
for _ in range(max_rounds):
4
    text_buf = ""
5
    async for chunk in chat_llm.stream_with_tools(messages, tools=schema):
6
        if delta.content:
7
            text_buf += delta.content
8
            yield delta.content              # Emit text tokens immediately
9
        if delta.tool_calls:
10
            accumulate(tool_calls_buf)       # Accumulate tool-call fragments
11

12
    ordered_tool_calls = _ordered_complete_tool_calls(tool_calls_buf)
13
    if not _should_execute_tool_calls(finish_reason, ordered_tool_calls):
14
        break   # No tools need to run; finish normally
15

16
    # Before execution: emit running status events
17
    for tc in ordered_tool_calls:
18
        yield ToolCallEvent(
19
            tool_id=tc["id"],
20
            tool_name=tc["name"],
21
            args=tc["arguments"],
22
            status="running",
23
        )
24

25
    # Execute all tools concurrently
26
    results = await asyncio.gather(
27
        *(_exec_tool(tc, tool_manager, ctx) for tc in ordered_tool_calls)
28
    )
29

30
    # After execution: emit completed / error status events
31
    for tc_info, result in zip(ordered_tool_calls, results):
32
        yield ToolCallEvent(
33
            tool_id=tc_info["id"],
34
            tool_name=tc_info["name"],
35
            args=tc_info["arguments"],
36
            status="completed" if result.ok else "error",
37
            result=result_text,
38
        )
39

40
    # Append tool results to the message list, then let the LLM continue
41
    # generating from those results in the next round

The six-round limit is an empirical trade-off between latency and capability. Voice interactions are sensitive to response time, while nearly all normal conversations finish after one or two rounds of tool calls—search → read a file → answer. Six rounds provide ample headroom without making users feel that they are waiting too long.

Two design details are worth emphasizing.

First, tool-call events flow out as structured objects, not textual tags. ToolCallEvent is a dataclass with fields including tool_id, tool_name, args, status, and result. Neither the frontend nor any downstream pipeline layer has to recognize tool events by parsing text tags with regular expressions; each can simply check the type. Tool status is also split into two event stages: status="running" before execution and status="completed" or "error" afterward. The frontend can switch its UI according to the current status.

Second, multiple tool calls within the same round execute concurrently through asyncio.gather() rather than waiting serially.

Token-Level Output Pipeline

The mixed str | ToolCallEvent stream emitted by AgentCore enters the decorator pipeline in MemoryAgent:

1
@tts_filter(config)           # Filter content TTS does not need (special symbols, etc.)
2
@display_processor(...)        # Process control tags for frontend display
3
@actions_extractor(live2d)     # Extract emotion/action tags to drive Live2D
4
@sentence_divider(...)         # Split on natural sentences and emit each immediately
5
async def chat_with_memory(input_data):
6
    async for token in core.run_turn(...):
7
        yield token

The processing logic in every decorator layer performs one crucial check:

1
async for chunk in stream:
2
    if isinstance(chunk, (AudioOutput, ToolCallEvent)):
3
        yield chunk         # Pass structured events through unchanged
4
        continue
5
    # Otherwise, process the text token in this layer

As a structured event, ToolCallEvent passes transparently through the pipeline. The sentence segmenter does not split it, the TTS filter does not try to synthesize it, and the display processor does not attach tags to it. Whenever a layer encounters a ToolCallEvent, it simply yields the event unchanged. The final stream received by the frontend contains two kinds of messages: processed sentences, including display text, TTS text, and action tags; and structured tool-call events. This is much cleaner than the early design, now removed, that mixed <tool>[name]</tool> text tags into the conversation stream.

sentence_divider is the most latency-sensitive part of the pipeline. As soon as it has enough tokens to form a natural sentence, it yields that sentence immediately rather than waiting for the entire passage. This allows the TTS engine to begin synthesizing the first half of a response while the LLM is still generating the second, greatly reducing the delay users perceive before hearing the first sentence.

Four Plugin Types

All capabilities are extended through plugins. We defined four plugin types:

Each plugin declares its identity and default configuration in a plugin.toml file:

1
[plugin]
2
id = "memory"
3
type = "hook"
4

5
[config]
6
base_url = "http://localhost:12393"
7
user_id = "xnne"
8
agent_id = "congyin"
9
search_limit = 10

A Profile TOML file determines which plugins are enabled and which configuration values they override:

1
[plugins]
2
enabled = ["web_search_ddg", "web_fetch", "memory", "diary"]
3

4
[plugins.memory]
5
agent_id = "baoqiao"     # Switch agent_id when switching characters
6
search_limit = 5

This means the same codebase can switch to an entirely different Agent persona simply by changing the Profile file: a different character Prompt, output format, set of enabled plugins, and memory space. Two characters can coexist in the same Neo4j graph while remaining isolated from one another through agent_id.