Beyond the Dialog Box: Productive UI, AG-UI, and Stack-Driven Application Frameworks Behind the Scenes

Most AI applications still show the model as a dialog box. That interface is simple, but hides what the agents actually do, such as scheduling steps, calling tools, and review status. Generative UI is about letting the agent drive elements of a real interaction, for example, tables, charts, forms, and progress indicators, so that the experience feels like a product, not a log of tokens.

What is Generative UI?

The CopilotKit team defines Generative UI as any user interface that is partially or fully generated by an AI agent. Instead of returning text only, an agent can call:

• respectable components such as forms and filters
• visuals such as charts and tables
• multistep flows like wizards
• status such as progress and intermediate results

The main idea is that the UI is still used by the application. The agent defines what should change, and the UI layer chooses how to render it and how to maintain the same state.

The three main patterns of Generative UI:

1. Fixed generation UI: agent selects from a fixed catalog of parts and fills props
2. Declarative generative UI: the agent returns a structured schema provider that maps to components
3. Fully generated UI: the model outputs raw markup as HTML or JSX

Most production systems today use static or descriptive forms, because they are easier to protect and test.

You can also download the file Generative UI Guide here.

But why is it needed for Devs?

The main pain point in using an agent is the communication between the model and the product. Apart from the standard approach, each team creates custom web sockets, ad event formats, and broadcast methods for tools and context.

Generative UI, along with a protocol like AG-UI, provides a consistent mental model:

• the agent backend exposes state, tool activity, and UI intent as scheduled events
• the frontend consumes those events and update components
• user interactions are converted back into formal symbols that the agent can consult

CopilotKit packages this into its SDKs with hooks, shared state, typed actions, and Generative UI helpers for React and other endpoints. This allows you to focus on agent logic and domain-specific UI instead of protocol development.

How does it affect End Users?

For end users, the difference is noticeable as soon as the workflow becomes trivial.

The data analysis assistant can display filters, metric selectors, and live charts instead of explaining plots in text. A support agent can display scheduling forms and time records instead of long descriptions of what they did. A work agent can display job queues, error badges, and retry buttons for the user to act on.

This is what CopilotKit and the AG-UI ecosystem call Agent UI, user interfaces where the agent is embedded in the product and updates the UI in real time, while users are always in control through direct interaction.

Protocol Stack, AG-UI, MCP Apps, A2UI, Open-JSON-UI

Several specifications describe how agents express the intent of the UI. The CopilotKit documentation and the AG-UI documentation summarize the three main aspects of a productive UI:

• A2UI from Google is a JSON-based declaration of Generative UI designed for live streaming and anonymous platform rendering.
• Open-JSON-UI from OpenAI, an open standard for OpenAI’s declarative Generative UI schema for structured interfaces.
• MCP applications from Anthropic and OpenAI, a Generative UI layer on top of MCP where tools can render iframe-based interactive environments

These are the upload formats. They define what UI should be rendered, for example a card, table, or form, and the associated data.

AG-UI lives in a separate layer. It is an Agent Interaction Protocol, a two-directory-driven runtime that connects any back-end agent to any frontend over transports such as server-sent events or WebSockets. AG-UI manages:

• lifecycle and message events
• state summaries and deltas
• tool work
• user actions
• loading a productive UI such as A2UI, Open-JSON-UI, or MCP Apps

MCP connects agents to tools and data, A2A connects agents to each other, A2UI or Open-JSON-UI defines declarative UI loading, MCP Applications defines iframe-based UI loading, and AG-UI moves all that between the agent and the UI.

Key Takeaways

1. Generative UI is structured UI, not just conversational: Agents abstract structured UI intent, such as forms, tables, charts, and progress, that are rendered by the application as real components, so the model controls the visuals, not just the text stream.
2. AG-UI is the runtime pipeline, A2UI and Open JSON UI and MCP applications are payloads: AG-UI handles events between the agent and the frontend, while A2UI, Open JSON UI, and MCP UI define how the UI is defined as JSON or iframe-based loading provided by the UI layer.
3. CopilotKit simulates an agent in UI-wiring: CopilotKit provides SDKs, shared state, typed actions, and Generative UI helpers so developers can build custom infrastructures for streaming state, tool functionality, and UI updates.
4. A static and declarative Productive UI is ideal for producing: Many real-world applications use static component catalogs or declarative specifications such as A2UI or Open JSON UI, which maintain security, auditing, and structural control in the host application.
5. User interactions become first-class events in the agent: Clicks, edits, and submissions are converted into structured AG-UI events, which the agent uses as input for editing and tool calls, closing the human in the control loop.

Generative UI sounds mysterious until you see it in action.

If you want to know how these ideas translate into real applications, CopilotKit is open source and actively used to build agent interfaces – from simple workflows to more complex systems. Jump into the repo and check out the patterns on GitHub. Everything is built in the open.

You can find here more learning materials for Generative UI. You can also download the file Generative UI Guide here.

Generative-UI