GenUI & Agentic UI in Flutter: The A-Z Integration Guide

GenUI & Agentic UI in Flutter: The A-Z Integration Guide

Moving beyond static Server-Driven UI. How to integrate LLMs, prompt for strict JSON schemas, and stream dynamically generated interfaces directly into your Flutter app.

Flutter Solutions Architecture • Advanced Implementation Guide • 15 min read

Table of Contents

• 1. The Shift: SDUI vs. GenUI
• 2. The Agentic Architecture
• 3. Prompting the LLM (Structured Outputs)
• 4. Building the Flutter Parser Engine
• 5. Streaming & Handling Latency
• 6. Action Bindings & State (Agentic Loop)
• 7. Fallbacks against Hallucinations

1. The Shift: SDUI vs. GenUI

If you've built Server-Driven UI (SDUI), you know the drill: your backend API returns a pre-defined JSON payload, and your Flutter app recursively maps that JSON into a tree of widgets. It allows you to update layouts without App Store reviews.

GenUI (Generative UI) and A2UI (Agentic UI) flip this paradigm. The backend does not send a hardcoded schema. Instead, an LLM (Large Language Model) evaluates the user's intent in real-time, generates a custom JSON layout tailored specifically to that moment, and sends it to the client.

Feature	Traditional SDUI	Agentic UI (GenUI)
Schema Author	Human developers (via backend code/CMS).	LLM (dynamically at runtime).
Use Case	Marketing banners, home feeds, structured A/B tests.	Personalized search results, conversational agents, custom data visualization.
Latency	Milliseconds (Standard API fetch).	Seconds (Requires streaming UI to hide LLM generation time).
Predictability	100% predictable.	Requires strict schema enforcement to prevent hallucinated widgets.

2. The Agentic Architecture Pipeline

Integrating GenUI requires a multi-step pipeline. The Flutter client acts as a "dumb browser," while the LLM acts as the rendering engine orchestrator.

💬

User Intent

"Show me a chart of my spending this week."

➔

🧠

LLM + RAG

Fetches DB data, decides a BarChart widget is best.

➔

📱

Flutter Engine

Parses JSON stream, renders BarChart dynamically.

3. Prompting the LLM (Structured Outputs)

The biggest risk in GenUI is the LLM hallucinating a JSON structure your Flutter app doesn't understand. To fix this, we rely on Structured Outputs (available in OpenAI, Google Gemini, and Anthropic APIs).

You must provide the LLM with a strict JSON schema that maps 1:1 with your Flutter widget dictionary. You instruct the LLM: "You are a UI engine. Reply ONLY using the widget components defined in this schema."

System Prompt / Function Tool Definition (JSON Schema)

// This is passed to the LLM (e.g., via Gemini's response_schema or OpenAI's tool definitions)
{
  "name": "generate_ui",
  "description": "Generates the UI based on user intent.",
  "parameters": {
    "type": "object",
    "properties": {
      "type": { "type": "string", "enum": ["Container", "Column", "Text", "Button", "BarChart"] },
      "data": { "type": "string", "description": "Text content if type is Text" },
      "action": { "type": "string", "description": "Intent to fire on tap, e.g., 'CONFIRM_PAYMENT'" },
      "children": {
        "type": "array",
        "items": { "$ref": "#" } // Recursive reference for nested UI
      }
    },
    "required": ["type"]
  }
}

4. Building the Flutter Parser Engine

Once the LLM yields the JSON, your Flutter app needs a robust recursive parser. The engine takes the dynamic Map<String, dynamic> and transforms it into real Flutter widgets.

Here is the core architectural pattern for a GenUI parser in Dart. Note the safety mechanisms in place.

lib/genui/widget_factory.dart

Widget buildWidget(Map<String, dynamic>? node) {
  if (node == null || !node.containsKey('type')) {
    return const SizedBox.shrink(); // Fallback for malformed nodes
  }

  String type = node['type'];
  List<Widget> children = [];
  
  if (node.containsKey('children')) {
    for (var child in node['children']) {
      children.add(buildWidget(child)); // Recursive parsing
    }
  }

  switch (type) {
    case 'Column':
      return Column(children: children);
    case 'Text':
      return Text(node['data'] ?? '');
    case 'Button':
      return ElevatedButton(
        onPressed: () => ActionHandler.execute(node['action']),
        child: Text(node['data'] ?? 'Click'),
      );
    default:
      // CRITICAL: LLM hallucinated a widget type! 
      Logger.warn('Unknown component type from LLM: $type');
      return const SizedBox.shrink();
  }
}

5. Streaming & Handling Latency

LLMs are slow. Waiting 3–5 seconds for the entire JSON payload to arrive before rendering results in terrible UX. You must parse the JSON as it streams in.

In Flutter, this requires combining a chunked HTTP response (or Server-Sent Events / WebSockets) with a streaming JSON parser (like the Dart `json_stream` package or a custom chunk compiler). As partial nodes arrive, your recursive widget builder should render "Skeleton" or "Shimmer" blocks for incomplete children, updating dynamically as the `StreamBuilder` yields new frames.

6. Action Bindings & State (The Agentic Loop)

A UI is only "Agentic" if it can act. The LLM doesn't just generate view components; it binds actions to them. When a user taps a dynamically generated button, it must trigger a local system intent.

✦ The Action Registry Pattern

Do not attempt to send executable Dart code from the LLM. Instead, the Flutter app maintains a registry of safe, native capabilities (e.g., "intent": "OPEN_CAMERA", "intent": "FETCH_ACCOUNT_BALANCE"). The LLM assigns these intent strings to buttons. When tapped, the Flutter client maps the string to the native function.

The Loop: User taps button → Flutter executes local action → Flutter appends the result of that action to the conversational context → Sends updated context back to the LLM → LLM generates updated UI.

7. Fallbacks against Hallucinations

Even with structured outputs, LLMs can fail. They might omit required parameters, nest components too deeply (causing Flutter layout exceptions like unbounded height in a Column), or invent widget names.

• The default switch case: Never throw an exception on an unknown widget type. Render an empty `SizedBox` and report telemetry.
• Layout Constraints: Wrap the root of your GenUI renderer in safe boundaries (e.g., `SliverToBoxAdapter` or rigid `Container` limits) so a hallucinated infinite list doesn't crash the host view.
• Timeout Fallbacks: If the LLM API takes longer than 4 seconds, abort and render a native Flutter fallback screen (e.g., a standard search bar or static menu).