MMIntentChat: An LLM-Routed Multimodal Chatbot Unifying Conversational Text, DiffusionBased Image Generation, and Automated Diagram Synthesis on Zero-Cost Inference Services

Abstract

Conversational assistants are increasingly expected to answer questions, create images, and produce technical diagrams within a single dialogue, yet most deployed chatbots handle only one output modality or depend on paid proprietary endpoints. This paper presents MMIntentChat, a modular web-based chatbot that unifies three output modalities—natural-language conversation, diffusion-based text-to-image generation, and automated architecture/flowchart diagram synthesis—behind a single chat interface built entirely on freely accessible inference services. A lightweight large language model acting as a constrained JSON intent classifier routes every user turn to one of three specialized handlers: a conversational responder, a FLUX-family diffusion image generator whose outputs are subsequently captioned by a vision-language model, and a Mermaid diagram pipeline in which a codeoriented LLM emits diagram syntax that is validated against the renderer and automatically repaired on failure through one error-feedback retry. A multi-model fallback chain over four free diagram models and a render-time syntax verification step make the pipeline robust to individual model outages and malformed outputs. The complete system is evaluated live over 30 end-to-end trials spanning all three modalities, measuring intent-routing accuracy, output validity, caption/summary coverage, and permodality end-to-end latency. The results demonstrate that a carefully orchestrated ensemble of zero-cost services can deliver reliable multimodal assistance without any local GPU or commercial API subscription.