Voice Assistant · Full-Stack

Iris AI

← Back to Applications

Overview

Iris is a full-stack voice assistant you talk to through a Flutter app. She listens, thinks, speaks back, remembers you across sessions, and calls out to a wide range of tools to actually do things on your machine and in the world.

The voice loop runs Silero VAD on-device to detect speech without a cloud round-trip, streams audio over WebSocket to a Node/Fastify backend for speech-to-text, drives the LLM (with tool calling), and pipes the reply through text-to-speech — with responses token-streamed back so you see and hear Iris answering as she generates.

Memory is persistent: long-term facts (name, job, location, preferences) are auto-extracted from chat, and a per-user knowledge graph of entities, relationships, and observations lives in MySQL for recall and search. The same backend runs locally with no auth (full filesystem/Docker access) or in the cloud on Railway with Supabase auth and a restricted toolset.

Video Demo

Iris demo

Tech Stack

FlutterNode.jsFastifyWebSocketMySQLGeminiOllamaWhisperKokoro TTSDockerRailway

Key Features

  • On-device Silero VAD + streaming STT/LLM/TTS voice loop
  • Persistent long-term memory and a per-user knowledge graph
  • Tool calling chained up to 5 rounds (list → query → answer in one turn)
  • Smart-home and Windows OS control (lights, processes)
  • Sandboxed filesystem and Docker container management (local mode)
  • Web search, page fetching, and crawling for research
  • Image generation via Stable Diffusion / ComfyUI
  • Read-only multi-database MySQL querying
  • Optional 3D VRM avatar that reacts to listening/thinking/speaking
  • Runtime LLM switching across local and cloud Ollama models