Ghost Vessel Gives a Local AI Agent a Face Instead of a Chat Box
A developer swapped his local LLM chat interface for a video-call-style avatar. Ghost Vessel uses pre-rendered clips and emotion tags to stay GPU-free, and is open source.
Tired of staring at a Telegram tab to talk to his self-hosted Hermes-based LLM agent, a developer built Ghost Vessel, a monitor-resident, video-call-style avatar client. Instead of printing raw text, the system runs every reply through an output contract that splits it into three planes: dialogue for spoken text-to-speech, data for code/logs rendered as chat cards, and action for inline emotion tags like [working], [confirm], and [happy] that drive the avatar's facial expressions and human-in-the-loop approvals.
The most notable engineering choice is that the avatar doesn't rely on real-time inference at all — it blends between roughly 30 pre-rendered video clips selected by emotion tags, keeping the GPU fully dedicated to the actual language model. The tradeoff is giving up precise lip-sync, which the developer argues doesn't matter much since an expressive face and talking-mouth loop already sell the illusion of a video call. Agent integration works by having the app register as just another connector to the agent's gateway, making the avatar indistinguishable from a messaging channel like Telegram from the agent's perspective.
The project also includes a mood/affinity system that shifts over time, two-way voice interaction via TTS and speech recognition, and always-on-top windows built with Tauri v2. The core engine is MIT-licensed and open source, letting users create their own avatar clip packs. The developer is particularly interested in feedback on the emotion-beat output contract as a general pattern for turning LLM text output into UI behavior.