Skip to content

Dia2: Streaming Conversational TTS Model

Source: Dia2

TL;DR

Dia2 is a streaming dialogue text-to-speech model developed by Nari Labs that can start generating audio as soon as the first few words are given as input, without needing the entire text. It can be conditioned on audio input for natural realtime conversations. Available in 1B and 2B parameter sizes, it generates up to 2 minutes of continuous audio (English only). The model uses the Kyutai Mimi audio codec operating at approximately 12.5 Hz frame rate. Licensed under Apache 2.0, it is intended for research and educational use. Quality and voices vary per generation unless the model is fine-tuned on a specific voice.

Streaming Architecture

Traditional TTS systems require the full text input before they can begin synthesizing speech. This creates an inherent latency that makes natural conversation feel stilted — the listener waits for the entire utterance to be processed before hearing anything.

Dia2 solves this by operating in streaming mode: it can begin generating audio as soon as the first few words are provided. As additional text arrives, the model seamlessly incorporates it into the ongoing audio generation. This eliminates the "dead air" problem and enables the kind of fluid, real-time speech that natural conversation requires.

Audio Conditioning for Dialogue

A key feature of Dia2 is its ability to be conditioned on audio input. In a dialogue context, the model can listen to the other speaker's audio (or a representation of it) and use that as context for generating its own response. This enables:

  • Natural turn-taking with appropriate timing
  • Prosodic matching to the conversational context
  • Voice adaptation based on the interaction

The audio conditioning makes Dia2 suitable for building conversational agents that don't just generate speech, but participate in genuine back-and-forth dialogue.

Technical Specifications

Model sizes: - 1B parameters: Lighter model suitable for deployment on consumer hardware - 2B parameters: Higher quality generation with greater computational requirements

Audio codec: Kyutai Mimi — a neural audio codec operating at ~12.5 Hz frame rate, providing efficient audio representation while maintaining quality

Output duration: Up to 2 minutes of continuous audio per generation

Language support: English only at launch

Quality and Voice Characteristics

The paper notes that quality and voice characteristics vary significantly between generations unless the model has been fine-tuned on a specific voice. This means:

  • Out of the box, the model produces competent but narrator-generic speech
  • Different prompts may produce noticeably different voice qualities and speaking styles
  • For consistent voice output (e.g., a branded assistant), fine-tuning on a specific voice is recommended
  • The model's quality is competitive with other open-source TTS systems, but not yet at the level of proprietary systems fine-tuned on extensive voice datasets

Licensing and Intended Use

Dia2 is released under the Apache 2.0 license, permitting commercial use, modification, and redistribution. The developers explicitly state that the model is intended for research and educational use, with the understanding that users will apply it responsibly. The open-source nature of the release enables the research community to build upon, evaluate, and improve the model.

Positioning in the TTS Landscape

Dia2 enters an increasingly competitive open-source TTS space alongside models like:

  • XTTS (Coqui) — Multi-speaker TTS with voice cloning
  • Bark (Suno) — Text-to-audio with expressive generation
  • CosyVoice (Alibaba) — Zero-shot voice cloning
  • VoiceCraft — High-quality neural codec TTS

Dia2's differentiating feature is its streaming dialogue focus — the ability to generate speech incrementally as text arrives, and to condition on audio context, making it specifically suited for realtime conversational applications rather than batch narration.

Key Takeaways

  • Dia2 is a streaming TTS model that starts generating audio from the first few words, eliminating latency in conversation
  • Can be conditioned on audio input for natural, context-aware dialogue
  • Available in 1B and 2B parameter sizes; generates up to 2 minutes of continuous audio
  • Uses Kyutai Mimi audio codec (~12.5 Hz frame rate); English only
  • Licensed Apache 2.0 for research and educational use
  • Quality and voice consistency require fine-tuning on a specific voice for production use