AI Development: Block Trains Voice Model on Audio Alone
The company behind Square and Cash App says its new system, named Dessa, was built without any written text to better capture human emotion and timing.
By Emmanuel Obasi ·
Jack Dorsey’s Block has introduced an artificial intelligence system that processes audio without converting it to text first. The model, named Dessa, was trained exclusively on human conversation recordings. This approach distinguishes it from standard language models that rely heavily on written data to learn grammar and context.
Ragavan Thurairatnam, who leads the project at Block, shared initial audio samples to demonstrate the system’s capabilities. In one exchange, the model initially mistakes the speaker for an individual named Ronaldo before correcting its course. Other examples on the Dessa website show the model drifting into tangents about Bryan Singer’s Superman films while discussing Christopher Nolan’s work. One clip features a user describing a surprise trip to Paris, prompting a response that the team describes as teary gratitude. When paired with OpenAI’s GPT-Live-1 voice model, the interactions appear more fluid.
The development team states that Dessa is a true speech-to-speech model, meaning all its knowledge derived directly from audio inputs. They note the system can run locally on a MacBook Pro M4 Max or higher-end hardware. This local capability suggests the model is designed for devices with strong processing power, rather than relying entirely on cloud servers.
Block outlines three primary reasons for creating the model. The first is curiosity, drawing a parallel to how infants learn to speak without reading. The developers argue that current voice assistants flatten natural speech into text and fail to match human timing. The third motivation is safety, specifically regarding emotional context. The team claims existing AI systems are nearly deaf to the emotion in human voices, potentially missing cues when a user is in pain or distress. They describe Dessa as a step toward addressing that gap.
Block plans to provide early access to a small group of AI safety researchers. The company has not announced a timeline for a wider public release. The initial release focuses on demonstrating that a model can learn language solely through listening, a task the developers acknowledge is very difficult.
The project reflects a shift in how some tech companies approach voice interfaces. By skipping the text translation step, the model aims to preserve the nuances of tone and pacing that are often lost in traditional pipelines. The team emphasizes that speech is the most natural interface to intelligence, and their design attempts to honor that directly.
Source: The Verge
