1/32
@justLV
Excited to share a peek of what I’ve been working on
We @sesame believe voice is key to unlocking a future where computers are lifelike
Here’s an early preview you can try!
We’ll be open sourcing a model, and yes…
we’re building hardware!
https://video.twimg.com/ext_tw_video/1895150509863903233/pu/vid/avc1/720x720/LhofMwjlpaebYz9H.mp4
2/32
@justLV
We're focused on making voice feel real, natural and delightful - to become the most intuitive interface for collaborating with AI
It's not just about words, but about pacing, expressivity & cues. We’re working on full end-to-end duplex models to capture these humanlike dynamics
3/32
@justLV
The demo you can try uses our contextual TTS, using both conversation text and audio to deliver natural voice generation.
Here is a real example of this in action (that you can try), where Maya's delivery starts matching the context after a few lines.
https://video.twimg.com/ext_tw_video/1895154182820413440/pu/vid/avc1/720x720/IiHKN-vLTFK7ZWvo.mp4
4/32
@justLV
We will be open-sourcing the contextual TTS base model (w/o this character's voice fine-tuning)
This will let anyone build voice experiences locally w/o external API’s.
This is something I would have loved for previous demos and so am personally passionate about.
5/32
@justLV
Lastly...
We can do with less screens in our lives.
We’re building comfortable, all-day wearable eyewear, for the most natural way for a personal companion to see, hear and respond.
Doing this right is tough, but we’ve made solid strides - I’ll be sharing more on this soon
6/32
@justLV
We believe in the magic of combining technology and storytelling to create rich characters and delightful experiences.
Try out our preview here:
Crossing the uncanny valley of conversational voice
7/32
@GregDNeilsen
Wow, exciting stuff Justin.
Definitely agree about less screens and intrigued by the wearable eyewear concept.
Keep it up!
8/32
@justLV
Thank you!
9/32
@DrOnwude
This is great! When is the open-source model coming out?
10/32
@justLV
Thank you! 1-2 weeks. The demo is a fine-tuned version of the base model on the talent's voice that we can't release, but the base model is still extremely capable - you can get a preview of capabilities on the research blog post.
11/32
@natjjin
fwiw, her jokes did land. i love maya already @justLV
12/32
@justLV
13/32
@chinguetti1
It’s amazing. Well done.
14/32
@0xTheWay
Wow. Really great work.
15/32
@weworkremotely
Open Sesame!
16/32
@RobCoreano
I tried earlier, and it was impressive and fun. The path I’ve been imagining since Kitt, Jarvis, Vision, Ultron, etc., makes me very eager to see how your team’s work is going to evolve..
17/32
@0FJAKE
any plans for Apple Watch?
18/32
@thisissharat
Wow it’s good!!
19/32
@azed_ai
Awesome
20/32
@atgorans_k
The future is here guys
21/32
@AlexanderTw33ts
absolutely smashed the eq vibe check!
awesome work!
22/32
@vapormensch
How can we be part of the beta?
I was also in Google Glass Explorer beta, it was super fun.
23/32
@minocrisy
I can't wait to play with the repo!
24/32
@stscott3
Very impressive, Justin. Looking forward to trying this out. What's the plan for durable memory, regarding past conversations?
25/32
@All4nDev
can i use this with custom voice models? like hypothetically if i were to have a lot of recordings of my own voice, upload that, then the voice would sound like me? on top of that, if it could digest the nuances in the way i speak, and output speech that sounds like how id say it, even better
26/32
@thecorysilva
This is amazing. I've seen a couple demos of Voice AI feeling really real, natural, and 'human'.
Great work! Excited to hear more about the open source stuff as well.
27/32
@dealer1943
tried it just now. incredible work. i have tried grok and chatgpt... this is on par with grok.
strange thing is when you are talking about top 99% assuming two LLMs have the same intelligence, the 1% is all about soft skills. which seems like a new frontier for LLMs.
28/32
@philippswu
exciting! congrats @justLV
29/32
@alexshye
This is amazing. Great job and excited to see where this goes. One q: Will be model be able to keep quiet if a person is thinking? It continually rambles which is kind of cool but I imagine feeling like talking to a person who doesn’t allow silence in a conversation.
30/32
@Saiyan3MD
Wow! Just... Wow
31/32
@JimGPT
Her!
32/32
@EquiTea_VC
This looks cool!
To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196