OpenAI introduces Sora

[bnew]

Spoiler: thread continued

Spoiler: full text & large images

1/11
@runwayml
Introducing, Act-One. A new way to generate expressive character performances inside Gen-3 Alpha using a single driving video and character image. No motion capture or rigging required.

Learn more about Act-One below.

(1/7)



https://video-t-2.twimg.com/ext_tw_...248/pu/vid/avc1/1280x720/2EyYj6GjSpT_loQf.mp4

2/11
@runwayml
Act-One allows you to faithfully capture the essence of an actor's performance and transpose it to your generation. Where traditional pipelines for facial animation involve complex, multi-step workflows, Act-One works with a single driving video that can be shot on something as simple as a cell phone.

(2/7)



https://video-t-2.twimg.com/ext_tw_...282/pu/vid/avc1/1280x720/Qie29gOWU42zMaGo.mp4

3/11
@runwayml
Without the need for motion-capture or character rigging, Act-One is able to translate the performance from a single input video across countless different character designs and in many different styles.

(3/7)



https://video-t-2.twimg.com/ext_tw_...696/pu/vid/avc1/1280x720/TcWzpRl3kMfHM4ro.mp4

4/11
@runwayml
One of the models strengths is producing cinematic and realistic outputs across a robust number of camera angles and focal lengths. Allowing you generate emotional performances with previously impossible character depth opening new avenues for creative expression.

(4/7)



https://video-t-2.twimg.com/ext_tw_...424/pu/vid/avc1/1280x720/3fupOI32Ck6ITIiE.mp4

5/11
@runwayml
A single video of an actor is used to animate a generated character.

(5/7)



https://video-t-2.twimg.com/ext_tw_...016/pu/vid/avc1/1280x720/Fh7WanCgSTR_ffHF.mp4

6/11
@runwayml
With Act-One, eye-lines, micro expressions, pacing and delivery are all faithfully represented in the final generated output.

(6/7)



https://video-t-2.twimg.com/ext_tw_...528/pu/vid/avc1/1280x720/ywweYvzLHe-3GO2B.mp4

7/11
@runwayml
Access to Act-One will begin gradually rolling out to users today and will soon be available to everyone.

To learn more, visit Runway Research | Introducing Act-One

(7/7)



8/11
@xushanpao310
@threadreaderapp Unroll



9/11
@threadreaderapp
@xushanpao310 Salam, please find the unroll here: Thread by @runwayml on Thread Reader App See you soon.



10/11
@threadreaderapp
Your thread is gaining traction! /search?q=#TopUnroll Thread by @runwayml on Thread Reader App @TheReal_Ingrid_ for unroll



11/11
@flowersslop
motion capture industry is cooked




To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

 

[bnew]

Spoiler: full text & large images

1/70
@RuudNL
Sora v2 release is impending:

* 1-minute video outputs
* text-to-video
* text+image-to-video
* text+video-to-video

OpenAI's Chad Nelson showed this at the C21Media Keynote in London. And he said we will see it very very soon, as @sama has foreshadowed.



https://video.twimg.com/ext_tw_video/1865424798098063360/pu/vid/avc1/1280x720/o2CgvaQxW-BwzuLv.mp4

2/70
@crownut984
Bro no one has access to to v1 why should I care about v2



3/70
@AI_Car_Pool
Monday - noon EST - 200$ per plan



4/70
@MicahJanke
Here’s to hoping the price isn’t insane



5/70
@caojiaming1
Looking forward to it!



6/70
@Goran_Majic
Put all the OpenAI employees in a room with all the computing power in the world, but no access to the internet and anyone else's content. Let them rebuild from scratch. It might be interesting to see what that would look like.



7/70
@clay_ripma
The convergence of this + VR is going to open up an entirely new category of personalized entertainment.

Most big releases are already 2 hours of CGI with a thin plot. Yawn.

But put me in there in VR where I can change around the story? It’ll be like lucid dreaming. I’m in.



8/70
@Nexus_AiAcademy
this is wild



9/70
@majdoubTn
Definitely need this



10/70
@TB_Motion
Holy s... I need more storage space



11/70
@pratjoey
Text and image to video will be a rich area.



12/70
@LittleTinRobot
I remain optimistic as have been waiting around this tech for a while now to test it out.
All sounds very intriguing.



13/70
@creativstrm
so there was a Sora V1? probably in another dimension... and even if a sora v2 will exist, it will be for professional users. So, whatever.



14/70
@boneGPT
big



15/70
@MaxZiebell
This is impressive—when is this from? Either it’s highly cherry-picked, or they’ve been hard at work. I had written off Sora as being overtaken, but could this mark a major comeback?



16/70
@TheAI_Frontier
It will be on day 3, I can feel it.



17/70
@koltregaskes
Looks great but I'm wondering release to who? Hollywood only?



18/70
@ZMadoc
Can’t wait!



19/70
@StevieMac03
V2? What happened to v1?



20/70
@jphilipp
Here's hoping for them to eventually release a lipsync tool, too... we need higher-quality and more face-detecting ones.



21/70
@MotionMark1111
Whoa



22/70
@rchitectopteryx
Where is v1?



23/70
@BenPielstick
If we can get 1 minute continuous shots and consistent characters between shots we are going to have a very interesting year.



24/70
@U79916881747113
Thats a big screen and the quality looks highly impressive so it must be 1080p or may be 2K and also the frame rate seems to be high so may be 24fps



25/70
@PlutusCosmos
Looking forward to it!



26/70
@iamdeepaklenka
Wow,it’s just incredible



27/70
@thegenioo
stunning and amazing



28/70
@imagineadil
wow



29/70
@PJaccetturo
Now I’m hyped.



30/70
@Emily_Escapor
I am ready the prompt is ready



31/70
@azed_ai
Wild!



32/70
@edalgomezn
@nicdunz



33/70
@bilawalsidhu
Okay this has me hyped — the military characters and spaceship generations are right up my alley



34/70
@NeuralHours
Incredible



35/70
@AliceAI314
I genuinely can't wait to get my hands on that. Look at the face consistency where the blood is splattered on her nose. It's identical to the other shot. Sora is looking stunning. And the biggest thing for me is little to no morphing and consistent characters.



36/70
@Yang_ML_Estate
Currently I have been using Kling. Will try Sora v2 after its release



37/70
@nebsh83
Bet they will drop it for xmas



38/70
@ChukwumaNwaugha
Wow, this is super impressive.



39/70
@mark_k
Very cool, thanks for posting!



40/70
@techguyver
What about API availability?



41/70
@Marconium2
What do you mean v2? We never got to see v1 lol



42/70
@DreschHorbach
Will it be pro only?



43/70
@Eric520CC
Amazing, very much looking forward to SORA V2, will it be released tomorrow?



44/70
@rajeshkannan_mj
I need to check it live before trusting them. Last time their demo was very cherry pick though. Only time will tell about the quality.



45/70
@AntiZionist1917
I think Sora is also image generator. Maybe it's just frame of videos that are acting as image generation. But it has some form of image generation.



46/70
@yc3t_
holy



47/70
@D41Dev
I've been waiting to see text+video-to-video models, this is where significant leaps can be made.

- Full length movies can be made by inserting script snippets and a 1min clip of the previous scene. This can be done in a loop.

- Enhanced and simplified video editing



48/70
@seeupsidedown
LET. US. COOK.



49/70
@ShivOperator
Gradually, then suddenly



50/70
@GinChou
@HuoQubot



51/70
@AlexanderNL
haha hoefde je geen NDA te tekenen?



52/70
@emon_whatever
holy! another big bomb is about to drop



53/70
@AI_Car_Pool


[Quoted tweet]
Is it finally coming? @OpenAI

54/70
@bowtiedwhitebat
we bout to end the woke in movies



55/70
@Emily_Escapor
I am ready



56/70
@evren_mercan
Unlimited 2000 $/month



57/70
@SomBh1
Crazy. Imagine on a million GPU cluster. Can make a 15 min short movie.



58/70
@crypto_sam_974
who cares if it is not released...



59/70
@BreezeChai
Crazy



60/70
@_NamelessBrain_
Sora Turbo, not v2. Very different beasts.



61/70
@api_prlabs
Unlock the power of advanced AI with PR Labs' ChatGPT4 API! Starting at just $5/month with up to 2.4M requests,it's perfect for developers and businesses aiming to scale effortlessly.
Check now:ChatGPT 4
website: PR LABS API
/search?q=#ChatGPT4 /search?q=#OpenAI /search?q=#NewsAPI



62/70
@nextdao

63/70
@PrescienceDS
Exciting to see the Sora v2 release on the horizon! The addition of text-to-video and text+image-to-video features will definitely open up new creative possibilities. Can't wait to see how this will impact content creation and storytelling!



64/70
@YolaoDude
Looks great. But if the price tag is high, $200 or so, then i will jump in IF...
-It can generate FAR more than just 5 to 10 seconds videos!
-you can have full control of the movement of the characters, video to video.
-the character are consistent.



65/70
@aidesignss
sora v1 has not been released yet



66/70
@JogishE45866
Wonderful



67/70
@nextdao
http://OpenAI.chat → http://OpenAI.com

http://OpenAI.tech → http://OpenAI.com

http://OpenAI.tools→ http://OpenAI.com

Now only two domain names for sale.
@sama @OpenAI

openai.im is available for purchase - Sedo.com

openai.gg is available for purchase - Sedo.com

/search?q=#OpenAI



68/70
@Opic3D
Cool!



69/70
@farzadhss
Is it open source?



70/70
@irrealer
wild.




To post tweets in this format, more info here: https://www.thecoli.com/threads/tips-and-tricks-for-posting-the-coli-megathread.984734/post-52211196

 

[Jesus H. Christ]

Anybody in here got access to Sora? I can't register as of now

 

[Jesus H. Christ]

 

[Stone Cold]

 

[newarkhiphop]

Anybody in here got access to Sora? I can't register as of now

I've had access since day 1

 

[bnew]

 

[bnew]

Spoiler: thread continued

Spoiler: full text & large images

1/11
@_philschmid
WTF?! New open-source physics AI engine absolutely insane! Genesis is a new physics engine that combines ultra-fast simulation with generative capabilities to create dynamic 4D worlds for robotics and physics.

TL;DR:
430,000x faster than real-time physics simulation, processes 43M FPS on a single RTX 4090
Built in pure Python, 10-80x faster than existing GPU solutions like Isaac Gym
Cross-platform support: Linux, MacOS, Windows, with CPU, NVIDIA, AMD, and Apple Metal backends
Unified framework combining multiple physics solvers: Rigid body, MPM, SPH, FEM, PBD, Stable Fluid
Extensive robot support: arms, legged robots, drones, soft robots; supports MJCF, URDF, obj, glb files
Built-in photorealistic ray-tracing rendering
Takes only 26 seconds to train real-world transferrable robot locomotion policies
Simple installation via pip: pip install genesis-world
Physics engine and simulation platform are fully open-sourced
”.generate” method/generative framework coming soon.



https://video.twimg.com/ext_tw_video/1869639134156840960/pu/vid/avc1/1280x720/rffXgweJaKVMxGFQ.mp4

2/11
@_philschmid
Code: GitHub - Genesis-Embodied-AI/Genesis: A generative world for general-purpose robotics & embodied AI learning.
Documentation: Genesis — Genesis 0.2.0 documentation

GitHub - Genesis-Embodied-AI/Genesis: A generative world for general-purpose robotics & embodied AI learning.



3/11
@ma5701458
@pikaso_me screenshot this



4/11
@pikaso_me
Enjoy your screenshot and...

Get 10 stickers for $1: $1 for 10 | Sticker Mule

5/11
@PamirSevincel
this is insane. you probably won't need that much real world data to train physical systems e.g. humanoid robots.

@adcock_brett what do you think?



6/11
@Mooie_89
@khouuba



7/11
@BalajiAkiri
A new era of Robotics begins



8/11
@360
Reality breaks more every day



9/11
@AIBuzzNews
@desy @desynews das könnte spannend und auch nützlich sein.



10/11
@glaksmono
This is super cool



11/11
@EducatingwithAI
Generative AI has hit a wall

In before: but this is not pure GenAI.

No, but it uses it with other tools - exactly what @ilyasut mentioned.

Spoiler: thread continued

Spoiler: full text & large images

1/11
@EHuanglu
This New AI Just Nuked the Competition

Genesis AI is here to wipe out every outdated simulation
and totally dominate AI world.

- Open sourced
- Create data from prompt
- 430,000x faster than real-time
- Create 4D physical worlds in seconds with text prompt

Let's dive in:



https://video.twimg.com/ext_tw_video/1869694710941618176/pu/vid/avc1/1280x720/dZ67k0pVaJpn20Ny.mp4

2/11
@EHuanglu
1. Generating 4D dynamical & physical world

• VLM-based generative agent
• Worlds provide diverse data modalities.
• Generates physically accurate, view consistent videos with camera and object motion.



https://video.twimg.com/ext_tw_video/1869709610762055682/pu/vid/avc1/1280x720/vznCJHqYHQh66u3C.mp4

3/11
@EHuanglu
2. Precise control

• Provides complete control over physics
• Handles camera motion and parameters with precision



https://video.twimg.com/ext_tw_video/1869709664302313472/pu/vid/avc1/1280x720/AnS2VyDilYPHlN2a.mp4

4/11
@EHuanglu
3. Character Motion Generation

• A Chinese soldier performs the Gangnam Style dance
• A Japanese samurai performs boxing
• A Roman soldier walks forward like a zombie



https://video.twimg.com/ext_tw_video/1869709732405182464/pu/vid/avc1/1280x720/tSN-btsf_6i4GXb_.mp4

5/11
@EHuanglu
4. Robotic Policy Generation

• Generative robotic agent + advanced physics engine
• Generates demonstration data for diverse skills
• Enables robots to learn and adapt autonomously.
• Powered by AI models for next-level automation.



https://video.twimg.com/ext_tw_video/1869709846528110592/pu/vid/avc1/1280x720/3fM-cF-Dmh5Ta9As.mp4

6/11
@EHuanglu
5. 3D & Fully Interactive Scene Generation

"A home interior scene with a living room (including a dinning space), a restroom, a study and a bedroom."



https://video.twimg.com/ext_tw_video/1869709919198515200/pu/vid/avc1/1280x720/frU1kO_8cGhMx4Fz.mp4

7/11
@EHuanglu
6. Speech Audio, Facial Animation & Emotion Generation



https://video.twimg.com/ext_tw_video/1869709968406142976/pu/vid/avc1/1280x720/LOXAphf2Ddtic8v1.mp4

8/11
@EHuanglu
Open sourced code:
GitHub - Genesis-Embodied-AI/Genesis: A generative world for general-purpose robotics & embodied AI learning.

Project Page:
Genesis



9/11
@EHuanglu
If you enjoyed reading this post,

Follow @EHuanglu for more great stuff!

And support with like/repost the post below

[Quoted tweet]
This New AI Just Nuked the Competition

Genesis AI is here to wipe out every outdated simulation
and totally dominate AI world.

- Open sourced
- Create data from prompt
- 430,000x faster than real-time
- Create 4D physical worlds in seconds with text prompt

Let's dive in:
[media=twitter]1869710532934254954[/media]

https://video.twimg.com/ext_tw_video/1869694710941618176/pu/vid/avc1/1280x720/dZ67k0pVaJpn20Ny.mp4

10/11
@zeng_wt
This is just 2024. Not even 2025. And we have this! wow



11/11
@EHuanglu
I always thought 2025 is the year of 3D, it seems I was short sighted

Spoiler: thread continued

Spoiler: full text & large images

1/11
@zhou_xian_
Everything you love about generative models — now powered by real physics!

Announcing the Genesis project — after a 24-month large-scale research collaboration involving over 20 research labs — a generative physics engine able to generate 4D dynamical worlds powered by a physics simulation platform designed for general-purpose robotics and physical AI applications.

Genesis's physics engine is developed in pure Python, while being 10-80x faster than existing GPU-accelerated stacks like Isaac Gym and MJX. It delivers a simulation speed ~430,000 faster than in real-time, and takes only 26 seconds to train a robotic locomotion policy transferrable to the real world on a single RTX4090 (see tutorial: Training Locomotion Policies with RL — Genesis 0.2.0 documentation).

The Genesis physics engine and simulation platform is fully open source at GitHub - Genesis-Embodied-AI/Genesis: A generative world for general-purpose robotics & embodied AI learning.. We'll gradually roll out access to our generative framework in the near future.

Genesis implements a unified simulation framework all from scratch, integrating a wide spectrum of state-of-the-art physics solvers, allowing simulation of the whole physical world in a virtual realm with the highest realism.

We aim to build a universal data engine that leverages an upper-level generative framework to autonomously create physical worlds, together with various modes of data, including environments, camera motions, robotic task proposals, reward functions, robot policies, character motions, fully interactive 3D scenes, open-world articulated assets, and more, aiming towards fully automated data generation for robotics, physical AI and other applications.

Open Source Code: GitHub - Genesis-Embodied-AI/Genesis: A generative world for general-purpose robotics & embodied AI learning.
Project webpage: Genesis

Documentation: Genesis — Genesis 0.2.0 documentation

1/n



https://video.twimg.com/ext_tw_video/1869510892880097280/pu/vid/avc1/1280x720/_dc5zlB1WmdE01Jh.mp4

2/11
@zhou_xian_
Nvidia brought GPU acceleration to robotic simulation, speeding up simulation speed by more than one order of magnitude compared to CPU-based simulation. This brought numerous amazing robotic skills to life by leveraging large-scale GPU-parallelized simulation. Genesis pushes up this speed by another order of magnitude. Note that the speed improvement is achieved with no compromise in simulation accuracy.
2/n



https://video.twimg.com/ext_tw_video/1869506856051261441/pu/vid/avc1/1280x720/F5sz3RnivNvuO6Dj.mp4

3/11
@zhou_xian_
Genesis supports simulating various types of physical phenomena. We developed from scratch a unified physics engine that integrates various SOTA physics solvers (MPM, SPH, FEM, Rigid Body, PBD, etc.), supporting simulation of a wide range of materials: rigid body, articulated body, Cloth, Liquid, Smoke, Deformables, Thin-shell materials, Elastic/Plastic Body, Robot Muscles, etc.

3/n



https://video.twimg.com/ext_tw_video/1869507363788541952/pu/vid/avc1/1280x720/NEU8FPHKy4Z_bAk6.mp4

4/11
@zhou_xian_
Genesis is the first-ever platform providing comprehensive support for soft muscles and soft robot and their interaction with rigid robots. Genesis also ships with a URDF-like soft-robot configuration system.

4/n



https://video.twimg.com/ext_tw_video/1869512601123594241/pu/vid/avc1/1280x720/UUrRbSSpOXEOIhY_.mp4

5/11
@zhou_xian_
Genesis's generative framework supports generating 3D and fully interactive scenes for training robotic skills
5/n



https://video.twimg.com/ext_tw_video/1869512781008973825/pu/vid/avc1/1280x720/62RDqV516piUnQM5.mp4

6/11
@zhou_xian_
Our generative agent autonomously proposes robotic tasks, design environments, write reward functions, and ultimately leading to automated generation of robotic policies.

6/n



https://video.twimg.com/ext_tw_video/1869512976299864064/pu/vid/avc1/1280x720/0YflDqvt8WnNtXNj.mp4

7/11
@zhou_xian_
Genesis's generative framework supports data generation beyond robotics, such as character motion:

7/n



https://video.twimg.com/ext_tw_video/1869513206672117760/pu/vid/avc1/1280x720/-sNINHa-_cgzgdiI.mp4

8/11
@zhou_xian_
Genesis's GPU parallellized IK solver is able to solve IK for 10,000 Franka arms simultaneously, under 2ms:

8/n



https://video.twimg.com/ext_tw_video/1869513330584338432/pu/vid/avc1/1280x720/JzGz03g2Il-_wzLD.mp4

9/11
@zhou_xian_
We support native non-convex collision handling:

9/n



https://video.twimg.com/ext_tw_video/1869513464055492608/pu/vid/avc1/1280x720/B6sCV4PMyOtFHggC.mp4

10/11
@zhou_xian_
Genesis supports a physically accurate tactile sensing simulation module: GitHub - Genesis-Embodied-AI/DiffTactile: [ICLR 2024] DiffTactile: A Physics-based Differentiable Tactile Simulator for Contact-rich Robotic Manipulation. (Will be integrated into the main branch in a future release soon)

10/n



11/11
@zhou_xian_
Finally, a cute interactive physical Tetris game made with Genesis

Thanks to all the amazing collaborators who together made everything possible over the last two years! There's no space here to @ every single one, but a huge kudos to the whole Genesis team!

We welcome everyone from the open-source community to come join us and build Genesis with us together!

11/11



https://video.twimg.com/ext_tw_video/1869514325922058240/pu/vid/avc1/1280x720/kOCJbJ4KQnUJw1ht.mp4