OpenAI introduces Sora

[Roger king]

This needs to be seriously regulated to avoid people getting cheated and duped too

 

[bnew]

https://archive.is/UCfa7

Apparently some folks don't get "data-driven physics engine", so let me clarify. Sora is an end-to-end, diffusion transformer model. It inputs text/image and outputs video pixels directly. Sora learns a physics engine implicitly in the neural parameters by gradient descent through massive amounts of videos.

Sora is a learnable simulator, or "world model". Of course it does not call UE5 explicitly in the loop, but it's possible that UE5-generated (text, video) pairs are added as synthetic data to the training set.

https://archive.is/ZZNvj

If you think OpenAI Sora is a creative toy like DALLE, ... think again. Sora is a data-driven physics engine. It is a simulation of many worlds, real or fantastical. The simulator learns intricate rendering, "intuitive" physics, long-horizon reasoning, and semantic grounding, all by some denoising and gradient maths.

I won't be surprised if Sora is trained on lots of synthetic data using Unreal Engine 5. It has to be!

Let's breakdown the following video. Prompt: "Photorealistic closeup video of two pirate ships battling each other as they sail inside a cup of coffee."

- The simulator instantiates two exquisite 3D assets: pirate ships with different decorations. Sora has to solve text-to-3D implicitly in its latent space.
- The 3D objects are consistently animated as they sail and avoid each other's paths.
- Fluid dynamics of the coffee, even the foams that form around the ships. Fluid simulation is an entire sub-field of computer graphics, which traditionally requires very complex algorithms and equations.
- Photorealism, almost like rendering with raytracing.
- The simulator takes into account the small size of the cup compared to oceans, and applies tilt-shift photography to give a "minuscule" vibe.
- The semantics of the scene does not exist in the real world, but the engine still implements the correct physical rules that we expect.

Next up: add more modalities and conditioning, then we have a full data-driven UE that will replace all the hand-engineered graphics pipelines.

edit:

had AI simply the text above into a conversation...

Person A: Hey, have you heard about OpenAI's new thing called Sora?

Person B: No, what's that? Is it like DALL-E, where you type something and it makes a picture?

Person A: Kinda, but way more advanced. Sora isn't just for making pictures; it makes videos! And not just any videos – it creates whole worlds with moving objects, like pirate ships sailing in a cup of coffee!

Person B: Whoa, that sounds crazy! So it's like a super-smart movie maker?

Person A: Yeah, exactly! It's like a virtual reality machine that uses math and data to figure out how things should move and look realistic. It learns from examples and can do things like simulate water, make things move in a believable way, and even change the camera angles to make it seem like a tiny world.

Person B: Okay, but how does it know how to make everything look so real? Like, does it have to watch a lot of videos first?

Person A: Well, imagine Sora as a student who watches a ton of stuff, like movies and video games made with Unreal Engine 5, which is a really good program for creating realistic visuals. By studying all that, Sora learns how to make things look and move like in those videos.

Person B: So it's like it's copying what it sees?

Person A: Sort of, but it's more like it absorbs the patterns and rules of how things work, then uses that knowledge to create new scenes on its own. Like, if you tell it to make two pirate ships fight in a coffee cup, it figures out the right size for the ships, how they should move, and even the foam on the coffee!

Person B: That's insane! It's like magic!

Person A: Almost! But remember, it's all based on math and learning. The tricky part is that Sora understands the logic of physics, even in situations that don't exist in real life. So, even though pirate ships in coffee cups aren't possible, Sora makes sure they float and move like they would in the ocean.

Person B: Wow, I can't wait to see what else it can do! It sounds like it could change how we make movies and games someday.

Person A: Definitely! If they keep improving Sora, it could become a tool that does all the hard work of creating stunning visuals, leaving humans to focus on the storytelling and creativity. It's like having a personal movie-making genius in your computer!

 

[bnew]

This needs to be seriously regulated to avoid people getting cheated and duped too

what sort of regulation do you think is needed?

Stable Video — Stability AI

Stability AI’s first open generative AI video model based on the image model Stable Diffusion.

stability.ai

stabilityai/stable-video-diffusion-img2vid-xt · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co

stabilityai/stable-video-diffusion-img2vid · Hugging Face

We’re on a journey to advance and democratize artificial intelligence through open source and open science.

huggingface.co

Civitai | Share your models

 

[bnew]

It's over for VFX artists.

maybe not, they're still artists meaning they can explore ideas faster with a tool like this.

 

[MegaManX]

Probly fake news. Key words "Released to a select few to not upset the industry."

In other words, we don't want to provide proof of it actually working with HEAVY post processing from real animators so we will just hyper up our make believe product to get more investor funds and a larger market cap while having absolutely no product for 5 years.

 

[O.Red]

Probly fake news. Key words "Released to a select few to not upset the industry."

In other words, we don't want to provide proof of it actually working with HEAVY post processing from real animators so we will just hyper up our make believe product to get more investor funds and a larger market cap while having absolutely no product for 5 years.

The video game playbook

 

[bnew]

Probly fake news. Key words "Released to a select few to not upset the industry."

In other words, we don't want to provide proof of it actually working with HEAVY post processing from real animators so we will just hyper up our make believe product to get more investor funds and a larger market cap while having absolutely no product for 5 years.

dude was literally taking requests yesterday.














the pieces for this tech exists already, openai just has a better product right now.

google product is fake too?

Google shows off Lumiere, a space-time diffusion model for realistic AI videos

Lumiere was trained on a dataset of 30 million videos, along with their text captions, and is capable of generating 80 frames at 16 fps. The source of this data, however, remains unclear at this stage.

venturebeat.com

they're not releasing it just yet so people can test it for creative purposes and test it to learn what guard rails will be needed to avoid major abuse. I strongly suspect they can produce video thats longer than a minute too.

 

[Vandelay]

Digital anarchy.

 

[Breh13]

Those geriatrics fools in power ain’t prepared for the AI fukkery we’re gonna to get in the next 2-3 years. It’s developing super fast.

 

[NoMorePie]

Possibly opportunity to make money using this. Draft a quick video and then loop it and put it on YouTube. People use it as background music

 

[IIVI]

Those geriatrics fools in power ain’t prepared for the AI fukkery we’re gonna to get in the next 2-3 years. It’s developing super fast.

Oh we are nowhere near ready

I said it before the time to prepare for and address A.I was back in 2016.

Now these old clowns in office can't even spell A.I much less know how to deal with it and the impact it can cause.

 

[↓R↑LYB]

I think this is a good place to stop putting points into the technology skill tree and to start leveling up humanity

You better save that shyt for when humanity rebuilds

We riding this AI train until skynet pull them dikks out

 

[Hater Eraser]

like he saids 7 minutes into the video.. jobs will be lost, stock footage sites, drone pilots etc will be out of a job when customers just need something generic. once they make it so that people can add clips or photos for reference, it's truely gonna be a wakeup call for a lot of people.

" Follow the flow, look
They say a midget standin' on the giant's shoulder can see much
Further than the giant.. (the giant..)

.. Jeah, look
I'm so far ahead of my time, I'm 'bout to start another life
Look behind you, I'm 'bout to pass you twice
Back to the future, gotta slow up for the present
I'm fast, nikkas can't get pass my past
How they propose to deal with my perfect present? "

“All the tools, techniques and technology in the world are nothing without the head, heart and hands to use them wisely, kindly and mindfully.” - Rasheed Ogunlaru

“A good tool improves the way you work. A great tool improves the way you think.” - Jeff Duntemann

“Technology is nothing. What's important is that you have a faith in people, that they're basically good and smart, and if you give them tools, they'll do wonderful things with them.” - Steve Jobs

 

[BaggerofTea]

It's over for VFX artists.

All of these tools depend on humans.


One thing we have that beats an ai is an organic brain

 

[Big Blue]

All of these tools depend on humans.


One thing we have that beats an ai is an organic brain

A lot of the expense of video effects is the manual labor and man hours involved in re-creating generic effects like green screen backgrounds etc. That's the concern.