“Creative software is dead.” This is a quote from Christobol Valenzuela, Co-Founder of Runway, a generative video startup. He’s saying that generative video AI will completely disrupt traditional photo and video editing software like Adobe Photoshop and Premiere Pro.
But he’s really burying the lede.
Generative video model announcements are quite the thing at the moment. OpenAI showed off Sora in February and Google demoed Veo in May. In June, Chinese social media company kaishou made Kling public. Just two and then five days later, startups Luma AI and then Runway ML launched their models.
But why does all of this matter?
For video models to be successful i.e., for them to generate believable and “photo realistic” video, the models need to understand our world better than today’s language models do. Video models will need to know the physics of all the objects and people in a scene and specifically how those objects can interact.
One example of this is cutting & eating a slice of cake. You need to show the person enjoying the cake, chewing, swallowing, and then having an empty mouth. You also need to show the cake with a slice missing, all its layers, and the residual crumbs on the platter. You can think of this as “world modeling.”
The OpenAI team plainly talks about how world modeling will eventually enable these video models to better entertain, educate, and communicate with us in real-time. If new videos can be generated on the fly, then effectively video can be personalized for an audience of 1.
Entertainment will be stickier than TikTok. Education will be tailored to our individual learning styles. And because it’s real-time, the experience can be interactive and not have a predetermined ending. Taken even further, the technology could combine education and entertainment together. Generative video will create new mediums of consumer experience.
That’s my take this week on GenAI.
Share this post