Not to be outdone by competitors such as Google, who recently showcased a text-to-video tool, AI startup OpenAI on Thursday unveiled its own text-to-video model, Sora.
Text-to-video has become the latest arms race in generative AI, as OpenAI, Google, Microsoft, and others look beyond text and image generation to cement their position in a sector expected to generate $1.3 trillion in revenue by 2032 — and to win over consumers who have been intrigued by generative AI since ChatGPT debuted a little more than a year ago.
In a blog post on Thursday, the company said that the maker of both ChatGPT and Dall-E, Sora, will be available to “red teamers”, or experts in areas such as misinformation, hateful content and bias, who will be “adversarially testing the model.” It will also be engaging with visual artists, designers and filmmakers to gain additional feedback from creative professionals. That adversarial testing will be especially crucial in addressing the potential for convincing deepfakes, a major area of concern for the use of AI to generate images and video.
In addition to garnering feedback from outside the organisation, the AI startup stated that it wants to share its progress now to “give the public a sense of what AI capabilities are on the horizon.”
Strengths of Sora
Prompt: “Several giant wooly mammoths approach treading through a snowy meadow, their long wooly fur lightly blows in the wind as they walk, snow covered trees and dramatic snow capped mountains in the distance, mid afternoon light with wispy clouds and a sun high in the distance… pic.twitter.com/Um5CWI18nS— OpenAI (@OpenAI) February 15, 2024
This is due in part to OpenAI’s previous work with the Dall-E and GPT models. The text-to-image generator Dall-E 3 was released in September 2023. Sora, in particular, uses Dall-E 3’s recaptioning technique, which, according to OpenAI, generates “highly descriptive captions for the visual training data.
“Sora is able to generate complex scenes with multiple characters, specific types of motion and accurate details of the subject and background,” the post said. “The model understands not only what the user has asked for in the prompt, but also how those things exist in the physical world,” it added.
Prompt: “A gorgeously rendered papercraft world of a coral reef, rife with colorful fish and sea creatures.” pic.twitter.com/gzEE8SwP81— OpenAI (@OpenAI) February 15, 2024
Similar to Lumiere, the model can generate video from still photos, extend existing videos, and fill in missing frames.
“Sora serves as a foundation for models that can understand and simulate the real world, a capability we believe will be an important milestone for achieving AGI,” the post added.
AGI, or artificial general intelligence, is a more advanced version of AI that is closer to human-like intelligence and has the ability to perform a greater range of tasks. Meta and DeepMind have also shown an interest in reaching this benchmark.
Sora’s weaknesses
OpenAI admitted that Sora has weaknesses, like struggling to accurately depict the physics of a complex scene and to understand cause and effect.
“For example, a person might take a bite out of a cookie, but afterward, the cookie may not have a bite mark,” the post said.
And anyone who still has to make an L with their hands to figure out which one is left can take heart: Sora also mixes up left and right.
Is OpenAI’s Sora available to the public?
OpenAI did not specify when Sora will be widely available but noted it wants to take “several important safety steps” first. That includes adhering to OpenAI’s existing safety standards, which prohibit extreme violence, sexual content, hateful imagery, celebrity likeness and the IP of others.
“Despite extensive research and testing, we cannot predict all of the beneficial ways people will use our technology, nor all the ways people will abuse it,” the post said, adding, “That’s why we believe that learning from real-world use is a critical component of creating and releasing increasingly safe AI systems over time.”
First Published: Feb 16 2024 | 12:14 PM IST
Note:- (Not all news on the site expresses the point of view of the site, but we transmit this news automatically and translate it through programmatic technology on the site and not from a human editor. The content is auto-generated from a syndicated feed.))
