New Features and Capabilities in Sora
A music video by Canadian art collective Vallée Duhamel made with Sora-generated video. “[We] just shoot stuff and then use Sora to combine it with a more interesting, more surreal vision.”
Explore Interface and Storyboard Feature
During a livestream on Monday—during Day 3 of OpenAI’s “12 days of OpenAi”—Sora’s developers showcased a new “Explore” interface that allows people to browse through videos generated by others to get prompting ideas. OpenAI says that anyone can enjoy viewing the “Explore” feed for free, but generating videos requires a subscription.
They also showed off a new feature called “Storyboard” that allows users to direct a video with multiple actions in a frame-by-frame manner.
Safety Measures and Limitations
In addition to the release, OpenAI also published Sora’s System Card for the first time. It includes technical details about how the model works and safety testing the company undertook prior to this release.
“Whereas LLMs have text tokens, Sora has visual patches,” OpenAI writes, describing the new training chunks as “an effective representation for models of visual data… At a high level, we turn videos into patches by first compressing videos into a lower-dimensional latent space, and subsequently decomposing the representation into spacetime patches.”
Sora also makes use of a “recaptioning technique”—similar to that seen in the company’s DALL-E 3 image generation, to “generate highly descriptive captions for the visual training data.” That, in turn, lets Sora “follow the user’s text instructions in the generated video more faithfully,” OpenAI writes.
Technical Details and Limitations
Sora-generated video provided by OpenAI, from the prompt: “Loop: a golden retriever puppy wearing a superhero outfit complete with a mask and cape stands perched on the top of the empire state building in winter, overlooking the nyc it protects at night. the back of the pup is visible to the camera; his attention faced to nyc”
OpenAI implemented several safety measures in the release. The platform embeds C2PA metadata in all generated videos for identification and origin verification. Videos display visible watermarks by default, and OpenAI developed an internal search tool to verify Sora-generated content.
The company acknowledged technical limitations in the current release. “This early version of Sora will make mistakes, it’s not perfect,” said one developer during the livestream launch. The model reportedly struggles with physics simulations and complex actions over extended durations.
Conclusion
While Sora has made significant progress in AI video generation, it is still an early technology and has limitations. OpenAI is committed to continuing to improve Sora and has implemented safety measures to ensure responsible use. As the technology advances, it has the potential to revolutionize the way we create and interact with video content.
FAQs
Q: What is the Explore interface?
A: The Explore interface is a new feature in Sora that allows users to browse through videos generated by others to get prompting ideas.
Q: Is the Explore interface free?
A: Yes, anyone can view the Explore feed for free. However, generating videos requires a subscription.
Q: What is the Storyboard feature?
A: The Storyboard feature is a new feature in Sora that allows users to direct a video with multiple actions in a frame-by-frame manner.
Q: Are there any safety measures in place?
A: Yes, OpenAI has implemented several safety measures, including C2PA metadata, visible watermarks, and an internal search tool to verify Sora-generated content.
Q: What are the technical limitations of Sora?
A: Sora currently struggles with physics simulations and complex actions over extended durations. However, OpenAI is committed to continuing to improve the technology.
