Adobe Firefly AI Video Model: A Game-Changer for Professionals
Kling, Sora, and Runway have been the main names to watch for AI video generation, but Adobe has just leapt into the pack with the beta release of its Firefly AI video model. And, based on the demo I was shown, my first impression is that anything it may lack in realism it makes up for in the amount of control it provides, making it potentially the most useful AI video generator for professionals.
Firefly AI Video Capabilities
Firefly AI video can generate video from text prompts and from still image references. And like with its Firefly image model, Adobe claims it’s the industry’s first "commercially safe AI video generation model to market".
Control and Customization
According to Adobe, the Firefly Video Model generates IP-friendly video content that can be used in commercial productions without fear of copyright infringement (though it hasn’t provided much info on training data). For now, it can generate up to five seconds of footage at a time in 1080p resolution at a default 24 fps in vertical or horizontal format. Each generation takes about 90 seconds. Adobe says a lower resolution model for high-speed ideation and a 4K model for professional production are coming soon.
It looks like the biggest advantage of Firefly’s video model is the amount of control it provides – something that’s been difficult in other models. Adobe’s recognized that being able to direct the output is critical to producing usable footage. You can choose shot size (close up, medium, long, etc.), camera angle (aerial, eye-level, high, low, top-down), and camera motion (zoom in, tilt, handheld, etc.) to provide more control over framing and the dynamism of a shot.
Image-to-Video Functionality
The image-to-video functionality allows two images to be uploaded to define the first and last frames of a shot, with the AI generating the movement in between. And like with many image generators, seed number can be used for more consistency in iterations, for example, to keep a similar piece of footage with a different camera angle.
Workflow and Integration
Since Firefly already provides still image generation, Adobe envisions a workflow in which users will generate images in Firefly, edit them in Photoshop, turn them into video in the Firefly video model and then take them into Premiere Pro to edit and integrate them into video projects. This complete workflow, benefiting from the powerful color and masking tools already in Premiere Pro, could give Adobe the edge over rival tools.
Limitations and Future Plans
When the Firefly image model was released, its output often looked less realistic than results from the likes of Midjourney and Stable Diffusion, which were trained with unlicensed material. I’ve not seen enough from Firefly video yet to make a call, but initial results don’t look so far from other models. A lot of the example footage from Adobe is very clearly AI – those whales (below) aren’t going to fool anybody. But the more precise controls make for some strong use cases for Firefly.
New Firefly Web App
To coincide with the addition of the video model in beta, the Firefly web application has been revamped to incorporate access to the new tool.
New AI Image Tool: Scene to Image
Adobe has also added another new AI image tool: Scene to Image in beta. This allows users to design a basic 3D structure in browser and then add a text prompt to turn it into an image.
New Translate Video Tool
There’s also a new Translate Video tool, which Adobe reckons will save time on dubbing and audio mixing. Just don’t think I would rely on it without a native speaker on hand to verify the results.
New Adobe Firefly AI Plans
New models mean new costs. The Firefly video model is free to try, but with limited generations. Adobe’s announced the launch of two Firefly monthly subscription plans: Firefly Standard and Firefly Pro.
Conclusion
Adobe Firefly AI video model is a game-changer for professionals, offering a high degree of control and customization. While it may lack in realism, its capabilities make it a valuable tool for a variety of applications.
Frequently Asked Questions
Q: What is the Firefly AI video model?
A: The Firefly AI video model is a commercially safe AI video generation model that can generate video from text prompts and from still image references.
Q: What are the benefits of Firefly AI video model?
A: The Firefly AI video model provides a high degree of control and customization, allowing users to choose shot size, camera angle, and camera motion to produce usable footage.
Q: What are the limitations of Firefly AI video model?
A: The Firefly AI video model may lack in realism compared to other models, but its capabilities make it a valuable tool for a variety of applications.
Q: What are the pricing plans for Firefly AI video model?
A: The pricing plans for Firefly AI video model are as follows: Firefly Standard starts at $9.99 for 2,000 video/audio credits per month, good for up to 20 five-second 1080p video generations; Firefly Pro starts at $29.99 per month for 7,000 video/audio credits – up to 70 five-second 1080p video generations.
