Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More
Adobe has announced a new addition to its suite of creative tools: the Adobe Firefly Video Model, its own foray into the increasingly competitive AI video generation space, built atop its existing and ever-expanding family of Firefly generative AI still image models, which the company claims as ethically trained and commercially safe thanks to using only data it owns or has the license to, uploaded by contributors to its Adobe Stock service.
However, as VentureBeat reported last year, some Adobe Stock creators dispute this since even though they had to agree to terms of service that allowed Adobe broad usage of their works in order to upload them to the service, they never imagined that the generative AI era would occur and use said works to create new ones potentially in their styles and that compete with their efforts.
Adobe’s Firefly Video Model will be available in beta later this year, and the company is offering early access through a waitlist — interested parties can apply here.
It supports text-to-video, image-to-video, and even video editing features all in the same model. However, generations are limited to up to 5 seconds, according to The Verge.
Impressive high-quality examples
Early examples of clips generated by Firefly Video posted to Adobe’s blog show off impressive quality and adherence to text prompts, generated in less than 2 minutes.
For example, this prompt: “Cinematic closeup and detailed portrait of a reindeer in a snowy forest at sunset. The lighting is cinematic and gorgeous and soft and sun-kissed, with golden backlight and dreamy bokeh and lens flares. The color grade is cinematic and magical.”
…results in this video:
This prompt: “Slow-motion fiery volcanic landscape, with lava spewing out of craters. the camera flies through the lava and lava splatters onto the lens. The lighting is cinematic and moody. The color grade is cinematic, dramatic, and high-contrast.”
…results in:
What it means for enterprise decision-makers
For enterprise decision makers looking to use AI to craft internal videos for employees and training or just “vibes,” or external videos for customers and marketing efforts, even full advertisements — Adobe’s new commercially safe Firefly Video may be a very compelling option.
After all, Adobe offers indemnification for users, that is, an agreement to defend users from infringement lawsuits and legal actions taken against them when using its AI models, though it has not yet explicitly stated whether Firefly Video will be covered.
But also, enterprise decision makers will need to weigh the costs of waiting to get access to Adobe Firefly Video versus jumping in and using one of the many other high-quality AI video generators publicly available now, such as Runway Gen-3 Alpha Turbo or Luma AI’s Dream Machine.
The next stage of Firefly’s advancement
Since launching Adobe Firefly in March 2023, Adobe has used the model as the basis of new AI features sprinkled throughout its widely used Creative Cloud software suite.
Firefly tools are already embedded in popular applications such as Photoshop, Illustrator, and Lightroom, empowering users with features like Generative Fill, Generative Remove, and Text-to-Template.
According to Adobe, over 12 billion images and vectors have been generated using Firefly since its launch, making it one of the company’s most rapidly adopted innovations.
Ashley Still, an Adobe executive leading the project and author of Adobe’s announcement blog post stated that the firm “worked closely with the video editing community to advance the Firefly Video Model.”
“Guided by their feedback and built with creators’ rights in mind, we’re developing new workflows leveraging the model to help editors ideate and explore their creative vision, fill gaps in their timeline and add new elements to existing footage.“
More than just AI video generation
As such, Firefly Video does far more than just generate new videos from text prompts. It also offers AI-powered editing features, including removing unwanted objects and perfecting transitions.
he rise of short-form video content, combined with ever-tighter production deadlines, has pushed editors and filmmakers to work across multiple disciplines.
Adobe’s Firefly Video Model helps address this by offering tools that enable editors to not only cut footage but also manage color correction, animation, visual effects, and more.
Editors can now leverage AI to speed up these processes while maintaining high-quality output, freeing them to focus on creativity.
The model also supports a broad range of creative effects, including generating b-roll, macro shots, and atmospheric elements like fire, smoke, or water that users can layer over prerecorded or animated footage.
One standout feature coming to Premiere Pro later this year is Generative Extend. This function will allow editors to extend video clips, fill gaps in footage, or smooth transitions—helping to match cuts to the pacing of audio and other visual elements.
Adobe demonstrated this capability by showcasing how Generative Extend could hold on a shot longer to perfectly align with a musical crescendo:
A look ahead at Firefly Video and Adobe’s plans
Adobe sees the Firefly Video Model as part of a broader push to integrate AI into creative workflows. The model is designed to handle various use cases, from generating 2D and 3D animations to creating atmospheric elements like smoke and fire.
Still’s blog post did not mention Adobe’s earlier stated and previewed plans to add rival AI video models from other companies — such as OpenAI’s Sora and Runway’s Gen-3 Alpha — to its Premiere Pro video editor software. This makes me wonder if Adobe is rethinking the approach given the increased competition in the space.
Adobe envisions that these tools will give creators more time to explore new ideas, enhance their projects, and ultimately deliver better results for their clients.
By incorporating generative AI into its suite of video editing tools, Adobe is positioning Firefly Video as an essential part of the modern editor’s toolkit—empowering creators to elevate their projects with the help of cutting-edge technology.
Source link lol
