Netflix Enters the AI Video Generation Race with Go-with-the-Flow
New Model Aims to Provide Easy Control over Motion Patterns
Researchers from Netflix have joined forces with other institutions to develop Go-with-the-Flow, a new AI video generator that allows users to control motion patterns in video diffusion models. This comes hot on the heels of Tencent’s HunyuanVideo, which also enables users to control object movement.
How Go-with-the-Flow Works
Go-with-the-Flow lets users define a trajectory for an object by dragging and cutting, similar to a mask in video editing software. The model first creates a crude animation, then adds warped noise to turn it into a smooth animation. The results may appear janky, particularly with people and animals, but it’s still a notable advance.
Key Features
- Allows users to animate an object by selecting an object and defining a trajectory
- Camera control and motion transfer from one video to another
- Uses warped noise instead of pure IID Gaussian noise
- Can create turntable animations using a 3D-rendered turntable camera motion as a guide
Availability and Code
The code for Go-with-the-Flow is available on GitHub and Hugging Face. A tutorial on the AI video model can be found below.
Netflix’s Involvement
Netflix’s involvement in the project is intriguing, as the company has already announced its work on using generative AI to make games. This project adds weight to suspicions that Netflix may be working on AI for video generation as well. This could take the form of using AI to make games or creating AI-generated shows.
Conclusion
Go-with-the-Flow is a significant step forward in AI video generation, allowing users to control motion patterns with ease. Netflix’s involvement in the project raises questions about the company’s plans for AI video generation. As the technology continues to evolve, we can expect to see more innovative applications of AI in the video generation space.
FAQs
Q: What is Go-with-the-Flow?
A: Go-with-the-Flow is a new AI video generator developed by researchers from Netflix and other institutions that allows users to control motion patterns in video diffusion models.
Q: How does Go-with-the-Flow work?
A: Go-with-the-Flow lets users define a trajectory for an object by dragging and cutting, similar to a mask in video editing software. The model first creates a crude animation, then adds warped noise to turn it into a smooth animation.
Q: What are the key features of Go-with-the-Flow?
A: The key features of Go-with-the-Flow include allowing users to animate an object, camera control, motion transfer from one video to another, and the ability to create turntable animations.
Q: Is the code for Go-with-the-Flow available?
A: Yes, the code for Go-with-the-Flow is available on GitHub and Hugging Face.
