Meta recently unveiled its own media-focused artificial intelligence model, Movie Gen, which can generate lifelike videos and audio clips. The company shared several 10-second sample videos, including footage of a baby hippo swimming, to demonstrate the capabilities of the new technology.
Movie Gen can not only generate videos from text but also edit existing videos, for example by adding objects or modifying visual elements. In one demonstration, a woman wearing a VR headset was transformed so that she appeared to be wearing steampunk binoculars.
With the Movie Gen model, not only visual content but also audio clips can be generated to accompany videos. The examples shown included a man next to a waterfall with the sound of rushing water and a symphony playing, the deep purr of a sports car engine and the screech of its tyres, and a snake slithering through the jungle with a rustling sound and a suspenseful soundtrack.
Meta shared information about Movie Gen in a detailed research paper. The Movie Gen Video model has 30 billion parameters, while the Audio model has 13 billion. For comparison, the largest version of Llama 3.1 contains 405 billion parameters. Movie Gen can create high-definition videos up to 16 seconds long, and according to Meta it delivers better quality than competing models.
Earlier, Mark Zuckerberg introduced Meta AI’s Imagine Me feature, which allowed users to place their own images in different contexts. A video version of this feature could become possible with Movie Gen.
The exact sources of the data used to train Movie Gen are not known. According to Meta’s announcement, “licensed and publicly available datasets” were used, but the sources of data used to train AI models are often disputed and rarely transparent. It is not yet known when Movie Gen will become widely available, but Meta’s blog post refers to a “potential future release”. OpenAI has also announced its own AI video model, Sora, which is likewise not yet available to the general public.
As Meta is a social media giant, it may eventually integrate Movie Gen into platforms such as Facebook, Instagram or WhatsApp. In September, Google announced that it would introduce some features of its Veo video model into YouTube Shorts over the coming year.
Although the big tech companies have not yet released their video models, smaller startups such as Runway and Pika already offer the chance to try AI-generated videos.