Google has teased an AI-based video generation tool, but it’s not clear when — or if — anyone outside the search giant will be able to kick the tires. It’s certainly fun to look at, though.
On Wednesday, Google’s Research arm released a video highlighting this new text-to-video model, which is called Lumiere.
In a LinkedIn post, team leader Inbar Mosseri said the tool “generates coherent, high-quality videos using simple text prompts” that New Atlas says run up to five seconds. Sample inputs include, “A fluffy baby sloth with an orange knitted hat trying to figure out a laptop” and “An escaped panda eating popcorn in the park.”
In the year or so that generative AI has been the hottest technology going, much of the attention has been focused on tools like ChatGPT that produce text answers to prompts, or those like Dall-E that create still images. Video creation from text prompts is arguably the next frontier, so if Lumiere really can “demonstrate state-of-the-art text-to-video generation results” as Google says, we may already be evolving beyond the “grotesque abominations” of the AI-generated images of 2023.
As the video illustrates, Lumiere’s capabilities include text-to-video and image-to-video generation, as well as stylized generation — that is, using an image to create videos in a similar style. Other tricks include the ability to fill in any missing visuals within a video clip.
That includes the ability to animate famous paintings, like Van Gogh’s Starry Night (“A timelapse oil painting of a starry night with clouds moving”) or Da Vinci’s Mona Lisa (“A woman looking tired and yawning”). While the Starry Night example works almost flawlessly, Mona Lisa looks far more like she’s laughing than yawning.
And while many of the animals — such as “a muskox grazing on beautiful wildflowers” and “a happy elephant wearing a birthday hat walking under the sea” — look realistic, there’s something off about some of the dogs. Both a toy poodle riding a…
Read the full article here