According to the paper thanks to “computationally efficient components, new techniques, and a new custom data set,” researchers at DeepMind say their best-performing model — Dual Video Discriminator GAN (DVD-GAN) — can generate coherent 256 x 256-pixel videos of “notable fidelity” up to 48 frames in length. And that makes it one of the world’s best – even though this is still a nascent area of research.
“Generation of synthetic video is an obvious challenge for generative modelling, but one that is plagued by increased data complexity and computational requirements,” wrote the coauthors. “For this reason, much prior work on synthetic video generation has revolved around relatively simple data sets, or tasks where strong temporal conditioning information is available. We focus on the tasks of video synthesis and video prediction … and aim to extend the strong results of generative image models to the video domain.”
… and it can go on creating them forever …
The team built their system using a cutting-edge AI architecture and introduced video specific tweaks that enabled it to train on Kinetics-600, a data set of natural videos “an order of magnitude” larger than commonly used corpora. Specifically, the researchers leveraged scaled-up Generative Adversarial Networks, or GANs which consist of two parts – an AI systems consisting of generators that generate the video samples and AI discriminators that attempt to distinguish between the generated samples and real-world samples to see how well they compare.
DVD-GAN contains dual discriminators: a spatial discriminator that critiques a single frame’s content and structure by randomly sampling full-resolution frames and processing them individually, and a temporal discriminator that provides a learning signal to generate movement. A separate module, a “Transformer” then lets the learned information propagate across the entire AI model, which then improves it.
As for the training data set it was made up of over 500,000 10 second long high-resolution YouTube clips that the researchers described as “diverse” and “unconstrained.” They then added that after being trained on Google’s specialist Tensor Processing Units for between 12 and 96 hours their DVD-GAN managed to create videos “with object composition, movement, and even complicated textures like the side of an ice rink.”
“We further wish to emphasise the benefit of training generative models on large and complex video data sets, such as Kinetics-600,” wrote the co-authors. “We envisage the strong baselines we established on this data set with DVD-GAN will be used as a reference point by the generative modelling [synthetic content] community moving forward. While much remains to be done before realistic videos can be consistently generated in an unconstrained setting, we believe DVD-GAN is a big step in that direction.”
Matthew Griffin, described as “The Adviser behind the Advisers” and a “Young Kurzweil,” is the founder and CEO of the World Futures Forum and the 311 Institute, a global Futures and Deep Futures consultancy working between the dates of 2020 to 2070, and is an award winning futurist, and author of “Codex of the Future” series.
Regularly featured in the global media, including AP, BBC, Bloomberg, CNBC, Discovery, RT, Viacom, and WIRED, Matthew’s ability to identify, track, and explain the impacts of hundreds of revolutionary emerging technologies on global culture, industry and society, is unparalleled. Recognised for the past six years as one of the world’s foremost futurists, innovation and strategy experts Matthew is an international speaker who helps governments, investors, multi-nationals and regulators around the world envision, build and lead an inclusive, sustainable future.
A rare talent Matthew’s recent work includes mentoring Lunar XPrize teams, re-envisioning global education and training with the G20, and helping the world’s largest organisations envision and ideate the future of their products and services, industries, and countries.
Matthew's clients include three Prime Ministers and several governments, including the G7, Accenture, Aon, Bain & Co, BCG, Credit Suisse, Dell EMC, Dentons, Deloitte, E&Y, GEMS, Huawei, JPMorgan Chase, KPMG, Lego, McKinsey, PWC, Qualcomm, SAP, Samsung, Sopra Steria, T-Mobile, and many more.
FANATICALFUTURIST PODCAST! Hear about ALL the latest futures news and breakthroughs!SUBSCRIBE
1000's of articles about the exponential future, 1000's of pages of insights, 1000's of videos, and 100's of exponential technologies: Get The Email from 311, your no-nonsense briefing on all the biggest stories in exponential technology and science.