Getting an AI to generate great video is alot harder than getting it to generate great images, but Runway’s founders are up to the challenge.


Love the Exponential Future? Join our XPotential Community, future proof yourself with courses from XPotential University, read about exponential tech and trendsconnect, watch a keynote, or browse my blog.

Artificial Intelligence (AI) startup Runway, which was created by the same people behind the Text-to-image generator MidJourney, has launched its first mobile app on iOS that lets people use the company’s Video-to-Video Generative AI tool – Gen-1 – directly from their phones.


Survey shows images of deepfake faces are now more real than genuine photos


Gen-1 allows you to transform an existing video based on a text, image, or video input. Functionally, it works a lot like a style transfer tool although, unlike style transfer, it generates entirely new videos as an output rather than applying filters. You can upload a video of someone cycling in the park, for example, and apply an aesthetic or theme. You can give the video the look of a watercolor painting or charcoal sketch, and so on.


The Future of Synthetic Content, by keynote Matthew Griffin


Of course, because this is a generative AI, the output is often… strange. If you add a claymation effect, for example, your resulting models won’t function like real claymation. The models will warp between each frame, limbs will grow and shrink, features will melt and smear. That’s all to be expected, though, and doesn’t take away from the fun.

Here, for example, are three different renderings of an iconic clip of Al Pacino in Heat (1995). Most notable is the clip in the bottom right, which uses a picture someone took of a cat as an intermediary.


This innovative AI algorithm finds a way to burst internet echo chambers


Without having to specify the model applied the cat’s face to Pacino’s and even gave his hands a bit of fur while leaving his suit more or less intact. The other two clips on the top row are preset filters.



There’s another example: a video of St. Paul’s Cathedral in London with the “paper and ink” filter applied. It’s not a mind-blowing effect, but it was incredibly easy to make. And in the hands of a more experienced and creative individual, I’m sure it could be spectacular.



People have been testing Runway’s app for a few days now, and it certainly makes the whole process of creating this sort of video much more fluid. Runway’s main software suite is available on the web, which makes the distance between capturing footage and generating it wider. It’s not a seamless experience, of course. There are the usual inefficiencies and unexpected errors you’d expect to find in the first release of an app. But, as Runway CEO Cristóbal Valenzuela told The Verge, making these tools mobile is the important thing.


Chinese researchers show off transistor gates that are just one carbon atom wide


“That’s why the phone makes so much sense because you’re recording directly from your device, and then you tell Gen-1 how to transform that video,” said Valenzuela.

There are other limitations worth mentioning. You can’t work with footage longer than five seconds, and there are certain banned prompts. You can’t generate nudity, for example, and it seems copyright-protected work is off-limits, too. And prompts to create a video “in the style of a Studio Ghibli film” get rejected.

Each video also takes around two to three minutes to create, which doesn’t sound like a lot but feels like an age in the era of instant mobile editing. The processing is done in the cloud and will likely speed up over time. The app only currently supports Runway’s Gen-1 model, but Valenzuela says the purely generative Gen-2 will be added soon.


Tech giants team up to create the world's first Deep Learning standard


What these notes don’t fully capture, though, is the huge sense of possibility of tools like this. The output of AI text-to-image models also started out as smeared and unrealistic. Now they’re being used to fool the public with swagged-out pictures of the pope.

Valenzuela has compared the current era of generative AI to the “optical toys” phase of the 19th century, when scientists and inventors were creating a whole range of devices that were trivial in their capabilities but also the ancestors of modern cameras.

Runway’s mobile app feels like one of these toys, but within two years some of Hollywood’s most famous directors like Jim Russo believe these apps will be good enough to make “movies that give you pause.”

About author

Matthew Griffin

Matthew Griffin, described as “The Adviser behind the Advisers” and a “Young Kurzweil,” is the founder and CEO of the World Futures Forum and the 311 Institute, a global Futures and Deep Futures consultancy working between the dates of 2020 to 2070, and is an award winning futurist, and author of “Codex of the Future” series. Regularly featured in the global media, including AP, BBC, Bloomberg, CNBC, Discovery, RT, Viacom, and WIRED, Matthew’s ability to identify, track, and explain the impacts of hundreds of revolutionary emerging technologies on global culture, industry and society, is unparalleled. Recognised for the past six years as one of the world’s foremost futurists, innovation and strategy experts Matthew is an international speaker who helps governments, investors, multi-nationals and regulators around the world envision, build and lead an inclusive, sustainable future. A rare talent Matthew’s recent work includes mentoring Lunar XPrize teams, re-envisioning global education and training with the G20, and helping the world’s largest organisations envision and ideate the future of their products and services, industries, and countries. Matthew's clients include three Prime Ministers and several governments, including the G7, Accenture, Aon, Bain & Co, BCG, Credit Suisse, Dell EMC, Dentons, Deloitte, E&Y, GEMS, Huawei, JPMorgan Chase, KPMG, Lego, McKinsey, PWC, Qualcomm, SAP, Samsung, Sopra Steria, T-Mobile, and many more.

Your email address will not be published. Required fields are marked *