AI can doctor videos to put words in the mouths of speakers
Artificial intelligence can put words right into your mouth. A new system takes a still image of a person and an audio clip, and uses them to create a doctored video of the person speaking the audio. The results are still a little rough around the edges, but the software could soon make realistically fake videos only a single click away.
It works by first identifying facial features using face-recognition algorithms. As the audio clip plays, the system then manipulates the mouth of the person in the still image so that it looks as if they are speaking. Very little pre-processing is required, so all of this can be done in real time.
“The application we’re thinking of is redubbing a video into another language,” says Joon Son Chung at University of Oxford, one of the creators of the system. In the future, the audio from news clips could be automatically translated into another language and the images updated to fit.
This isn’t the first system to automatically adjust images to new audio, but others have needed large amounts of video to work. They would pair up the way a person’s mouth moved when they made different sounds and then use that part of the image in edited footage.
“With the old method, if you’re trying to redub footage of David Cameron, for example, it’s easy because there are hours of video,” says Chung. But for less well-known figures this would be a lot harder because there isn’t the wealth of images to take from. Chung also thinks the new method could be useful for redubbing animations.
At the moment, these systems focus on changing the shape of the mouth, but it’s only a matter of time before they can also change facial expressions and posture, says Alex Champandard, co-founder of creative.ai, which makes artificial intelligence tools for artists.
“Pretty soon we will have trouble distinguishing between real and fake video,” says Champandard. Given enough time, experts can already create fake videos that are virtually indistinguishable from genuine ones. Artificially intelligent tools are making the process so quick and easy, that eventually almost anybody could do it.
This could cast doubt on the reliability of video evidence when it comes to court cases. This could mean more reliance on witness testimony and forensic science. “What does a post AI legal system look like? Nobody knows,” says Champandard.
Journal reference: arXiv:1705.02966
More on these topics: