Real-Time DeepFake Streaming With DeepFaceLive

The world's most-used deepfake software, and the one most strongly associated with deepfake porn, is getting a new implementation – the ability to superimpose deepfakes of a celebrity likeness into live webcam footage.

The open source repository DeepFaceLive is a Windows 10 application built on the foundations of DeepFaceLab, which quickly became the leading resource for porn-based face-swapping after the creator of the original deepfake software abandoned the project shortly after the advent of deepfaking created worldwide controversy in 2018. Both projects list the forum section of deepfake porn site mrdeepfakes as an official support resource.

In a video of the DeepFaceLive interface released by developer Ivan Petrov in April, we see downloaded footage of impersonator Miles Fisher being transformed into Tom Cruise in real time. Source: https://martinanderson.substack.com/p/real-time-deepfake-streaming-is-apparently

Yesterday the lead on both projects, Russian developer Ivan Petrov, announced a new version of DeepFaceLive on the project's Discord group, noting that DeepFaceLive now incorporates a trained model of actress Margot Robbie, a frequent subject of pornographic videos at mrdeepfakes.

Four sample video clips have been uploaded to demonstrate the ability of DeepFaceLive to attach the Robbie identity to any speaker in a webcam feed.

One of the Margot Robbie demonstrations of live deepfaking. Source: https://github.com/iperov/DeepFaceLive/tree/master/doc/celebs/Margot_Robbie

The Robbie transformation videos were originally shared on Twitch, but subsequently deleted.

The three other examples of faceswapping provided by DeepFaceLive. The requirement for an approximate face match with the target subject is still in evidence. In the bottom left image we see that the software cannot effectively mask obstructions, as can be achieved in DeepFaceLab, since this requires training masks in the XSeg application of DeepFaceLab, a method that can't be applied to real-time faking.

An additional default model, of British comedian and actor Rowan Atkinson, is promised to be made available for DeepFaceLive soon. Originally the DeepFaceLive repository featured a CELEB_MODEL category, where it was possible to download other celebrities, including Tom Cruise, as Open Neural Network Exchange (.onnx) files, though these directories have since been deleted.

Tom Cruise was an incorporated identity for a DeepFaceLive beta back in April of 2021, but has since apparently been removed as a default in the package. However, the model can be downloaded and installed, along with trained models from many other celebrities, and custom models trained by end users.

DeepFaceLive runs exclusively as a Windows 10 application, and is reported by one user to work well with NVIDIA's live streaming Broadcast application.

Obtaining Identities

Users do not need to go through the laborious process of gathering face material for a dataset, followed by the 3-10 days necessary to obtain a very good match, since the FAQ for DeepFaceLive states that it can use off the shelf ‘public face models' that have already been compiled by other creators. In effect, the largest source of such models is the Forum-Trained Models (archive link) thread at the mrdeepfakes porn site, which can be seen in use by one Russian end-user in a recent YouTube video.

Vladislav Pedro using DeepFaceLive to transform himself into Tom Cruise in real time. The model he loads in the video is hosted at mrdeepfakes, and has been used until now for offline deepfake processing. Source: https://www.youtube.com/watch?v=0TcCKtObSnI

Though the best results will be obtained by training the end-user's face against the target celebrity, the quality of the simulation seems to be in direct proportion to how closely the end user resembles the target. When the user is relatively well-fitted to the celebrity, a convincing result can be obtained.

As is evident in the last three Margot Robbie swaps in the images above, the software can't perform miracles where the end-user's face has radically different general characteristics. However, DeepFaceLab can also perform entire head swaps, a process which completely overwrites the source head, albeit with limitations in regard to hairstyles. Comments on the Discord server confirm that these can also be implemented in DeepFaceLive.

DeepFaceLive currently supports only NVIDIA GPUs, with the GTX 750 the lowest hardware that can obtain a reasonable result with the application. On a graphics card with 4GB of VRAM, it also requires 32GB of swap disk space.

Training the Margot Robbie model – an image posted at the DeepFaceLab/DeepFaceLive Discord by Ivan Petrov. Here we see that the source dataset is extremely diverse, apparently with the intent of creating a robust and well-generalized model that can transform a wide range of facial types into the image of the Australian actress. Most ‘A' sets are usually of a single individual.

Users report that DeepFaceLive is extremely demanding in terms of hardware resources, but Petrov has responded to this by asserting that DeepFaceLive is intended for well-specced PCs in an indoor environment.

The DeepFaceLab GitHub has been forked over 6,000 times, while the new DeepFaceLive currently has 85 forks.

Possible Uses

Commenters in a thread (Russian language) at Russian social site VK.com speculate on the possible uses of DeepFaceLive, and seem to have discerned all the likely applications for the software: as an ‘anonymization' or recreational filter in Zoom and similar applications; as a means for committing fraud or deceiving the person watching the live stream; and – perhaps most likely, in light of the software's provenance and user-base – as a means of implementing celebrity-focused webcam-based sex work (see image at end of article).

The possibility of ‘deepfake cam girls' emerged in the media a couple of years ago. In 2020 the Avatarify app announced deepfake celebrity webcam functionality, but at a quality significantly lower than the standard set by DeepFaceLab, which is now the repository of choice for recent headline-grabbing Tom Cruise deepfaker Chris Ume, as well as high-level film and TV production companies around the world.

DeepFaceLab, the seed application for DeepFaceLive and the reported (Russian language) framework for 95% of all the deepfakes on YouTube, is also the software used by Shamook, the deepfaker whose extraordinary work on improving Luke Skywalker's guest appearance in The Mandalorian recently inspired Lucasfilm to hire him as a VFX consultant.

Adapted To Deepfake Limitations

The methodology of deepfake software such as DeepFaceLab and FaceSwap is well-served by a typical webcam environment, and removes many of the obstacles to authenticity that can still plague off-line deepfakes.

For instance, deepfakes perform less convincingly when the user turns to profile (or looks up), moves rapidly, or is subject to scene changes or sudden changes of lighting.

Therefore the ‘static' nature of a typical Zoom call or webcam plays to the strengths of the software. Many of the most revered examples of deepfake trickery over the past three years have featured speakers facing the camera directly and moving very little in a constrained environment, such as the imposition of Steve Buscemi onto Jennifer Lawrence at an Oscar speech, and the transformation of various ‘to camera' political broadcasts.

Arguably the most likely use of the default Margot Robbie model included in DeepFaceLive – a cam implementation test video published on the DeepFaceLab Discord by Russian user ‘NoFace'.

Amended 8:52pm 7th August – Corrected surname.
Amended 8:30pm 8th August – added images from Discord.