• Most new users don't bother reading our rules. Here's the one that is ignored almost immediately upon signup: DO NOT ASK FOR FANEDIT LINKS PUBLICLY. First, read the FAQ. Seriously. What you want is there. You can also send a message to the editor. If that doesn't work THEN post in the Trade & Request forum. Anywhere else and it will be deleted and an infraction will be issued.
  • If this is your first time here please read our FAQ and Rules pages. They have some useful information that will get us all off on the right foot, especially our Own the Source rule. If you do not understand any of these rules send a private message to one of our staff for further details.
  • Please read our Rules & Guidelines

Deepfake AI Tech Revolutionize Fan Edits

hbenthow

Well-known member
Messages
1,598
Reaction score
168
Trophy Points
68
It is now possible to create a video of an actor delivering a monologue that he never delivered in real life. Case in point:

 

Nic

Well-known member
Messages
1,695
Reaction score
104
Trophy Points
68
TFW you realize Adam Sandler introduced you to Deep Fake technology:

 

DonKamillo

Well-known member
Faneditor
Messages
401
Reaction score
218
Trophy Points
53
There's definitely a lot of potential for comedy :)

 

macmilln

Well-known member
Messages
576
Reaction score
60
Trophy Points
53
Also just the fact that pretty much only one half of the face moves at a time, if their mouths are moving, their eyes/forehead and whatever else are frozen, and vice versa.

Albeit, it is worse when you’re viewing it like this, when you’re watching the movie you stop noticing it within a short time (compared to the length of the movie, that is).
 

addiesin

Well-known member
Messages
5,889
Reaction score
1,502
Trophy Points
163
The biggest difference/improvement to me was the look of the eyes, age does something recognizable to eyes that the cg people don't fix. Pesci's nose was noticeably better in the deepfake too. At some points DeNiro was unrecognizable as DeNiro in the Netflix half. Overall I'm impressed with how sharp the deepfake looks on the close-ups.

I have not seen the movie, just the above video.
 

Moe_Syzlak

Well-known member
Messages
3,454
Reaction score
1,165
Trophy Points
118
I think it’s loads better but still has moments in the uncanny valley. That said, the biggest issues I had with the movie were in the scenes where the actors we’re moving in agile ways which this video conveniently avoids.
 

Masirimso17

Well-known member
Cover Artist
Messages
2,138
Reaction score
238
Trophy Points
93
Thing is though this was made over an already de-aged cast. I wonder how it’d be on the current aged real life versions
 

Moe_Syzlak

Well-known member
Messages
3,454
Reaction score
1,165
Trophy Points
118
Masirimso17 said:
Thing is though this was made over an already de-aged cast. I wonder how it’d be on the current aged real life versions

Does it matter? To me it’s not about choosing one technique or the other; it’s about getting to an acceptable final product, which I feel The Irishman did not achieve.
 

iridium_ionizer

Well-known member
Messages
94
Reaction score
0
Trophy Points
6
I just saw a trailer for the new Star Wars: Battlefront 2 game (Rise of Skywalker expansion).
In my opinion, most of the clothing, hair, helmets, environments, and spaceships are detailed enough to use in a RoS fanedit. The faces are not, but with a deepfake replacement with the real actors faces they probably would be. They are probably not usable for dialogue scenes but for action scenes (with the appropriate change in film grain). 

What do you think? Would it be worth a fan editor's $23.99 if they were trying to add new scenes to an edit? Or would it just be a waste of money because only the briefest moments wouldn't stick out?
 

DigModiFicaTion

DᴉმWoqᴉԷᴉcɑꓕᴉou
Staff member
Faneditor
Messages
8,609
Reaction score
3,508
Trophy Points
168
While they look nice, they are extremely noticeable when put next to real live footage. You could possibly use some far space shots, but it still would look a bit off as there are always added colors that just don't fit with the movie, although tROS does have a fair amount of blue in it. You also have to deal with the difference in frame rates that creates a very different visual look. In short, I'd save your shekels unless you plan to actually play the game for fun.
 

Mizukitron

New member
Messages
1
Reaction score
0
Trophy Points
1
Was going to make a separate thread - but this seems to fit here. There is also a whole parallel track of deepfaking voices, which has made insane leaps in the last year or two - much like visual deepfakes.

Most of the people/companies who have cracked it are keeping it to themselves or offering paid services (Resemble, Lyrebird etc). The main guy behind Resemble though has published a few papers and still has hosted a now-abandoned Github repo that can recreate voices from 3-4 five seconds clips of them talking and let you text to speech with it. It's nuts, but has some limitations and i'm currently experimenting with it for a GOT Season 8 Fanedit i'm working on.

Currently - there is not a directable way you can change intonation or inflection, instead the result is very 'matter of fact' as the network has been trained on a database of thousands of audiobook narrations. The text parser cannot process punctuation, only line breaks for pauses, so you have to work around that. I have also found (much to my chagrin for the british accents in GOT and things like Star Wars and LOTR for example) that it only seems to be able to replicate mainly american accents.

It will work with whatever you give it, but it will often 'americanize' the output, probably because so many audiobooks have american narrators. With that in mind though, it can be scarily convincing if you have clean audio to feed it, which we often do as nerdy film fans. Not to mention many films and characters speak with an american accent already.

If you are in this thread, then the benefit should be as obvious to you as to me. New lines of dialogue that didn't exist before, recitations of lines from scripts that didn't make it, absolute hilarious garbage. My main aim with this if I can ever get round the accent issue is to have one or two original lines and some lines from the ASOIAF books ADR-ed into my fanedit of S8 of GOT, cinematically over landscape shots/flashbacks etc. You obviously have to be a bit creative, not having the visual of the actor speaking.

You have to do a very specific install and have an NVIDIA graphics card with at least 2GB ram as it runs exclusively on your GPU. It is quicker to process less words, but can process a lot if you have time. For small ones, one or two lines - it only takes about as long as it would take the lines to be read, maybe twice as long - which is close to real time. You don't have to have your computer screaming computationally for 30 hours as you have to with some visual deepfake things.

Because the install order and install versions of everything are vital to get this working, I highly recommend this guide that I followed as was able to get working as someone who is massively interested in AI, but has no practical knowledge or skills.

Foolproof Guide

God speed fan editors and be responsible with your power to have actors say all kinds of ridiculous and/or cool things!
 

TM2YC

Take Me To Your Cinema
Staff member
Donor
Faneditor
Messages
14,870
Reaction score
2,384
Trophy Points
228

^ If any studio exec has seen this clip... the remake has already been greenlit :D .
 

TM2YC

Take Me To Your Cinema
Staff member
Donor
Faneditor
Messages
14,870
Reaction score
2,384
Trophy Points
228
When you can already do a fake AI voice this accurate...


...and a deepfake face replacement like this...


...it creates mind-blowing new fanediting possibilities in the next few years as the tech gets better.

For example:

- Put Mark Ruffalo into the first MCU Hulk movie.
- Make all the female Vulcan characters in the Star Trek movies the same character and actor from WoK (as intended).
- Replace the Oracle in the Matrix sequels.
- Christopher Reeve in Superman Returns.
- Fix all the recasting in all the Terminator sequels! :D
 

Jrzag42

Well-known member
Faneditor
Messages
4,000
Reaction score
1,019
Trophy Points
138
I was going to make a joke, but then I realized that you said Terminator and not Transformers, so my joke can't work.
 

hbenthow

Well-known member
Messages
1,598
Reaction score
168
Trophy Points
68
DeepFaceLab can do full head replacement now (ears, hair, and all).

 

Gaith

Well-known member
Faneditor
Messages
5,785
Reaction score
291
Trophy Points
123
TM2YC said:
...and a deepfake face replacement like this...


A Gibson in Fury Road should be as old as Gibson was at the time. Get the Gringo would be a good yardstick:

6379_15.jpg


(Ah, hell, I don't believe in Deepfaking anyway. Not for more than brief and subtle cameos at the most.) ;)
 

hbenthow

Well-known member
Messages
1,598
Reaction score
168
Trophy Points
68
I've seen other deepfakes attempting to fix Henry Cavill's upper lip in "Justice League" before, but this new one is the most convincing that I've seen so far.

 
Top Bottom