• Most new users don't bother reading our rules. Here's the one that is ignored almost immediately upon signup: DO NOT ASK FOR FANEDIT LINKS PUBLICLY. First, read the FAQ. Seriously. What you want is there. You can also send a message to the editor. If that doesn't work THEN post in the Trade & Request forum. Anywhere else and it will be deleted and an infraction will be issued.
  • If this is your first time here please read our FAQ and Rules pages. They have some useful information that will get us all off on the right foot, especially our Own the Source rule. If you do not understand any of these rules send a private message to one of our staff for further details.
  • Please read our Rules & Guidelines

    Read BEFORE posting Trades & Request

Deepfake AI Tech Revolutionize Fan Edits

It is now possible to create a video of an actor delivering a monologue that he never delivered in real life. Case in point:

 
TFW you realize Adam Sandler introduced you to Deep Fake technology:

 
There's definitely a lot of potential for comedy :)

 
Also just the fact that pretty much only one half of the face moves at a time, if their mouths are moving, their eyes/forehead and whatever else are frozen, and vice versa.

Albeit, it is worse when you’re viewing it like this, when you’re watching the movie you stop noticing it within a short time (compared to the length of the movie, that is).
 
The biggest difference/improvement to me was the look of the eyes, age does something recognizable to eyes that the cg people don't fix. Pesci's nose was noticeably better in the deepfake too. At some points DeNiro was unrecognizable as DeNiro in the Netflix half. Overall I'm impressed with how sharp the deepfake looks on the close-ups.

I have not seen the movie, just the above video.
 
I think it’s loads better but still has moments in the uncanny valley. That said, the biggest issues I had with the movie were in the scenes where the actors we’re moving in agile ways which this video conveniently avoids.
 
Thing is though this was made over an already de-aged cast. I wonder how it’d be on the current aged real life versions
 
Masirimso17 said:
Thing is though this was made over an already de-aged cast. I wonder how it’d be on the current aged real life versions
Click to expand...

Does it matter? To me it’s not about choosing one technique or the other; it’s about getting to an acceptable final product, which I feel The Irishman did not achieve.
 
I just saw a trailer for the new Star Wars: Battlefront 2 game (Rise of Skywalker expansion).
In my opinion, most of the clothing, hair, helmets, environments, and spaceships are detailed enough to use in a RoS fanedit. The faces are not, but with a deepfake replacement with the real actors faces they probably would be. They are probably not usable for dialogue scenes but for action scenes (with the appropriate change in film grain). 

What do you think? Would it be worth a fan editor's $23.99 if they were trying to add new scenes to an edit? Or would it just be a waste of money because only the briefest moments wouldn't stick out?
 
While they look nice, they are extremely noticeable when put next to real live footage. You could possibly use some far space shots, but it still would look a bit off as there are always added colors that just don't fit with the movie, although tROS does have a fair amount of blue in it. You also have to deal with the difference in frame rates that creates a very different visual look. In short, I'd save your shekels unless you plan to actually play the game for fun.
 
Was going to make a separate thread - but this seems to fit here. There is also a whole parallel track of deepfaking voices, which has made insane leaps in the last year or two - much like visual deepfakes.

Most of the people/companies who have cracked it are keeping it to themselves or offering paid services (Resemble, Lyrebird etc). The main guy behind Resemble though has published a few papers and still has hosted a now-abandoned Github repo that can recreate voices from 3-4 five seconds clips of them talking and let you text to speech with it. It's nuts, but has some limitations and i'm currently experimenting with it for a GOT Season 8 Fanedit i'm working on.

Currently - there is not a directable way you can change intonation or inflection, instead the result is very 'matter of fact' as the network has been trained on a database of thousands of audiobook narrations. The text parser cannot process punctuation, only line breaks for pauses, so you have to work around that. I have also found (much to my chagrin for the british accents in GOT and things like Star Wars and LOTR for example) that it only seems to be able to replicate mainly american accents.

It will work with whatever you give it, but it will often 'americanize' the output, probably because so many audiobooks have american narrators. With that in mind though, it can be scarily convincing if you have clean audio to feed it, which we often do as nerdy film fans. Not to mention many films and characters speak with an american accent already.

If you are in this thread, then the benefit should be as obvious to you as to me. New lines of dialogue that didn't exist before, recitations of lines from scripts that didn't make it, absolute hilarious garbage. My main aim with this if I can ever get round the accent issue is to have one or two original lines and some lines from the ASOIAF books ADR-ed into my fanedit of S8 of GOT, cinematically over landscape shots/flashbacks etc. You obviously have to be a bit creative, not having the visual of the actor speaking.

You have to do a very specific install and have an NVIDIA graphics card with at least 2GB ram as it runs exclusively on your GPU. It is quicker to process less words, but can process a lot if you have time. For small ones, one or two lines - it only takes about as long as it would take the lines to be read, maybe twice as long - which is close to real time. You don't have to have your computer screaming computationally for 30 hours as you have to with some visual deepfake things.

Because the install order and install versions of everything are vital to get this working, I highly recommend this guide that I followed as was able to get working as someone who is massively interested in AI, but has no practical knowledge or skills.

Foolproof Guide

God speed fan editors and be responsible with your power to have actors say all kinds of ridiculous and/or cool things!
 

^ If any studio exec has seen this clip... the remake has already been greenlit :D .
 
When you can already do a fake AI voice this accurate...


...and a deepfake face replacement like this...


...it creates mind-blowing new fanediting possibilities in the next few years as the tech gets better.

For example:

- Put Mark Ruffalo into the first MCU Hulk movie.
- Make all the female Vulcan characters in the Star Trek movies the same character and actor from WoK (as intended).
- Replace the Oracle in the Matrix sequels.
- Christopher Reeve in Superman Returns.
- Fix all the recasting in all the Terminator sequels! :D
 
I was going to make a joke, but then I realized that you said Terminator and not Transformers, so my joke can't work.
 
DeepFaceLab can do full head replacement now (ears, hair, and all).

 
TM2YC said:
...and a deepfake face replacement like this...

Click to expand...

A Gibson in Fury Road should be as old as Gibson was at the time. Get the Gringo would be a good yardstick:

6379_15.jpg


(Ah, hell, I don't believe in Deepfaking anyway. Not for more than brief and subtle cameos at the most.) ;)
 
I've seen other deepfakes attempting to fix Henry Cavill's upper lip in "Justice League" before, but this new one is the most convincing that I've seen so far.

 
You must log in or register to reply here.
Back
Top Bottom