• Most new users don't bother reading our rules. Here's the one that is ignored almost immediately upon signup: DO NOT ASK FOR FANEDIT LINKS PUBLICLY. First, read the FAQ. Seriously. What you want is there. You can also send a message to the editor. If that doesn't work THEN post in the Trade & Request forum. Anywhere else and it will be deleted and an infraction will be issued.
  • If this is your first time here please read our FAQ and Rules pages. They have some useful information that will get us all off on the right foot, especially our Own the Source rule. If you do not understand any of these rules send a private message to one of our staff for further details.
  • Please read our Rules & Guidelines

    Vote now in wave 1 of the FEOTM Reboot!

Anyone experimenting with AI Voice Generation?

henzINNIT

Well-known member
Faneditor
Messages
387
Reaction score
121
Trophy Points
53
Couldn't find a thread on this. I'm super curious about the potential to generate dialogue. I have seen some impressive things around the interwebs, even some fan edit stuff, but I haven't seen much chatter about it here.

What are your thoughts on AI voices? Anybody worked with any software? Had any good results?

I'd love to know. I'm thinking a Patrick Stewart bot could be very useful to a project of mine, reading up on it.
 
I'm not a fan of AI in most cases. I don't like lip syncing or backing tracks at concerts or holograms of musicians or (the idea of) scripts written by AI etc. Seems like soulless garbage to me. That being said, surely it has some upside. I know some youtubers that are already using it for celebrity guest appearances or spoofs. That's kind of funny.

I've thought about using if for one specific scene to change literally ONE WORD of dialogue in a scene but I haven't looked into it much yet. I will be looking into it and I'll probably use it at some point for the scene I mentioned. I'm a little surprised it hasn't really taken off around here. I thought the fan edit community would be buzzing about this new AI stuff.
 
What are your thoughts on AI voices? Anybody worked with any software?

Based this whole edit on AI generated Forrest Gump style narration:
It wouldn't work without it.

Opening scene clip:

In my "Kombat Bloody Kombat" edit there are some AI generated dialogue lines added for Sub-Zero. He is masked all the time so lip syncing isn't an issue.
 
Last edited:
I've been using AI voice cloning a lot in my edits to change/add lines when I need to. I've worked out a pretty effective workflow for it.
First you have to comb through what your editing and make a superset of every line that the character you're cloning says, avoiding lines with other people talking over them.
Then use a program like Spleeter to isolate the voice from any music or sound fx, and comb through it again to remove any lines that are unclear or still have background noise. Clarity is super important here.
When you have your final voice samples, which should hopefully be close to 5mins long, you can upload them straight to the voice cloner (I use Elevenlabs AI), and from there you can type in the line you want it to read and tweak the settings until it sounds how you want it to.

The major downside of voice cloning right now is that it takes a lot of time to get a short line out of it. You'll have to generate a lot of 'takes' to get the line you want and you'll probably end up having to combine two or more takes for a single line in the end. Especially if you're trying to lipsynch it to the original footage. It's not a quick and easy process.
I've seen people suggest using AI for narration before, but that seems like it would be a nightmare to me. Unless you want the narration to be purposefully wooden, you'd probably have to generate it sentence-by-sentence and you'll need multiple takes for each sentence.
It's woth trying for the sake of advancing the craft, but you'd have to be super dedicated.
 
I think Vader would be more effective in Rogue One if he didn't talk at all, but if his lines are retained, AI should definitely replace the elderly James Earl Jones' recordings.
 
James Earl Jones didn't voice Vader in Rogue One; he allowed for his lines to be done via AI (from memory, the company 'Respeecher' got the job).


I think Vader would be more effective in Rogue One if he didn't talk at all, but if his lines are retained, AI should definitely replace the elderly James Earl Jones' recordings.
 
I've seen people suggest using AI for narration before, but that seems like it would be a nightmare to me. Unless you want the narration to be purposefully wooden, you'd probably have to generate it sentence-by-sentence and you'll need multiple takes for each sentence.

Of course you need to to generate it sentence-by-sentence and with multiple takes.
And it will still be wooden most of the time. But as a non native english speaker I cant really say how wooden it really is, so it works just good enough for me. And since this particular narration was supposed to be funny, it worked for most of the viewers even in that form (see the reviews).

And yeah, I guess speech-to-speech will give better results.
 
Last edited:
James Earl Jones didn't voice Vader in Rogue One; he allowed for his lines to be done via AI (from memory, the company 'Respeecher' got the job).
Do you have a reference for that? My understanding is JEJ provided the voice and was processed in a manner similar to that of the OT to create the Vader effect in R1. But AI voice processing was used in Kenobi. Kenobi is still JEJ’s voice but the AI processing is similar for voice as the de-aging faces done for Luke or Indiana Jones where earlier references are used to bring the new performance more in line with the younger character.
 
James Earl Jones didn't voice Vader in Rogue One; he allowed for his lines to be done via AI (from memory, the company 'Respeecher' got the job).
Nope. He performed it. He just sounds old, which is probably why you think it's not him. In Kenobi they simply altered his voice to sound younger.
I must say I find the Rogue One performance off-putting too. AI to fix his performance like they did on Kenobi would instantly improve Rogue One.
 
@Moe_Syzlak @DonkeyKonga

I suppose it depends upon which article you read! In Variety and Vanity Fair, they state that JEJ signed-off on Respeecher using archived audio to create the new dialogue for Rogue One:



 
@Moe_Syzlak @DonkeyKonga

I suppose it depends upon which article you read! In Variety and Vanity Fair, they state that JEJ signed-off on Respeecher using archived audio to create the new dialogue for Rogue One:



Rogue one was released in 2016, this article is from 2022/2023 and do not mention R1? Or am I blind?
 
@Moe_Syzlak @DonkeyKonga

I suppose it depends upon which article you read! In Variety and Vanity Fair, they state that JEJ signed-off on Respeecher using archived audio to create the new dialogue for Rogue One:



Those are all referencing the AI work for Kenobi in articles dated 2022. I have yet to hear of any AI work done to the Rogue One Vader which was released six years earlier.

I definitely think Rogue One is a good candidate for a SE in order to make use this technology and more advanced AI deepfake for Leia and Tarkin.
 
On a side note, I wish Earl Jones's voice would not be used for Vader if he doesn't want to do it anymore. I'd rather have a new voice actor given the opportunity.
 
I definitely think Rogue One is a good candidate for a SE in order to make use this technology and more advanced AI deepfake for Leia and Tarkin.
A YouTuber did a pretty good job of improving Rogue One's deepfakes.


IIRC, he wound up getting a job at LucasFilm based on this and his fixing of Mandalorian Luke Skywalker.
 
Which is not a subject of this discussion.
 
Couldn't find a thread on this. I'm super curious about the potential to generate dialogue. I have seen some impressive things around the interwebs, even some fan edit stuff, but I haven't seen much chatter about it here.

What are your thoughts on AI voices? Anybody worked with any software? Had any good results?

I'd love to know. I'm thinking a Patrick Stewart bot could be very useful to a project of mine, reading up on it.
I did use a rather crude AI voice for my Spider-Man edit. It was a phone call with a hospital employee. It's not perfect, but after some tweaking it was good enough for a fanedit. I just used some generic voice available on a website lol.
 
Did the same for some dialogue in "Machete Sharpened". Seagal is on the phone and there's some random generated voice used for a woman. Then she speaks in the same voice in her scene but only when her lips are not visible.
And in "Kombat Bloody Kombat" I used two lines of Morgan Freeman's AI generated voice during phone call in additional scene where he's informing Sonya that "we have a problem".
 
I've been using AI voice cloning a lot in my edits to change/add lines when I need to. I've worked out a pretty effective workflow for it.
First you have to comb through what your editing and make a superset of every line that the character you're cloning says, avoiding lines with other people talking over them.
Then use a program like Spleeter to isolate the voice from any music or sound fx, and comb through it again to remove any lines that are unclear or still have background noise. Clarity is super important here.
When you have your final voice samples, which should hopefully be close to 5mins long, you can upload them straight to the voice cloner (I use Elevenlabs AI), and from there you can type in the line you want it to read and tweak the settings until it sounds how you want it to.

The major downside of voice cloning right now is that it takes a lot of time to get a short line out of it. You'll have to generate a lot of 'takes' to get the line you want and you'll probably end up having to combine two or more takes for a single line in the end. Especially if you're trying to lipsynch it to the original footage. It's not a quick and easy process.
I've seen people suggest using AI for narration before, but that seems like it would be a nightmare to me. Unless you want the narration to be purposefully wooden, you'd probably have to generate it sentence-by-sentence and you'll need multiple takes for each sentence.
It's woth trying for the sake of advancing the craft, but you'd have to be super dedicated.

Very useful. Thanks for posting!

I'm thinking Patrick Stewart could be a solid candidate. He has a bunch of audio books which should provide great sources for a cloner. Will definitely have a go using this advice.

Re: Narration. Seems like a big ask to generate a convincing monologue, but the potential is pretty staggering if you could.
 
Ahh okay! It looks like I was thinking of the wrong show haha! :LOL:

Those are all referencing the AI work for Kenobi in articles dated 2022. I have yet to hear of any AI work done to the Rogue One Vader which was released six years earlier.

I definitely think Rogue One is a good candidate for a SE in order to make use this technology and more advanced AI deepfake for Leia and Tarkin.
 
Back
Top Bottom