• Most new users don't bother reading our rules. Here's the one that is ignored almost immediately upon signup: DO NOT ASK FOR FANEDIT LINKS PUBLICLY. First, read the FAQ. Seriously. What you want is there. You can also send a message to the editor. If that doesn't work THEN post in the Trade & Request forum. Anywhere else and it will be deleted and an infraction will be issued.
  • If this is your first time here please read our FAQ and Rules pages. They have some useful information that will get us all off on the right foot. More details on our policies, especially our Own the Source rule are available here. If you do not understand any of these rules send a private message to one of our staff for further details.
  • Favorite Edit of the Year (FEOTY) 2020 Awards are here.

Acheiving smoother animation using DAIN, RIFE etc.

The Scribbling Man

Tenant of the Tower of Flints
Staff member
Donor
Faneditor
Messages
3,054
Reaction score
449
Trophy Points
113
So I'm a layman with a lot of the technical side of framerates etc., but bear with/humour me while I lay out my predicament.

So for a while I've been experimenting on and off using DAIN and RIFE and creating new frames/increasing framerate, both as ways of fixing jump frames and also in order to improve animation. You can see examples of people doing similar things using the same software online:

This has been producing some great results for me in particular circumstances, but there are others where I'm a bit stumped as to how to get the desired result/whether the desired result is actually possible. Additionally, examples like the video above are simply of putting entire scenes through the software and doubling the framerate. Playing everything back at 60fps is not the same as achieving smoother motion at the original framerate, if you get what I mean? Plus, I don't actually like the look of high framerates - it feels unnatural in the context of a film.

So here's a hypothetical:

For the sake of maths ease, let's pretend you're looking at a film that's in 30fps. Let's say we have a live action scene with stop motion animation effects, and while the general motion of the film is as smooth as you'd expect it to be, the animation FX are noticeably jittery. You realise that while the live action footage might be playing back at 30fps, the animation fx only move every other frame, so are in effect playing back at 15fps. So, by:

  • dropping the null frames on the animation (effectively doubling the speed)
  • rendering it out at 30fps
  • doubling the framerate of that footage to 60fps
  • importing that footage back in and halving the playback speed to 30fps

...you then get the same animation playing back at the same speed as before, only with the null frames replaced by newly invented ones that allow for visibly smoother motion. This allows you to pair smoother animated footage with the live action footage.

Awesome.

Ok, so here's scenario B, which I also come across and don't know what to do with.

Scenario B is the same as above, except the animation both looks jittery AND is at a framerate consistent with the rest of the footage. AKA every frame shows movement.

So, you can render that animation out and you can double, even quadruple, the framerate with incredible results; but none of this will matter, because your source film is at 30fps and needs to stay at that consistently.

So I guess my question is, if I increase the framerate of a piece of animation in order to make it look slicker, is it then possible to somehow translate that newly achieved "smoothness" back to the original framerate for slotting back into the source footage? Or will I inevitably always be discarding the very same frames that were created to make it smooth in the first place? Is there any way around this? Obviously one can halve the playback rate and get some wonderfully smooth motion from the animation, but it will also be at half the desired speed...
 

Captain Khajiit

Well-known member
Donor
Messages
2,685
Reaction score
7
Trophy Points
48
So, you can render that animation out and you can double, even quadruple, the framerate with incredible results; but none of this will matter, because your source film is at 30fps and needs to stay at that consistently. So I guess my question is, if I increase the framerate of a piece of animation in order to make it look slicker, is it then possible to somehow translate that newly achieved "smoothness" back to the original framerate for slotting back into the source footage?
My instinct is to say no: if the animation is jittery at the frame-rate at which it was composed, it will probably always be jittery at that frame-rate. Smoothing out the motion by doubling the frame-rate through frame-interpolation is no good if the interpolated frames are discarded when conforming the footage to the original frame-rate, unless maybe you truncate (top and tail) the shot instead, allowing the smoother portion that was originally the middle to take the place of the whole shot. That's probably not ideal though.

A couple more ideas:

Approach 1

Decimate the clip in question (30fps to 15fps). Apply frame-interpolation (15fps to 30fps), and hope that the interpolated frames are smoother than the originals (the ones lost in decimation).

Approach 2

Use frame-interpolation to go from 30fps to 15fps (in other words, reducing rather than increasing the frame-rate via interpolation), thus throwing away all the original frames and constructing new (hopefully less jittery) ones. [EDIT: This could be done in AviSynth-based frame-interpolation methods, but I don't know if it's possible with the tools that you are using.] Then use frame-interpolation a second time to go from 15fps to 30fps, the desired frame-rate.

I hope that this gives you a few ideas and that I've correctly understood your problem.
 
Last edited:

The Scribbling Man

Tenant of the Tower of Flints
Staff member
Donor
Faneditor
Messages
3,054
Reaction score
449
Trophy Points
113
Thanks, Cap. Much appreciated. I'll have a look into those options!
 

The Scribbling Man

Tenant of the Tower of Flints
Staff member
Donor
Faneditor
Messages
3,054
Reaction score
449
Trophy Points
113
Approach 1 has actually worked really well. Thanks again for the advice!
 

Captain Khajiit

Well-known member
Donor
Messages
2,685
Reaction score
7
Trophy Points
48
You're most welcome. I'm pleased to hear that one approach worked out well, especially given that both ideas were largely theoretical on my part.
 

Jrzag42

Well-known member
Faneditor
Messages
2,886
Reaction score
235
Trophy Points
88
This video doesn't entirely apply here since you're dealing with stop motion and this is more about 2d animation, but the thread title reminded me of it and I figure it's worth sharing anyways:

Anyways, smoothing out sop motion animation sounds interesting, I look forward to seeing your results if you choose to share them.

I've been a bit curious about the possibility of achieving the opposite. By this I mean, taking a movie with cgi creatures (specifically the creatures in the coliseum in Attack of the Clones), and reducing the frame rate so that they look more jittery like stop motion, while also somehow keeping the live action actors at the original framerate. I have no idea if this would be possible, but it's something I've wanted to look into.
 

The Scribbling Man

Tenant of the Tower of Flints
Staff member
Donor
Faneditor
Messages
3,054
Reaction score
449
Trophy Points
113
I look forward to seeing your results if you choose to share them.

I'll def be sharing :) It's for my next B+ edit which is 90% done and I keep forgetting to announce with an in-the-works thread.

I've been a bit curious about the possibility of achieving the opposite. By this I mean, taking a movie with cgi creatures (specifically the creatures in the coliseum in Attack of the Clones), and reducing the frame rate so that they look more jittery like stop motion, while also somehow keeping the live action actors at the original framerate. I have no idea if this would be possible, but it's something I've wanted to look into.

That's an interesting idea, though I'm not sure reducing the framerate alone would necessarily make it look like stop motion. Certainly possible to try though. The way I've kept my changes separate from the live-action is by having two versions of the clip. The processed version sits above the original in my timeline, and then I mask the FX shot so that the two elements are kept separate. Unless the actors are interacting quite heavily with the FX or the camera is hectic, you shouldn't need to be precise with it; since both clips are more or less the same they should blend seamlessly unless there's any movement that's interrupted. I'm working with an older film that does have a bit of grain and jitter, so I sometimes have to feather it a bit to help it blend, but that shouldn't necessarily be needed with newer, slicker footage.
 

The Scribbling Man

Tenant of the Tower of Flints
Staff member
Donor
Faneditor
Messages
3,054
Reaction score
449
Trophy Points
113
This video doesn't entirely apply here since you're dealing with stop motion and this is more about 2d animation, but the thread title reminded me of it and I figure it's worth sharing anyways:

Thanks for sharing that. The guy makes some good points and I agree with a lot of what he says. I think to some extent it still applies to stop motion, as you can have similar problems (and people are sharing vids with stop motion at high framerates). A lot of what's on youtube has just been thrown into the flow-frame blender and it isn't an improvement and sometimes does look bad (or at least unnatural). I don't agree that any attempt to improve on animation is an "F U" to the artist though.

Personally, I am trying to remain respectful to the original intent and improve the motion without going overboard. Stop motion had its limitations, and sometimes the dropped framerate is a result of time and budget as opposed to intent. I am not throwing every clip through the software, but only portions where I feel the animation becomes noticeably jittery (sometimes literally a second or less) - and even then, I'm being careful to make sure it does what I want. If frames start getting fuzzy/blurred/confusing the motion, then I leave the clip as it is originally - or sometimes (because I'm throwing away some frames and creating new ones) I manually remove botched frames and replace them with the originally ditched frame. The software is clever though and more often than not I've found it creates some amazing motion without anything looking funky. I am also keeping things at the original framerate as opposed to doubling or quadrupling the framerate like many on youtube.

Still, it's controversial and I'm sure there are some who won't like what I'm doing. When I finally share the results i'll be open to criticism. I'm spending a lot of time with the material, so it'll be interesting to know how it comes across to unbiased eyes.
 

Jrzag42

Well-known member
Faneditor
Messages
2,886
Reaction score
235
Trophy Points
88
I've been a bit curious about the possibility of achieving the opposite. By this I mean, taking a movie with cgi creatures (specifically the creatures in the coliseum in Attack of the Clones), and reducing the frame rate so that they look more jittery like stop motion, while also somehow keeping the live action actors at the original framerate. I have no idea if this would be possible, but it's something I've wanted to look into.
I made a rough proof of concept test,
 

The Scribbling Man

Tenant of the Tower of Flints
Staff member
Donor
Faneditor
Messages
3,054
Reaction score
449
Trophy Points
113
@The Scribbling Man , if you manage to perfect this, I'd love to team up with you for a Jason and the Argonauts edit. :)

Funnily enough, my unannounced project is Jason and the Argonauts! It is basically finished, otherwise I would have been happy to team up. All I am doing now is the smoothing of stop motion. Talos was finished recently, and I need to work my way through the harpies and the hydra. I have done a small amount on the skeleton battle, but honestly that sequence already looks so good it just feels sacrilege to touch it. I will likely leave most of that scene as it is.
 

lapis molari

Better edits through feedback.
Staff member
Donor
Faneditor
Messages
1,220
Reaction score
357
Trophy Points
113
Wonderful! Excellent! I look forward to it!

Did you post this on youtube back in March, or is CaptRobau someone else doing remarkable work to achieve smoother animation in Jason?
Interesting editing possibility to improve old stop-motion films. Now I'm thinking about fanediting possibilities for Jason and the Argonauts!
 

The Scribbling Man

Tenant of the Tower of Flints
Staff member
Donor
Faneditor
Messages
3,054
Reaction score
449
Trophy Points
113
^^ no, that's not me. And since the film is at 23.976 fps, the stop motion is not exceeding that framerate in the edit. I really should have posted an in-the-works by now. I'll post one today and try and whip up a video example.
 

The Scribbling Man

Tenant of the Tower of Flints
Staff member
Donor
Faneditor
Messages
3,054
Reaction score
449
Trophy Points
113
I made a rough proof of concept test,

Cool to see in action, though I'm not sure it quite works. Or at least, I think it would need more than just the framerate being dropped to convey a stop motion vibe. The very start when it's just the rhino thing walking out the cave worked reasonably well, but I think the illusion was dropped the moment the screen got busier.
 

asterixsmeagol

Well-known member
Donor
Messages
1,167
Reaction score
165
Trophy Points
73
I'd like to see somebody smooth out the animation on Into the Spider-Verse. I know they used a low frame rate on purpose to help give them a more comic book feel, but it really didn't work for me and I get a bit of a headache watching it.
 

Jrzag42

Well-known member
Faneditor
Messages
2,886
Reaction score
235
Trophy Points
88
Cool to see in action, though I'm not sure it quite works. Or at least, I think it would need more than just the framerate being dropped to convey a stop motion vibe. The very start when it's just the rhino thing walking out the cave worked reasonably well, but I think the illusion was dropped the moment the screen got busier.
Of course, the more I watch it the more I realize just how much work needs to be done. Personally I think the third creature looks the best, the mantis creature definitely looks the worst.
I think some color correction and possibly some other sort of filter would be needed to make them look less like obvious cgi and less detailed. And of course the creatures can be masked better, and something has to be done about the original creatures occasionally poking through behind my modified version.

Edit: Yikes, I'm so sorry to derail your thread, I should probably bring this to my ideas thread.
 
Last edited:

The Scribbling Man

Tenant of the Tower of Flints
Staff member
Donor
Faneditor
Messages
3,054
Reaction score
449
Trophy Points
113
^^ it's fine! It's only a soft derail ;) it's related to where the conversation evolved :)
 

DigModiFicaTion

DᴉმWoqᴉԷᴉcɑꓕᴉou
Staff member
Faneditor
Messages
5,964
Reaction score
720
Trophy Points
163
I'd like to see somebody smooth out the animation on Into the Spider-Verse. I know they used a low frame rate on purpose to help give them a more comic book feel, but it really didn't work for me and I get a bit of a headache watching it.
It's actually a narrative device. Miles goes from 12 frames per second at the beginning and steadily increases to the full 24 frames as he is trained and builds his skills. I believe the back flip/fall from the building at the end is when he first is in full 24fps. The janky quality to his movements were intentionally so to show his lack of growth and refinement. Personally I think it's brilliant and one for the first real technical geek out moments I've had with a movie in a long time.
 
Top Bottom