• Most new users don't bother reading our rules. Here's the one that is ignored almost immediately upon signup: DO NOT ASK FOR FANEDIT LINKS PUBLICLY. First, read the FAQ. Seriously. What you want is there. You can also send a message to the editor. If that doesn't work THEN post in the Trade & Request forum. Anywhere else and it will be deleted and an infraction will be issued.
  • If this is your first time here please read our FAQ and Rules pages. They have some useful information that will get us all off on the right foot, especially our Own the Source rule. If you do not understand any of these rules send a private message to one of our staff for further details.
  • Please read our Rules & Guidelines

    Read BEFORE posting Trades & Request

A discussion on Review Ratings

Status
Not open for further replies.

DigModiFicaTion

DᴉმWoqᴉԷᴉcɑꓕᴉou
Staff member
Faneditor
Messages
8,791
Reaction score
3,685
Trophy Points
168
This thread is meant as a calibrating discussion on rating review criteria. The review rating guidelines state:


IV. Rating Fanedits

All ratings/reviews on The Internet Fanedit Database (IFDb) are moderated to ensure quality control. Faneditors are not allowed to rate their own fanedits, however they may comment on reviews if they feel the need to. Ratings need to be fair and include a comment explaining how a score was ascertained. Lengthy or detailed reviews are typically not a requirement but are highly encouraged. Detailed reviews are required when giving a low rating (see below). When rating, please consider ALL the following criteria:
  • Technical quality: Were the video and sound quality up to par? Is the quality at or near the quality of the source material?
  • Editing quality: Did you notice any jarring video or audio cuts? Did the fanedit flow well?
  • Narrative: How well did the editor achieve his/her goals? Were there plot-holes created (or resolved?) Does it work?
  • Enjoyment: Were you entertained by the fanedit?
Rating Score Guide
10 = Flawless
9
= Almost Undetectable
8
= Smooth, but slightly noticeable
7
= Noticeable, but not jarring/abrupt
6
= Slightly jarring/abrupt
5 =
Jarring/abrupt and distracting
for scores 4 or less, please contact a staff member

Any rating of 5 stars or lower requires a clear and detailed explanation of the reasoning for such a low score because ratings this low suggest the quality may be insufficient for our community. Ratings below 5 stars and lacking a sufficient comment will be rejected. Be fair when rating and reviewing edits. Don’t abuse the ratings system to get back at someone who rated your fanedit or your friend’s fanedit lower than you expected.


Click below links to quick jump to the beginning post of each rating discussion:
10 = Flawless
9 = Almost Undetectable
8 = Smooth, but slightly noticeable
7 = Noticeable, but not jarring
6 = Slightly jarring
5 = Jarring/distracting
 
Last edited:
With the recent trend of perfect 10 ratings, we'd like to open this discussion to calibrate what a perfect 10 means.

As stated above, a perfect 10 score implies that the quality, editing and narrative are flawless. This means the quality is the same as the source, the editing is undetectable and the narrative is free of any inconsistencies. For edits that add new material, such as vfx or audio, it means that such matches perfectly with the source material.

I'd invite us to use the top rated edit by Adywan, Star Wars Revisited to help us see how a perfect 10 might be applied.

Those who have rated all or aspects of that edit a 10, can you share what made it flawless to help us calibrate that rating as a community?
 
Last edited by a moderator:
This might be splitting hairs, but could we make 10 an "above and beyond" rating for things that are either incredible accomplishments that you wouldn't have thought possible, or great changes that no one else has thought of/managed to do.

For example, an A/V 10 would be an edit that manages to be super high quality with a tiny filesize or restoring deleted scenes to finished quality.
An audio editing 10 would be something like the flawless music replacement in Mathilda, or DonkeyKonga rejoicing an entire character.
A video 10 would be impressive vfx work or flawless integration of new footage.
And a narrative 10 would be reserved for edits that completely change the story, combine two or more movies, or condense entire seasons into one movie (and do it well).

Enjoyment 10s would obviously just be viewer's discretion, but would generally indicate that its an improvement over the original.
 
With the recent trend of perfect 10 ratings, we'd like to open this discussion to calibrate what a perfect 10 means.

As stated above, a perfect 10 score implies that the quality, editing and narrative are flawless. This means the quality is the same as the source, the editing is undetectable and the narrative is free of any inconsistencies. For edits that add new material, such as vfx or audio, it means that such matches perfectly with the source material.

I'd invite us to use the top rated edit by Adywan, Star Wars Revisited to help us see how a perfect 10 might be applied.

Those who have rated all or aspects of that edit a 10, can you share what made it flawless to help us calibrate that rating as a community?

Not going to lie I've given perfect 10s to edit that imo werent 'perfect'. Simply because we've created an environment where an 8 is considered a low score. Its basically the same issue the gaming industry has created, where if it isnt a perfect 10 it must be flawed. IMDb used to have a pretty good system in check before casuals just gave perfect 10s to marvel movies lol. Back in the day any movie on imdb with a 7.5 or higher was basically a must see movie. This is now not the case anymore unfortunately and imdb now has forgettable popcornflicks with higher ratings than actual masterpieces lol.

Like I think maybe having a popup on screen when someone selects a perfect 10 rating with a 'Are you certain the narrative was *flawless* and nothing could improve it?' sort of thing might help people consider their ratings more.
 
Last edited:
Not going to lie I've given perfect 10s to edit that imo werent 'perfect'. Simply because we've created an environment where an 8 is considered a low score. Its basically the same issue the gaming industry has created, where if it isnt a perfect 10 it must be flawed. IMDb used to have a pretty good system in check before casuals just gave perfect 10s to marvel movies lol. Back in the day any movie on imdb with a 7.5 or higher was basically a must see movie. This is now not the case anymore unfortunately and imdb now has forgettable popcornflicks with higher ratings than actual masterpieces lol.

Like I think maybe having a popup on screen when someone selects a perfect 10 rating with a 'Are you certain the narrative was *flawless* and nothing could improve it?' sort of thing might help people consider their ratings more.
Most fanedits by decently skilled editors should be 10/10. at least in terms of video quality and audio/video editing. If you're watching the movie and you don't notice the cut then in your case, it is perfect, right? I think there could be room for other scores. Maybe to reflect creativity, innovation? Maybe a score to reflect how much the editor did to differentiate it? an edit that has 1000 cuts and removed a ton and added in extra stuff, is very different from an edit that cut out one scene.
 
Most fanedits by decently skilled editors should be 10/10. at least in terms of video quality and audio/video editing. If you're watching the movie and you don't notice the cut then in your case, it is perfect, right? I think there could be room for other scores. Maybe to reflect creativity, innovation? Maybe a score to reflect how much the editor did to differentiate it? an edit that has 1000 cuts and removed a ton and added in extra stuff, is very different from an edit that cut out one scene.

These are my in depth opinions on ratings:

Video and audio editing:
I agree there. Most video and audio editing scores should be 9s or 10s if it's simple cutting and pasting.

Audio and video quality:
Already more slippery considering the variables possible. Technically speaking only 4k HDR releases can be 10/10 considering that's the highest achievable quality now.

let's say 1080p is the standard for fanedits and those with good quality encoding are 10/10

Audio has 5.1 as the gold standard for edits, technically stereo cant be a 10/10 unless it's a very old movie.

Narrative: This one has most meaning when its possible to compare it to the original. A 6/10 turned into a 7/10 shouldn't result in a 10/10 imo. And an unaltered story shouldnt result in a 10/10 neither. Narrative is just that: narrative. I think we should always be be to first rate the original narrative and then score the new narrative. That way the scoring is actually meaningful. Right now 10/10 for narrative can mean anything lol.

Enjoyment: See narrative.
 
Audio and video quality:
Already more slippery considering the variables possible. Technically speaking only 4k HDR releases can be 10/10 considering that's the highest achievable quality now.
Ratings are supposed to be versus the source though. If an edit is released in HD, then the source to compare against would be standard bluray.
I've seen people rating things down because they are not 4k, but that's against the guidelines I think. If you are editing from a bluray and can't afford those expensive HDblurays, why should you be penalised? You can't make it look better than the source you have.

Audio has 5.1 as the gold standard for edits, technically stereo cant be a 10/10 unless it's a very old movie.
Hard disagree. Not everyone has a 6 speaker system. If the viewer has a stereo system and the audio sounds the same quality as the bluray they own, then they absolutely can score it a 10. Each reviewer can make their own decision on that of course, but it's not fair to say stereo can never get a 10.
 
Last edited:
Ratings are supposed to be versus the source though. If an edit is released in HD, then the source to compare against would be standard bluray.
I've seen people rating things down because they are not 4k, but that's against the guidelines I think. If you are editing from a bluray and can't afford those expensive HDblurays, why should you be penalised? You can't make it look better than the source you have.


Hard disagree. Not everyone has a 6 speaker system. If the viewer has a stereo system and the audio sounds the same quality as the bluray they own, then they absolutely can score it a 10. Each reviewer can make their own decision on that of course, but it's not fair to say stereo can never get a 10.

By that logic a dvd should get the same rating for video quality as a 4k HDR edit. So yeah agree to disagree there.

And the fact that not everyone has a 6 speaker setup is no argument for the standard.
 
Ratings are supposed to be versus the source though. If an edit is released in HD, then the source to compare against would be standard bluray.
I've seen people rating things down because they are not 4k, but that's against the guidelines I think. If you are editing from a bluray and can't afford those expensive HDblurays, why should you be penalised? You can't make it look better than the source you have.


Hard disagree. Not everyone has a 6 speaker system. If the viewer has a stereo system and the audio sounds the same quality as the bluray they own, then they absolutely can score it a 10. Each reviewer can make their own decision on that of course, but it's not fair to say stereo can never get a 10.
For what it's worth, I agree with @tremault . There is a reason I stopped editing.
 
By that logic a dvd should get the same rating for video quality as a 4k HDR edit. So yeah agree to disagree there.

And the fact that not everyone has a 6 speaker setup is no argument for the standard.
But that's how it works. I felt it was made pretty clear that the ratings are to be based on the format it's released in. so yes, a DVD quality file that is released as a DVD format edit should be rated according to how good it looks and sounds compare to other DVD's.
If you don't like that then maybe that's a discussion to be had, but that is what the mods have maintained as long as I've been here. (I'm pretty sure)
 
Last edited:
This thread is meant as a calibrating discussion on rating review criteria. The review rating guidelines state:


IV. Rating Fanedits

All ratings/reviews on The Internet Fanedit Database (IFDb) are moderated to ensure quality control. Faneditors are not allowed to rate their own fanedits, however they may comment on reviews if they feel the need to. Ratings need to be fair and include a comment explaining how a score was ascertained. Lengthy or detailed reviews are typically not a requirement but are highly encouraged. Detailed reviews are required when giving a low rating (see below). When rating, please consider ALL the following criteria:
  • Technical quality: Were the video and sound quality up to par? Is the quality at or near the quality of the source material?
  • Editing quality: Did you notice any jarring video or audio cuts? Did the fanedit flow well?
  • Narrative: How well did the editor achieve his/her goals? Were there plot-holes created (or resolved?) Does it work?
  • Enjoyment: Were you entertained by the fanedit?
Rating Score Guide
10 = Flawless
9
= Almost Undetectable
8
= Smooth, but slightly noticeable
7
= Noticeable, but not jarring/abrupt
6
= Slightly jarring/abrupt
5 =
Jarring/abrupt and distracting
for scores 4 or less, please contact a staff member

Any rating of 5 stars or lower requires a clear and detailed explanation of the reasoning for such a low score because ratings this low suggest the quality may be insufficient for our community. Ratings below 5 stars and lacking a sufficient comment will be rejected. Be fair when rating and reviewing edits. Don’t abuse the ratings system to get back at someone who rated your fanedit or your friend’s fanedit lower than you expected.


Click below links to quick jump to the beginning post of each rating discussion:
10 = Flawless
9 = Almost Undetectable
8 = Smooth, but slightly noticeable
7 = Noticeable, but not jarring
6 = Slightly jarring
5 = Jarring/distracting

There should be the same level of scrutiny and oversight to 10/10 ratings as 5/10. Both ratings should be rare.
 
But that's how it works. I felt it was made pretty clear that the ratings are to be based on the format it's released in. so yes, a DVD quality file that is released as a DVD format edit should be rated according to how good it looks and sounds compare to other DVD's.
If you don't like that then maybe that's a discussion to be had, but that is what the mods have maintained as long as I've been here.

I get what you're saying. I'm saying it makes the whole review system meaningless. By that logic I could make a lotr edit with an old VCR and get 10/10s and get upset when someone gives it a 6/10 because it objectively looks shit lol.

It also would give a skewed impression of expectations. It could have 32 10/10s for visual quality but be actually VCR.

Which brings me back to my point: This has turned into the same scoring system the gaming industry utilises where every that's 9 or higher is considered good, and anything lower is considered bad. Basically making the whole system useless.

Also with the tools available you definitely can make an image better than the source these days, depending on the source.
 
There should be the same level of scrutiny and oversight to 10/10 ratings as 5/10. Both ratings should be rare.

Agree. Only visual editing and audio editing and enjoyment should realistically get consistent 10/10s.
 
I get what you're saying. I'm saying it makes the whole review system meaningless. By that logic I could make a lotr edit with an old VCR and get 10/10s and get upset when someone gives it a 6/10 because it objectively looks shit lol.

It also would give a skewed impression of expectations. It could have 32 10/10s for visual quality but be actually VCR.

Which brings me back to my point: This has turned into the same scoring system the gaming industry utilises where every that's 9 or higher is considered good, and anything lower is considered bad. Basically making the whole system useless.

Also with the tools available you definitely can make an image better than the source these days, depending on the source.
I understand where you're coming from, and I sort of agree.
The trouble comes though when HDblurays are costing so much money and an editor just can't afford that. Then it becomes a system that is inherently rife with inequality, where the people with money automatically get higher ratings.
Perhaps it should be easier to filter for the different size formats? a movie that is an excellent quality at 1080p should be recognised as such, but maybe also be very clear that it is only in 1080p. That way people know what to expect.
There is currently an option on the edit pages for "is it HD", so perhaps that's not fit for purpose any more?
I mean at the end of the day, it's called 'video quality', it's not called 'pixel density'.

edit: incidentally, that also applies to audio, it should be audio quality, not how many channels there are.
 
Last edited:
I understand where you're coming from, and I sort of agree.
The trouble comes though when HDblurays are costing so much money and an editor just can't afford that. Then it becomes a system that is inherently rife with inequality, where the people with money automatically get higher ratings.
Perhaps it should be easier to filter for the different size formats? a movie that is an excellent quality at 1080p should be recognised as such, but maybe also be very clear that it is only in 1080p. That way people know what to expect.
There is currently an option on the edit pages for "is it HD", so perhaps that's not fit for purpose any more?
I mean at the end of the day, it's called 'video quality', it's not called 'pixel density'.

Video quality and pixel density are very much related. Im not sure if only rich people would be able to enjoy high ratings tbh. However, I render on low end hardware (it was decent in 2012 for reference) and basically have to render edits for days on end. Also, depending on the film plenty of 1080p bluray are superior to their 4k counterparts in color or filmgrain for example.

However as I said I think 1080p should realistically be the standard for fanedits and worthy of a 10/10. I realise proper 4k DTS is unrealistic as there will always be a little loss. 5.1 audio however is very realistic.

My favourite bluray is Hara Kiri and it has stereo and terribly dated audio. If I make a Hara Kiri edit (utterly useless since it's a perfect film lol) i suppose a high audio scoring for that film makes sense. Its an old film and up to the standard for a film of its era. There is no better alternative available.

However stereo for more modern films? They usually have 5.1 mixed ready and available. Nothing is stopping you from editing in 5.1 other than learning how to. I dont have a 5.1 audio setup myself but all my edits are 5.1 simply because I want people who have the luxury of owning such hardware to also enjoy the edits without feeling like it's a step down. You can edit in 5.1 even if you dont own the hardware to play it.
 
Last edited:
However stereo for more modern films? They usually have 5.1 mixed ready and available. Nothing is stopping you from editing in 5.1 other than learning how to. I dont have a 5.1 audio setup myself but all my edits are 5.1 simply because I want people who have the luxury of owning such hardware to also enjoy the edits without feeling like it's a step down. You can edit in 5.1 even if you dont own the hardware to play it.
I would be concerned if I were you. I have worked with 5.1 in the past and have come across nefarious issues such as the audio tracks getting swapped. Since I don't have a 5.1 system myself, I realised that if I can't hear those channels properly, I can't properly judge their positioning. I literally didn't notice an issue with one of my edits until somebody with a 5.1 system pointed out a noise was coming from the completely wrong direction. I couldn't hear that at all, so I decided not to use 5.1 in future, but to try and find a way to use prologic if I can find a system to do that reliably. I have yet to find a reliable solution. I'm not sure if it exists.
 
I would be concerned if I were you. I have worked with 5.1 in the past and have come across nefarious issues such as the audio tracks getting swapped. Since I don't have a 5.1 system myself, I realised that if I can't hear those channels properly, I can't properly judge their positioning. I literally didn't notice an issue with one of my edits until somebody with a 5.1 system pointed out a noise was coming from the completely wrong direction. I couldn't hear that at all, so I decided not to use 5.1 in future, but to try and find a way to use prologic if I can find a system to do that reliably. I have yet to find a reliable solution. I'm not sure if it exists.

This happened to me and therefor I reworked all my edits recently lol. In proper 5.1 this time. I found a great tutorial if you're using premiere pro i can link it for you.

I found out because of my TDK edits with Nolans bwaaaaaas music lol. Someone with proper 5.1 said it sounded like shit.
 
This happened to me and therefor I reworked all my edits recently lol. In proper 5.1 this time. I found a great tutorial if you're using premiere pro i can link it for you.
I appreciate that. I use resolve though.
 
Most fanedits by decently skilled editors should be 10/10. at least in terms of video quality and audio/video editing. If you're watching the movie and you don't notice the cut then in your case, it is perfect, right? I think there could be room for other scores. Maybe to reflect creativity, innovation? Maybe a score to reflect how much the editor did to differentiate it? an edit that has 1000 cuts and removed a ton and added in extra stuff, is very different from an edit that cut out one scene.
I've rarely ever watched an edit that had a true flawless level of editing. The categories are based on the review process to ensure the same level of quality. Edits on IFDb should be of a higher quality than edits that aren't vetted for such quality, but that doesn't mean that they are perfect 10's.

This might be splitting hairs, but could we make 10 an "above and beyond" rating for things that are either incredible accomplishments that you wouldn't have thought possible, or great changes that no one else has thought of/managed to do.

For example, an A/V 10 would be an edit that manages to be super high quality with a tiny filesize or restoring deleted scenes to finished quality.
An audio editing 10 would be something like the flawless music replacement in Mathilda, or DonkeyKonga rejoicing an entire character.
A video 10 would be impressive vfx work or flawless integration of new footage.
And a narrative 10 would be reserved for edits that completely change the story, combine two or more movies, or condense entire seasons into one movie (and do it well).

Enjoyment 10s would obviously just be viewer's discretion, but would generally indicate that its an improvement over the original.
This would be difficult and cause confusion. If an editor does a ton of additional work, that shouldn't increase their editing score, especially if the additional work is not flawless. I can understand how that would feel unfair when compared to a movie that is an extended edition with a few additions, but again, we are looking at the quality of the editing, not the quantity of the editing.

Ratings are supposed to be versus the source though. If an edit is released in HD, then the source to compare against would be standard bluray.
I've seen people rating things down because they are not 4k, but that's against the guidelines I think. If you are editing from a bluray and can't afford those expensive HDblurays, why should you be penalised? You can't make it look better than the source you have.

Hard disagree. Not everyone has a 6 speaker system. If the viewer has a stereo system and the audio sounds the same quality as the bluray they own, then they absolutely can score it a 10. Each reviewer can make their own decision on that of course, but it's not fair to say stereo can never get a 10.
This is true. The quality is based on the source. If it's 4k, the quality score should reflect how close it is to the source. Same for Blu Ray or digital. Editors can help that process by clearly stating in their information sections what source they used.

5.1 is NOT required for edits. If a flawless 2.0 or 2.1 audio is produced for an edit from a 5.1 source, it would be disingenuous to score any lower than a 9. That acknowledges that it's not the same as the source, but superbly done.

By that logic a dvd should get the same rating for video quality as a 4k HDR edit. So yeah agree to disagree there.
And that would be accurate. If it's a DVD source and it matches that quality perfectly, there is no reason it couldn't receive a flawless 10, if it truly is flawless. You can't compare apples and oranges, especially when the editor didn't have access to such.

There should be the same level of scrutiny and oversight to 10/10 ratings as 5/10. Both ratings should be rare.
This 1000%

I get what you're saying. I'm saying it makes the whole review system meaningless. By that logic I could make a lotr edit with an old VCR and get 10/10s and get upset when someone gives it a 6/10 because it objectively looks shit lol.

It also would give a skewed impression of expectations. It could have 32 10/10s for visual quality but be actually VCR.

Which brings me back to my point: This has turned into the same scoring system the gaming industry utilises where every that's 9 or higher is considered good, and anything lower is considered bad. Basically making the whole system useless.
Why is it meaningless. IFDb is about objective quality edits. If it clearly states that it's available in SD, HD, and/or UHD, no one is being fooled or misguided. Adywan's Star Wars Revisited is perhaps the best looking edit I've ever watched, and it's SD. It is flawless. To give that edit a 6/10 because it's SD quality is inaccurate as the base source it was made from is a DVD. It exceeds the quality of its source, which I've never seen before or since.

The inflation of scores is why we are having this discussion and calibration.
Also with the tools available you definitely can make an image better than the source these days, depending on the source.
The work we collectively produce implies that none of us know about such tools. Perhaps we need to have more tutorials on how to produce such. Even the upscales that people make are far from flawless.

Agree. Only visual editing and audio editing and enjoyment should realistically get consistent 10/10s.
If they truly are flawless, but this is more often than not not the case.

Would we have individuals who would be willing to share clips that we can collectively calibrate scores for? I think that approach would be the best to help us talk about a specific edit. I'm willing to put mine in to the ring to pick a part. Anyone else willing?

If you are willing, please make one or more videos that highlight some edited areas, set them to non downloadable (or upload to youtube, vimeo, odysee, etc.) and share them here. If possible, upload an original flow of the scene(s) and an edited flow of the scene(s).
 
Status
Not open for further replies.
Back
Top Bottom