• Most new users don't bother reading our rules. Here's the one that is ignored almost immediately upon signup: DO NOT ASK FOR FANEDIT LINKS PUBLICLY. First, read the FAQ. Seriously. What you want is there. You can also send a message to the editor. If that doesn't work THEN post in the Trade & Request forum. Anywhere else and it will be deleted and an infraction will be issued.
  • If this is your first time here please read our FAQ and Rules pages. They have some useful information that will get us all off on the right foot, especially our Own the Source rule. If you do not understand any of these rules send a private message to one of our staff for further details.
  • Please read our Rules & Guidelines

Swimming in a Sea of Perfect 10’s

krausfadr

Well-known member
Donor
Faneditor
Messages
686
Reaction score
1,011
Trophy Points
118
The review section is an interesting area of IFDB as a lot of reviewers seem to be timid to provide any criticism or scores less than 10’s across the board. I initially found that very weird. For me as an editor any criticism in a review is very valuable. It helps me find anything I might have missed after mind numbing, endless watches of the same scenes over and over.

Should anyone provide criticism to an edit I make (as long as it’s politely constructive) I’ll never thumbs down a review just because I got a little butt hurt about it. Sure my butt may hurt a little at first but in the end I am grateful that someone watched my edit and then took the time to give their HONEST review.

And should you leave a critical review for me, rest assured you will still have access to my future edits if desired, in fact you will get the red carpet to the front of the line.

3.5 stars out of 5 is not a “bad review.” That’s well above average.

When we swim in a sea of perfect 10’s that can lead to floating in a pool of mediocrity as the reviews might not have meaning anymore.

Thanks

Hans K
 

Gieferg

Well-known member
Faneditor
Messages
1,489
Reaction score
1,008
Trophy Points
133
Agreed with every word.

Most of the "All 10" reviews I don't even read (unless the name of the reviewer tells me that I can treat them seriously). I mean I read all reviews for my edits, but not necessarily all reviews for other edits. If I see some 9s, 8s, or 7s, I have a feeling that this must be more thoughtful and valuable review than all 10s.

Frankly, pointing out things that in reviewer's opinion do not work is usually more helpful for the editor than just praising everything.
 

ArtisDead

Banned
Messages
3,582
Reaction score
3,519
Trophy Points
143
I will remember that both of you pointed that out! Now where is my meat cleaver? 🧐
 

M4_

Well-known member
Faneditor
Messages
210
Reaction score
260
Trophy Points
83
Absolutely. I've noticed this as well, especially when a review leaves perfect 10 for enjoyment but then describes in the writing part that they they didn't fully enjoy it. Seems like there's a stigma around leaving lesser ratings because it can be seen as insulting, when really it shouldn't. Honestly I can't really even trust reviews anymore, I've seen ones get perfect 10s across the board but then when I download them there are clear noticeable audio transitions or visual errors. Then don't even get me started on all the review downvoting going on under reviews that are perfectly legitimate. Maybe sometimes I'm too critical though lol idk.
 

Moe_Syzlak

Well-known member
Messages
3,450
Reaction score
1,161
Trophy Points
118
As I said in the recent thread suggesting changes to the review process, I don’t put any stock in the star/number reviews. For me, personally as strictly a viewer, they don’t serve any purpose and, truthfully, I think it provides a disincentive to provide thoughtful commentary in the written section of the review. The written section is the only part I value. And again, the reviews there are probably too kind. But a “this didn’t work for me but that’s not the fault of the editor because you can only do so much with the source material” is a lot more helpful than just seeing 9s or 10s in the scores.
 

Wraith

Well-known member
Donor
Faneditor
Messages
2,430
Reaction score
3,159
Trophy Points
133
I almost started a thread on a not dissimilar vein...I agree with the broad sentiments but there are things to also remember...

Most members are not editors ergo most reviewers are not editors.

The world increasingly peddles a view in social media that all onions are equally valid.

HORSE FEATHERS I say.

There is:
  • Ill informed opinion
  • informed opinion
  • expert opinion

also translates to
  • Horsefeathers
  • valid
  • truth (ish)

I'm sure no one is advocating that only the experts or informed have valid views...Rotten Tomatoes being a case in point.

The question for me is around proportionality.

It can be frustrating after spending years planning an edit that takes an average or decent movie and improves it ...to see it score lower than a quickly put together extended edit with 6 deleted scenes of a great movie, score higher. Are either valid or invalid? That is subjective, and I don't believe you can compare apples and pears and most casual viewers know and realise this as Moe points out...THE WORDS are also important.

I for one thought that maybe introducing a new CREATIVE EFFORT category would go some way to redressing that balance i.e. the creative improvement of an average film gets a 9/10, the simple adding of 6 deleted scenes may merit a 4 or 5. Which brings me to proportionality.

Seeing some edits get a technical score of say 4 on a category, when there could be over 500 plus changes; that does happen too... and it smacks of "expert opinion" shouting "I'm an expert" forgetting that this is also just suppose to be "a bit of fun". Frankly, scores of below 5 say "UNWATCHABLE" to me. In the absence of a defined scale, that can nevertheless be jarring, so again, per Moe's comment...WORDS COUNT.

I also agree that constructive critique is vital; I CRAVE IT, and more importantly I ACT ON IT, and so should others. However there should be a "quid pro quo" IMHO. I recently got a few dud scores, and I went back in and fixed all the issues that someone had taken the care to note and share (thanks). I think if editors do that (go and fix it), it improves all of our output and skills HOWEVER I also believe that if an editor takes such corrective action, the informed/expert reviewer who provided the diligent insights and critique should go back in and similarly adjust/revise their review/scores respectively. That friends, is a rare happening...

I for one am guilty of doing quick extended editions, farming out critique, giving perfect scores and producing works of creative effort; but again lets not forget...

"this is for fun", so lets be kind, constructive and own when we get it wrong (both as editors and reviewers).
Anything we can do to improve on that, gets a big thumbs up from MOI !

ps
and as experts, the more we mentor, the better...

...I should add, an ill informed 10 from someone who enjoyed an edit but cannot articulate why is equally valid to me to the expert critique at 7 that permits me to improve, and then who then hopefully revises their score up...

What do people think about a Creative Effort slot?

If AUDIO/VISUAL QULITY is one category, why not combine the Audio and Visual Edit category, and add this new one. in the freed up slot..? It would reduce the technical impacts and add another artistic POV to balance things out?
 
Last edited:

Darth Kermit

Well-known member
Messages
245
Reaction score
178
Trophy Points
58
The world increasingly peddles a view in social media that all onions are equally valid.
As Shrek would say, “some onions have more layers than others.”

To add to the discussion more than a mere referential joke, I would like to add that as a reviewer, I do take the numbering and my opinions seriously. If I find what I suppose to be a mistake, I PM the editor and find out if it is in the original movie, and I try to provide time stamps as well. I PM’ed Wraith about his CEOT3K edit when I thought there were some errors of the audio kind, discovered they were in the original theatrical cuts, and did not account for that in my review. And I will agree with Wraith that the words are definitely the best form of critique and review.
 

Wraith

Well-known member
Donor
Faneditor
Messages
2,430
Reaction score
3,159
Trophy Points
133
I'd add that I went on to correct a bad music transition in the Theatrical Version that I would have left had it not been pointed out..
"not my fault"
...but fix it anyway!...(that is the idea after all)
 
Last edited:

Malthus

Well-known member
Donor
Faneditor
Messages
1,598
Reaction score
1,632
Trophy Points
148
What do people think about a Creative Effort slot?

You brought up this idea in a previous thread about reviews.

What do you think about proposing an additional Review category; Creative Effort?

I can only reiterate my strong disapproval of the idea. I had two reasons I didn't agree with it:

A. Effort doesn't equal quality
B. Effort is subjective

I elaborated on both points in the aforementioned thread but let me ask this:

How does one measure creative effort exactly?

One of my most creative edits is Driver. It mixes Drive with a handful of scenes from La La Land to further explore the motivations of the central character. While the idea is novel the execution was incredibly simple. In fact, it is technically my most simplistic edit with only 5 scenes added and nothing removed. Aside from those 5 scenes (and the 10 transitions) the only other changes I make were to replace the title card, add some additional credits and use some stock footage at the end of one sequence. That's it. It's a unique take on the film Drive that people have really enjoyed but it took very little effort to achieve it. So is it creative or is it not?

There are some incredible editors who almost exclusively make extended editions and I both respect and admire their work. I do not feel they are any less "creative" than me or any other editor within the community for choosing to "simple add six deleted scenes". Speaking of which, most deleted scenes I've worked with have needed to be colour matched to the source material. Many have required scoring and often the transitions needed to make them sit seemlessly within the film have required real finesse. Some deleted scenes need to be deftly pruned to remove incomplete effects. An extended edit might not be the most original concept but it is in no way lacking creativity and to claim otherwise is unfair. Personally I think assessing "creative effort" has a strong whiff of gatekeeping to me.
 

DigModiFicaTion

DᴉმWoqᴉԷᴉcɑꓕᴉou
Staff member
Faneditor
Messages
8,607
Reaction score
3,505
Trophy Points
168
Gratitude for and then reaching out for clarity of why would take care of this. Simple, but requires work on the part of the editor. To quote Weezer, "If you want it. You can have it. But you've got to learn to reach out there and grab it."
 

Wraith

Well-known member
Donor
Faneditor
Messages
2,430
Reaction score
3,159
Trophy Points
133
You brought up this idea in a previous thread about reviews.



I can only reiterate my strong disapproval of the idea. I had two reasons I didn't agree with it:



I elaborated on both points in the aforementioned thread but let me ask this:

How does one measure creative effort exactly?

One of my most creative edits is Driver. It mixes Drive with a handful of scenes from La La Land to further explore the motivations of the central character. While the idea is novel the execution was incredibly simple. In fact, it is technically my most simplistic edit with only 5 scenes added and nothing removed. Aside from those 5 scenes (and the 10 transitions) the only other changes I make were to replace the title card, add some additional credits and use some stock footage at the end of one sequence. That's it. It's a unique take on the film Drive that people have really enjoyed but it took very little effort to achieve it. So is it creative or is it not?

There are some incredible editors who almost exclusively make extended editions and I both respect and admire their work. I do not feel they are any less "creative" than me or any other editor within the community for choosing to "simple add six deleted scenes". Speaking of which, most deleted scenes I've worked with have needed to be colour matched to the source material. Many have required scoring and often the transitions needed to make them sit seemlessly within the film have required real finesse. Some deleted scenes need to be deftly pruned to remove incomplete effects. An extended edit might not be the most original concept but it is in no way lacking creativity and to claim otherwise is unfair. Personally I think assessing "creative effort" has a strong whiff of gatekeeping to me.
I completely concur, especially as someone who has made a few Extended edits too. I can attest first hand that "dropping in" those scenes is not always quite so straightforward, though sometimes it can be easier if they are completed scenes already. I certainly am not seeking to diminish Extended Editions, since they are among may faves. It goes to the operative word; "proportionality".

To your question, "how does one measure Creative Effort"...? How does one measure "quality of sound or visual editing", or "Narrative" etc...

When editors make a submission, their description, overview and change lists are an attestation to intent, methodology and attainment. That does give reviewers a good guide to judge all the criteria, including "Creative Effort"...or maybe it should be "Creative Attainment"?

If an edit contains 100s of changes and a half a dozen audio glitches, how does one measure that? Is it 10-2=8... if it's a mess all over it may be a 4, but then I'd suggest it should not be out there if its that bad...what would be proportionate?

The thread is about a "Sea of 10s". I was trying to bring some perspective to issues that further underlie that oceanic vista of 10s, and seeking to explore mechanisms that might both engage the reviewer/viewer and reviewer/editor while factoring in an element that appears to not quite get capture other than maybe by "Narrative/Enjoyment" categories namely, the vision one has when embarking on an edit and that effort of creativity needed to execute. That's all.

Maybe it aint broke, but clearly some people are not enamored with the status quo either.

It's about the quality too, not just the volume of changes (which could be counterproductive as well) . Given your description, I would give an 8 for Creative Effort as an example (though I have not seen it, so it could be a 9) .
 

krausfadr

Well-known member
Donor
Faneditor
Messages
686
Reaction score
1,011
Trophy Points
118
Wraith, I enjoyed reading your perspective and find it a completely valid viewpoint. I admit I'm tough on audio (from a glitch/error) perspective. I'm tough on myself. But I'm more tolerant of "jump cutting" the musical score in cuts as long as it doesn't cause a tempo change or chordal dissonance.

From my perspective, the bit of fun edits should generally not reach 10's (if they have mistakes) and the polished, professional one's should be able to reach a 10 level. That is unless you want to lower the bar and penalize someone who puts a very extended amount of time into an edit. But that's any reviewers' prerogative.

I do believe this is for fun but I don't accept that since this is for fun, expect errors, that's just how "fanedits" are. I know this is not what you're saying, but it's ultimately how I interpret the end result. I do notice the edits that are so polished-- the fact that they are edits is undetectable or nearly so, and rate them accordingly.

The home audience generally doesn't notice the number of cuts, but they often do notice the number of errors. While some are oblivious from watching in their living room while the dishwasher is running and checking their phone, others wear airpods on the ipad, and then hear ALL the errors but are too shy to mention it. So if an editor makes a hundred cuts with no errors and another editor makes a thousand cuts with ten errors, then technically speaking the second edit is worse.

I agree if an editor fixes an edit the reviewer should update their rating!!
If an editor PM's me advising about making a change, then I'll gratefully take the improved version and then go back and revise the score!

For me a score of 4 on audio editing alone doesn't mean "unwatchable." Below are the rating explanations I use. If there are a couple of audio glitches then depending on what they are I might call that average (glitches often happen in theatrical releases, for example in ZSJL they accidentally put Batman's voice SFX onto Aquaman's voice in one scene). If the glitches keep recurring in an edit then, for me, it gets into below average technical territory. It doesn't matter how complicated the edit was.

I very much agree with revamping the rating system to rate factors that go into the edit such as creativity which I would more quantifiably term as "Transformative creativity with vast improvement" to the original.

Thinking more about what you're saying about creativity, I would remove the "ENJOYMENT" rating entirely which factors in too much subjectivity of enjoyment from the original release, and replace it with this question:

Did this edit improve the original source(s)?

Not at all or made worse = 1
Some minor improvement = 4
Moderate improvement or improvement mostly by removing (not changing/adding) content = 6
Creative additions or changes with much improvement = 8
Transformative creativity with vast improvement = 10

Here's my general rating explanations:

Best in Class / Perfect (or nearly so) = 10
Excellent = 9
Good = 8
Decent / Above Average = 7
Fair / Slightly Above Average = 6
Slightly Below Average = 5
Below Average = 4
Poor = 3
Bad = 2
It really can't get worse = 1
 
Last edited:

Wraith

Well-known member
Donor
Faneditor
Messages
2,430
Reaction score
3,159
Trophy Points
133
Thinking more about what you're saying about creativity, I would remove the "ENJOYMENT" rating entirely which factors in too much subjectivity of enjoyment from the original release, and replace it with this question:
I think that is a great idea, to filter out the built in element ....."Improvement of Source" ergo, Creative and so much more to boot.

I'd like to be able to see a bad movie which is improved, score well and not badly just coz the source was a car crash.
DUNE seems to do well mind you!
 

krausfadr

Well-known member
Donor
Faneditor
Messages
686
Reaction score
1,011
Trophy Points
118
I fleshed out how I rate/will rate edits so my seeming madness shows actual method (hopefully) and my ratings remain relatively consistent. And this way a 10 is above and beyond, showing creativity.

Audio/Video Quality
(if the video and audio separately have different quality levels, I score them separately and calculate the average of the two)
Frequent jarring or irritating video or audio artifacts which ruin the experience = 1
Occasional, very noticeable video or audio issues (significant video blockiness may happen in darker scenes, audio may sound muddy, missing high frequencies) = 4
Some noticeable issues, fair quality (minimum 1080p*, stereo audio) = 6
Rare occurrence of any issues or artifacts, good quality (minimum 1080p*, stereo audio) = 8
No apparent degradation compared to original source, perceptually lossless video and audio, best quality (minimum 2160p*, surround sound*) = 10
* if source available in 1080p/2160p and/or surround

Visual Editing incl. Color Grading, VFX
Three or more instances of jarring issues such as jump cuts, missing frames, flash frames = 1
One or two instances of jarring issues or five or more instances of minor issues such as choppy frame rate, color grading shifts, face/hair/skin have graded color = 4
Four or less instances of minor issues, noticeable but not jarring, any added color grade or VFX is fair = 6
One or two instances of minor issues, mostly seamless, professional, any added color grade or VFX is good = 8
Completely seamless, professional. Must include color grading or VFX and at best level of quality = 10

Audio Editing incl. Musical Score
Three or more instances of jarring issues such as very rough transitions, complete loss of sound, or major audio syncing errors = 1
One or two instances of jarring issues or five or more instances of minor issues such as clicks, low audio; or added music masks the dialogue = 4
Four or less instances of minor issues, noticeable but not jarring, added music fits the mood but has minor volume issues = 6
One or two instances of minor issues, nearly seamless, added music improves the scene(s) and blends well = 8
Completely seamless, no mistakes, professional. Must include audio restoration or compelling changes to musical score = 10

Narrative
Hard to follow or introduces new, major plot holes = 1
Makes sense but feels rushed or empty = 4
Makes sense while not significantly changing the original narrative = 6
Fixes story problems found in the original = 8
Significantly changes/enhances the story with a new and compelling dramatic impact (or new comedic or action impact, etc.) = 10

Enjoyment (for me enjoyment = creativity + improvement)
Overall Improvement: Did this edit CREATIVELY improve upon the original source(s)?
Not at all or made worse = 1
Some minor improvement = 4
Moderate improvement or improvement mostly by removing (not changing/adding) content or mostly by inserting deleted scenes = 6
Creative additions or changes with much improvement = 8
Transformative creativity with vast improvement = 10

***
Jarring audio editing issues:
Loud clicks, pops, glitches
Audio gaps
Mid scene fade outs
Very rough transitions
Out of Sync

Minor audio editing issues:
Small clicks, pops
Background ambience loss
Audio fragments (cut bits still audible)
Dissonant transition (music in different keys crossfaded at a bad spot)
Transition with tempo mismatch or beat skip
Transition into the middle of music cue
Transition swoosh noise
Volume swell
Sudden bass boost or thinness
Sudden missing foley sounds
Music changed but no new processing on center channel (sounds thin, flat)
 
Last edited:

The Scribbling Man

Tenant of the Tower of Flints
Staff member
Donor
Faneditor
Messages
4,644
Reaction score
2,416
Trophy Points
148
Audio/Video Quality

No apparent degradation compared to original source, perceptually lossless video and audio, best quality (minimum 2160p, surround sound) = 10

If it's relative to the source, how can it only be qualified for minimum 2160p? If the source is a DVD and the quality hasn't dipped from the quality of the DVD, wouldn't that get under a 6 by your system (since 6 is minimum 1080p)?

I don't think it's fair to rate the quality of the audio based on whether it is stereo or surround either.
 

krausfadr

Well-known member
Donor
Faneditor
Messages
686
Reaction score
1,011
Trophy Points
118
If the source is only 1080p then of course the result would max at 1080p. 2160p minimum only as feasible. 5.1 quality is always better than 2.0. No 10's for stereo though, unless of course again the source is ONLY stereo.

EDIT: I updated my explanation for the source issue. And if the source comes in surround, but the release is only in stereo, the highest audio quality I'd probably give that is an 8. You will hear the mix much better in 5.1 than stereo, so IMO when dealing with a movie soundtrack-- stereo vs surround is an issue affecting audio quality.

EDIT 2: I watch films available in HD, so watching an edit based on a DVD didn't even cross my mind.
 
Last edited:

addiesin

Well-known member
Messages
5,887
Reaction score
1,502
Trophy Points
163
if it's a mess all over it may be a 4, but then I'd suggest it should not be out there if its that bad...what would be proportionate?

I think this is a big factor of the sea of 10s issue here. It is seemingly contradictory to have a 1 to 10 rating scale when anything being rated by said scale "can't" reasonably go below a 5. Even if it's not really contradictory, it seems to be initially, and that maybe generates a tendency to go towards the higher scores anyway.

But another big one is just that everyone interprets every aspect of reviews their own way.
One person just saw their first fan edit and had their mind opened to the concept for the first time: perfect 10, inspiring.
Another really loves the source material except for something the fan edit removes: perfect 10, they "fixed it".
Another noticed flaws but likes the editor as a person or doesn't feel like spending time to detail the problems or understandably didn't feel like taking notes while watching and forgot by the timecodes by the time the edit was over: perfect 10, good intentions/better than no review at all
Another hates the source material and is extremely impressed by the amount of improvement even though they still don't like it, perfect 10, you did the best you could
 

ArtisDead

Banned
Messages
3,582
Reaction score
3,519
Trophy Points
143
in which the process is and will always be subjective based on perspective.
 

Murikamir

Well-known member
Faneditor
Messages
76
Reaction score
83
Trophy Points
28
I think the scale rating system is inherently flawed. Most people vote the extremes of 10's or 1's, and since the is a very positive website overall, it ends up being 10's across the board. I think the best thing to do is change the rating system to something that doesn't end up being a like/dislike button.

One good option might be make a Pro and Con list, and have most commonly shared Cons and Pros feature as the overall review (similar to a product review). Or make the ratings more meaningful. It is something I always appreciated about Jeremy Jahns as a reviewer, he doesn't use a rating scale (even if some of his ratings can fit into one). You get more a sense of the movie if you hear "It's a good time, if your drunk" and "Good time no alcohol required", or "I would buy this on blu ray" or "Awesometacular" and "Dogshit" than a 1-10 scale. We could do something similar here, like, "Better than the original", or "Was an interesting take" or "Good idea, didn't work" or "Weird, but I liked it"
 
Top Bottom