Forgive me for posting again, and also for the length of this post, but this is an important topic to me, and there has been some great sharing going on.
As far as I can gather, our system for reviewing edits is meant to be flexible, and allow each reviewer to decide how to rate an edit based on any system he or she feels is best. I am completely fine with this, and all the different posts in this thread provide fine examples of how people can approach reviewing edits.
For the time being, the following is how I would like to approach my reviews. I am drawing on all the feedback already provided, plus my own personal feelings on the topic. I want to emphasize that these will be my own personal guidelines, and I am not in any way asking others to agree or follow the same approach. However, if something I say here is problematic to those in charge, please point it out and I will gladly change it! (this is mostly why I am posting it.)
Once I have about 10 or so reviews completed, I will revisit this, and eventually I hope to migrate it to my profile (or some other location) so I can provide a reference to it inside all my reviews (again, to allow the editor access to my own personal system of reviewing).
SECIORS APPROACH TO REVIEWING FAN EDITS (DRAFT)
GENERAL PRINCIPLES
- We are a community of peers with common goals; we try to help each other improve our skills through respectful, constructive feedback.
- Reviews should be honest, but not harsh; evaluation should take into account that this is a hobby and not a profession for both the editor and reviewer.
- Reviews should take into account the editor's stated intention and goals.
- For all categories, a textual description will be provided to explain and support the rating. For lower ratings, the description will be more detailed.
- For edits that I have created a version of myself, I will enforce an "embargo" period to allow myself to gain enough distance and perspective so as not to unintentionally impose my own personal choices on someone else's vision/work. The length of time is a judgement call. I should never refer to my own edit in these reviews, nor use my own approach as any sort of evaluation criteria. It is especially important in these reviews to use the editor's own stated intention and goals to establish the framework of approaching these reviews.
- While I will strive not to provide spoilers, in some special cases, I might feel compelled to discuss an editor's narrative choice that might considered a spoiler. In these cases, I plan on stating in ALL CAPS as the first line of my edit that this review contains spoilers, and those who have not yet seen the edit may not want to read the text due to that fact. I will also surround the specific place in the text that contains a spoiler with a similar warning, so the reader can attempt to jump over the text. However, if spoilers should never be included in reviews, please let me know and I will excise this from my approach!
A/V QUALITY
- Ratings should be based on the quality of the original source material used by the editor.
- If the editor had a specific intention or goal with regards to A/V quality (such as adding film grain or recoloring) this can be considered when rating this category. Depending on the situation though, it may be more appropriate to reserve this type of evaluation for the technical categories.
- The choice to use stereo instead of Surround sound can be noted, but will not cause a drop in rating.
- The choice to use SD instead of HD can be noted, but will not cause a drop in rating.
- Examples of issues that could affect the rating include: interlacing issues, aspect ratio problems, poorly executed upscaling of SD material within an HD edit, sound levels that are incorrectly balanced throughout the entire edit, or cropping that severely degrades the original picture quality.
RATING KEY
10 - A/V quality matches the source material
9 - One or two minor issues exist.
8 - More than two minor issues exist, or a major issue exists.
7 or lower - More than one major issue exists. I seriously don't expect to ever have to use this rating, but if I do, such ratings will be carefully and fully explained within the text of the review.
VISUAL AND AUDIO EDITING
- These are purely technical categories, and thus should focus on pure technical issues. By technical, I mean perceived mistakes in the actual editing process. This does not cover artistic choices, though poorly executed editing intended to be artistic can be covered here.
- If the editor had a specific intention or goal with regards to the technical approach (such as intentional use of jump cuts, freeze frames, vignettes, or silence), this can be considered when rating this category.
- I have a very keen eye for editing issues, but will be guided by the aforementioned general principles when coming up with ratings for these categories.
- Examples of issues that could affect the rating include: flash frames, audio drop-outs, abrupt video or audio transitions, or anything else that is purely technical in nature.
RATING KEY
10 - No technical issues detected, or at most one or two very minor issues detected that do not detract from overall enjoyment.
9 - A few minor issues detected that have a slight impact on overall enjoyment.
8 - Enough minor issues or a major issue that does have an impact on overall enjoyment. Again, these must be strictly technical in nature.
7 or lower - More than one major issue exists the have a larger impact on overall enjoyment. I don't expect to need to use this that often. However, such ratings will be carefully and fully explained within the text of the review.
NARRATIVE
- I pay very close attention to the narrative aspects of all film and TV shows I watch, and I will do the same for fan edits.
- I expect a fan editor's ultimate goal is to tell a cohesive story that should make sense. Problems that exist in the original story should be addressed, either by editing them out or by using them for a new purpose.
- When evaluating the narrative of a fan edit, the editor's stated intent and goals will be my guideline for evaluation.
- My enjoyment of narrative choices will not be factored into this category; rather, this should be an objective evaluation of the quality of the narrative based on its own merits. Thus, I might provide a very high narrative rating, but nonetheless still not enjoy the story (no matter how well it is constructed), which would result in a lower enjoyment rating.
- Examples of narrative issues include: plot holes, discontinuities, incomplete plot lines, story elements that are not well integrated, lack of an established narrative framework, or inconsistencies with an established narrative framework.
- In all cases, I will provide detailed explanations regarding the narrative issues that I detected.
RATING KEY
10 - Outstanding - very few works will receive this rating, as it requires that essentially no narrative problems exist. The editor has perfectly achieved his or her previously stated narrative intention and goals.
9 - Excellent - only minor narrative issues detected.
8 - Very Good - multiple minor issues, or perhaps one major issue detected.
7 - Good - more than one major issue detected.
6 - Problematic - This is my lowest planned rating for narrative. It means I had many issues with the narrative.
ENJOYMENT
- I feel like this is the "real" review category. Thus, I would generally treat the work the same way as I would any regular movie.
- All of the previous categories can factor into my enjoyment rating.
- For comparison, I will provide my rating for the original work.
- I will also provide a detailed explanation as to what I liked and didn't like.
RATING KEY
10 - Outstanding - This is going to be a very rare rating for me.
9 - Excellent
8 - Very Good - This is generally the level where I would recommend an edit, meaning I enjoyed it enough that I'd be willing to watch it again and recommend it to others.
7 - Good - This is not intended to be a bad rating; it means I enjoyed and/or appreciated many aspects of the edit.
6 - I had problems - For whatever reason, I did not enjoy the edit, and of course I would provide details as to why, in the most respectful manner possible. I don't plan on going below a 6 in this category.
OVERALL RATING
- While overall rating is automatically calculated, before I submit the review, I will take a step back and compare the overall rating against what my feeling is for the edit as a whole.
- I would never change my rating of the three technical categories in order to change the overall rating, as the technical ratings should be purely objective and based on factual information.
- In some cases, I might slightly revise my narrative and/or enjoyment rating based on circumstances described below.
- For narrative, I would only make small changes if I feel I might have been too critical (which is my tendency).
- For enjoyment, I would only make small changes if I felt there was too big discrepancy between the overall rating and my enjoyment rating. I think this most likely will not happen but I feel that there are situations that might warrant such adjustment. A prime example would be to factor in a fan editor's originality or boldness. To me, these are some of the factors that make fan editing a great hobby -- to take risks and try to be original -- and thus, to be able to recognize these things do contribute to my enjoyment, if only from a cerebral point of view. Plus I feel they deserve encouragement. So this is why I might slightly modify the enjoyment rating, and if I do so, I would include mention in the textual description that I included this as part of my evaluation of the edit.
- I hope this is an acceptable approach to the powers that be, but if not, please let me know and I will not do it.
RATING KEY
9+ - Excellent/Outstanding
8-9 - Very Good / Excellent
7-8 - Good / Very Good
6-7 - Fair / Good
RECOMMENDATION
- Generally speaking, if my enjoyment rating was an 8, I will probably recommend the edit. This is not a rule, just a guideline.
- I might provide caveats within the text if I feel I can only recommend the edit to a specific audience, or only if the viewer is willing to overlook certain aspects of the edit.
- I do not plan on ever checking off the "No" box. My reviews are meant to be as positive as possible, while of course being honest. My review text might spark someone's interest in the edit based on certain things I said within my review, even if I personally couldn't provide an "official" recommendation. The "No" rating also creates negative connotations that I prefer not to create; as long as I've been honest in my review, I see no need to say I don't recommend an edit.
Well, there you have it. I am again very sorry for the length of this post.
If this post is in any way not appropriate, it was not my intention at all, and would be an honest mistake for which I aplogize. But I wanted to share what I have learned from everyone else that has already posted, as well as state what I think my own approach will be, and get early warning if I am straying down the wrong path.
Thanks again to everyone for your excellent feedback and advice!
