Thoughts on weighting Oscar pool?

Posted by · 7:36 pm · February 21st, 2011

UPDATE: I think I’ll do a 1-5 system based on difficulty of category, as decided by you in the poll below. So please vote and let us know what you think the toughest category to predict is. (Also, I screwed up and didn’t include Best Documentary Short Subject in the poll and it’s too late to change it given all the voting, so if you were going to pick doc short as the most difficult, please let me know in the comments section. I can tally up any votes spoken there. Sorry, guys.)

Just a reminder, if you haven’t joined our annual Oscar pool at Picktainment, you can do so here.  Any advice on weighting the categories?  I’m open to the consensus.  Not sure what’s best.

What is the most difficult category to predict this year? free polls

→ 39 Comments Tags: | Filed in: Daily

39 responses so far

  • 1 2-20-2011 at 7:46 pm

    Chad Hartigan said...

    5 points per category I get right, 1 point per category I get wrong.

  • 2 2-20-2011 at 7:55 pm

    DarkLayers said...

    Maybe you could weigh it towards rewarding toss-ups, with stuff like director and art direction compared to male in a leading role or visual effects.

  • 3 2-20-2011 at 8:04 pm

    Kristopher Tapley said...

    Can be subjective, I guess. What can we agree are the toss-ups?

  • 4 2-20-2011 at 8:09 pm

    Sam C. said...

    I mean, seriously, Sorkin, TS3 and Firth are the ONLY locks in the race, if you ask me most of all techs are toss-ups, with a few others (Seidler, Bale) relative locks.

    I like Eberts Lean-Lock pool, personally

  • 5 2-20-2011 at 8:11 pm

    Andrej said...

    Yeah, something like what DarkLayers said. It could be weighted depending on how much of a toss-up it is.


    Two very viable contenders (directing) = x points.
    Three very viable conteders (foreign film) = x+2 points.
    Four or five very viable contenders (supporting actress) = x+4 points.

    You could also add extra points to the categories depending on how many BP films are in that category, so categories like Make Up won’t be weighted as much as Art Direction, for instance.

  • 6 2-20-2011 at 8:26 pm

    The Dude said...

    I don’t know anything about Picktainment, but would you be able to do something where you get more points for choosing an upset? As an example…for Best Actor, if Firth were to win and you picked Firth then you get three points, but if you picked Bridges (a longshot) and actually got it right then you get 5 points. Doing something like this makes it so that going for those risky upsets could pay off. But it would also be very complicated to keep track of, and since I know nothing of Picktainment I have no idea if they’re able score something like that.

  • 7 2-20-2011 at 8:30 pm

    Filipe said...

    Best Picture = x+2
    Best Director = x+4
    Best Actor = x
    Best Actress = x
    Best Supporting Actor = x+2
    Best Supporting Actress = x+8
    Best O. Screenplay = x
    Best A. Screenplay = x
    Best Original Score = x+4
    Best Animated Feature = x
    Best Foreign Film = x+2
    Best Documentary = x+4
    Best Art Direction = x+4
    Best Costume Design = x
    Best Makeup = x
    Best Sound Editing = x+2
    Best Sound Mixing = x+4
    Best Visual Effects = x

    I think all the ‘x’ are toss-ups.

  • 8 2-20-2011 at 8:58 pm

    DarkLayers said...

    Sam, I think vfx and Inception is pretty solid. Yeah, it is tricky to classify–because something that inspires splits on say Gold Derby or Gurus can feel clear to folks on either side. Should users vote on what could go either way? And toss-ups are different from upsets (eg Hurt Locker and Sound Editing, Precious and Screenplay versus Original Screenplay or Editing last year.)

  • 9 2-21-2011 at 12:04 am

    RC of strangeculture said...

    I like “The Dudes” thought – not sure if that’s possible or not….????

  • 10 2-21-2011 at 12:18 am

    Kristopher Tapley said...

    It’s not, unfortunately.

  • 11 2-21-2011 at 1:28 am

    Matthias Zucker said...

    I think you should weigh them as you feel the degree of difficulty is of getting it right in each category.

    I’d say Animated Feature, Actor, Actress, VFX, the Screenplays are easy to call, everything else is somewhat fluid. The biggest headscratchers for me are Art Direction, Score, Foreign, the Shorts as always. I could also see Documentary going several different ways. Pick them as you see them.

  • 12 2-21-2011 at 1:34 am

    Hextall454 said...

    In the past I’ve used:
    5: Picture
    4: Director, Screenplays (2), Actors (4)
    3: Animated, FLF, Score, Cinematography, Editing
    2: Art Direction, Costume, Makeup, Song, Doc, VE, Sound (2)
    1: LAS, AniS, DocS

  • 13 2-21-2011 at 1:53 am

    Patriotsfan said...

    I don’t know how possible this is for you to do, but here’s a more objective way you could do it than simply assigning arbitrary difficulty levels for certain categories: look at the statistical spread of the predictions. For instance, if Colin Firth got 98% of the vote, correctly forecasting his win would get you 1 point, where as if you correctly picked Javier Bardem who had only 1% of the ballot, you would 9 points. In other words, you could have a sliding scale like this:

    Picking correct with 90%-100% of people also correctly picking: 1 point

    Picking correct with 80%-89.99999% of people also correctly picking: 2 points

    Picking correct with 70%-79.99999% of people also correctly picking: 3 points

    and so on until

    Picking correct with 0.00001%-9.9999% of people also correctly picking: 9 points

    This way you wouldn’t be guessing the degree of difficulty. You would be using the actual ballots to determine the degree of difficulty after the fact, making it more objective.

  • 14 2-21-2011 at 6:19 am

    Maxim said...

    The answer to me, is obvious: Wait till AFTER the Oscars to decide weighing. Not only will it allow you the hindsight needed to see which categories were truly close. Yes, I know this may seem like a less fun way to do this thing but I actually think that keeping people in the dark would make it more challenging, and ultimately more rewarding.

    And I disagree with Dude’s approach and here’s why. The game theoretic scenario it sets up, disproportionally rewards going for upsets. To put it in layman’s terms, if I want to play in an anonymous Oscar pool and my main abjective is to win I would have absolutely no incentive going with consensus picks. In the desire to stand out from the crowd, I would be then motivated
    to pick upsets not primarily because I actually believe they will happen but because they offer significantly large reward for the risk. When this things up are set up poorly just one correctly (read: luckily) picked upset could make up for three or more incorrect picks. This could still work, mind you, because there’s still thinking involved but setting up weighting properly may require too much consideration.

    As Dude himself said:
    “Doing something like this makes it so that going for those risky upsets could pay off. ”

    I think I’d rather see people pick what they really think would actually win and use the more difficult categories (based on the actual results) for tie brackers.

  • 15 2-21-2011 at 7:16 am

    Andrej said...

    Your points sound reasonable, Maxim, but I think there are a couple of problems with scoring after the Oscar show:

    1. I’m not sure how Picktainment works, but I believe that what you say could render the standings useless. We wouldn’t know how much a certain category is worth until its award is given, right? The thing is that I’m not sure if Picktainment allows you to modify all the categories’s values once the show is over, since it would leave a wide room for cheating. If it allows it, sure, it could be an interesting way, but it if doesn’t, then it’d be horrible to start manually adding and substracting points to all 195 contestants (as of now) depending whether they got the upset right or wrong in all categories, and THEN make the defintive standings with the modified results. Too cumbersome.

    2. Even if we were doing it based on ‘how close were they’, then I still think that it’d be required for all of us to establish before the Oscar show how much a nominee’s win is worth in all categories. That ‘in the dark’ feeling you mention removes any sense of transparency the game has. Also, unless we can all agree in advance on how much each nominee’s worth in points if he were to win in his category, the scoring system might lend itself to conflict among players.

    o_@ That’s pretty much it.

  • 16 2-21-2011 at 7:47 am

    Zac said...

    I would reverse Hextall454’s point system.

    The more obscure the category, the more points it should be worth since the likelihood of a majority of people picking the same movie in the Doc Short category is lower than in the Best Picture category.

  • 17 2-21-2011 at 8:17 am

    Maxim said...

    Thanks for the thoughtful response, Andrej. In regards to your first point, I admit I haven’t really considered the way Picktainment works at all (I actually never used that website before). That’s an oversight on my part. I guess I was more interested in coming up with the approach I thought was most effective without giving much thought to its feasibility.

    I understand your second point but, parodoxically, it is one of the main reasons why I argued for waiting till after the oscars to decide on scoring. This is because I am not so much interested in how close someone gets to making a correct prediction (which is something that cannot be known with absolute certainty for people who made a wrong pick even after the ceremony) as I am in knowing which categories contained true (read: unexpected upsets) .

    In other words, the goal is not to disproptinally reward people for picking something we think could have an upset over people who picked an upset that no one even saw coming. It’s a very subtle point but is something that makes sense to me.

    To make the whole thing more simple, I would say make each category be worth one point by default and then make every category that scores an upset be worth some multiple of 1. (You could go further and decide that multiple based on its degree of apparent unlickliness but that seems like way too much work). The only problem here is that, again, it would require adjusting scores based on actual results. That said, any site that deals with those types of predictions worth its salt should allow you to do that type of recalculation for each ballot easily.

  • 18 2-21-2011 at 8:27 am

    DylanS said...

    I agree with SamC about the locks and relative locks. I also like the idea of weighting categories based on # of BP nominees.

  • 19 2-21-2011 at 8:44 am

    Sam C. said...

    Maybe we could take a poll…have the predictors predict what they think the hardest category is. Sure Best Picture is the big cheese and should be worth the most, I guess, but it’ll be far more impressive for someone to get Costumes and Supporting Actress correct. The most hard-to-pick category–as chosen by the pickers–is worth the most if chosen correctly.

    There are, what, 27 categories, the hardest to pick is work 27 points and the slam dunk–cough, Best Actor–is worth 1.

  • 20 2-21-2011 at 8:59 am

    Andrej said...

    If I’m reading this right, then, for example… in the case of John Hawkes (an outsider) winning, all those who picked him to win get more points that those who voted for Geoffrey Rush (the more expected upset) could have gotten if he had won instead?

    It’s not a bad idea, but I still think that some previous notion of how likely is a certain nominee to win is needed to appropiately score their victories. Probably the polls on the left side could serve as a way to see how much an unexpected win could score, those less voted net more points than the more popular ones. I know it says ‘what do you think it’s the most deserving’, instead of ‘what do you think it’s more likely to win’, but it could still work, we all voted there anyways.

    But this would work as long as Picktainment allows changing the score system in such an advanced fashion, obviously.

  • 21 2-21-2011 at 9:20 am

    Kristopher Tapley said...

    We can’t weigh after the Oscars, either.

    I may end up just taking the easy route and, on a scale of 1-5, weighing the categories in the order of difficulty to predict based on my own take. Though I could try to put together a poll to get your thoughts on that, I guess.

  • 22 2-21-2011 at 9:31 am

    Kristopher Tapley said...

    Okay, I’ve added a poll and a note to explain in the post. Please vote and let us know what you’re thinking.

  • 23 2-21-2011 at 9:53 am

    Bryce said...

    I believe you’re missing Documentary Short Subject. The one that I’d pick as the most difficult to predict…

  • 24 2-21-2011 at 11:27 am

    Kristopher Tapley said...

    Crap. If you’d pick doc short subject as the most difficult, please let me know by commenting here. I can’t change the poll because the code would be different, but I can tally up any votes spoken here. Sorry, guys.

  • 25 2-21-2011 at 11:39 am

    Jeremy said...

    I’ll say what I said last year: I think the shorts should all be worth one point, regardless of weighting. I realize this ignores the difficulty criterion you’ve instituted, but it’s unrealistic to expect more than a handful of readers to have seen the shorts, so we’re at a severe disadvantage on those categories.

  • 26 2-21-2011 at 11:40 am

    Sergiu said...

    I think that the most easy to pick should have a greater weight. I mean if someone picks a total shocker should be rewarded for taking chances.

    Also the more important categories should have a greather weight also. It’s more rewarding to pick the correct Best Picture winner than some short movies that almost nobody saw.

    Also the Foreign, Documentary and Animated should value less since they are secondary categories.

    Let’s say
    Best Picture 10
    Director, actor, actress 8
    Screenplays 7
    Cinematography 6
    Supporting roles 5
    All the others not mentioned 3
    Foreign, Documentary, Animated 2
    Shorts 1

  • 27 2-21-2011 at 11:45 am

    Paul Outlaw said...

    Documentary short is the hardest; supporting actress is the next hardest; then documentary feature.

  • 28 2-21-2011 at 11:58 am

    Maxim said...

    I’ll just repeat what I’ve said earlier: there is a difference between perceived difficulty of predicting a category vs actual difficulty.

    If nearly everyone predicted Inception to win Best Visual Effects and it actually went to, say, Alice in Wonderland, it would be silly to give the person who picked it correctly just the minimum amount of points just because it was perceived as an “easy category”. The point being, a category is a category and some years best score could be as hard as best animated short.

    I understand how weighing the categories after the fact would be impossible (though I insist that for a site like Picketenment it’s a clear oversight), but do we really need a poll to tell us that documentary shorts are hard? Why not just go with a simple one point per category prediction and save the shorts for tie brackers?

    In all likelihood, even under one point rule, the only candidates left for contention will have gotten nearly everything correctly anyway.

  • 29 2-21-2011 at 2:12 pm

    Guy Lodge said...

    I’d say Documentary Short is the hardest, followed by Live Action Short. Amazed so few people picked the latter — what do you all know that I don’t?

  • 30 2-21-2011 at 2:16 pm

    Guy Lodge said...

    Maxim does have a point: if somebody has the balls to predict an upset in a category widely perceived as locked, only to be proven right, they deserve more than the minimum number of points.

    For example, Adapted Screenplay would have been deemed an easy category last year — but anybody who went out on a limb by predicting Up in the Air would lose to Precious should have been duly rewarded.

  • 31 2-21-2011 at 4:47 pm

    Fitz said...

    Why does supporting actress have so many votes? If anything the documentary or animated shorts should be weighted most.

  • 32 2-21-2011 at 5:34 pm

    Speaking English said...

    Agreed. Melissa Leo wins the majority of the precursor awards, the Golden Globe, the SAG, and the BFCA and she’s not considered by-and-large the frontrunner? I don’t get it.

  • 33 2-22-2011 at 6:51 am

    Maxim said...

    Thanks, Guy.

    “Why does supporting actress have so many votes?”

    I can understand that. On the one hand, there’s a lot of evidence that Leo is a frontrunner. On the other’s there’s just the opposite. In other words, it’s the perceived closeness that makes it hard. People are torn.

  • 34 2-23-2011 at 8:19 am

    Evan said...

    These results are wonky. We’re seriously going to judge Best Supporting Actress, where we have 102,281 precursors but there’s still *a little* mystery, as more up-in-the-air than the shorts, where there’s absolutely no precursor and nobody knows the make-up of the voting body in that category?

    Kris/Guy, I think you guys would be much better judges of what’s difficult to predict than the less informed (and main category-skewed) public.

  • 35 2-24-2011 at 11:40 am

    Kristopher Tapley said...

    Never mind on this. The damn poll stopped working for some reason. I’ll just decide myself and you’ll all DEAL WITH IT! :)

  • 36 2-24-2011 at 11:43 am

    Kristopher Tapley said...

    Also, like I said, it’s impossible to be so nuanced with this silliness about giving someone their due for picking a total upset. It’s just a friggin’ pool, so, again, you’ll have to deal with it.

  • 37 2-24-2011 at 11:50 am

    Maxim said...

    Well, you *did* ask for opinions. So I guess you’d have to DEAL with the responses ;) .

  • 38 2-24-2011 at 11:51 am

    Kristopher Tapley said...

    I also said immediately that I couldn’t do what you asked, but you stated the point again for some reason. Trying to please everyone, you end up pleasing no one. Meh.


  • 39 2-24-2011 at 12:05 pm

    Maxim said...

    You can’t easily stop the thinking process once it starts like that.

    Besides, all of my subsequent comments were repsonses to others and adjusted for realism (while still maintaining the hip PG13 rating). There is value is entertaing hypotheticals. It’s not like life is vexing or anythig, right?


    What are you, 12? We went a whole two days without an annoying comment like that on this column, but I knew one was coming…