Prognosticators – An invitation to the dance (updated 8oct12)

George Rebane

Over the year’s RR has attracted a bevy of readers with keen minds and extraordinary abilities to peer into our collective futures.  The certitude of some of these Que Sera predictions have most certainly been breathtaking to behold.  In this election season these prognositcators make their considerable talents known to one and all at a prodigious rate in these pages.  But how good are they really, or are they just blowing some cleverly contoured smoke?

Not too long ago we ran into a similar question about ourselves as members of a small informal investor group which met about every two weeks or so to puzzle out the markets and discuss what each was investing in or considering.  Assessing the future of certain critical markets was part of our regimen, and we wondered how good we were at it.  The answer to that question will hold until another time, however the methodology we used to obtain the answer may be of interest in the present.

Being a ‘quant’ versed in these things, I devised a simple scoring algorithm based on the probabilistics of the boxcar (or uniform) density function.  The boxcar is the simplest measure of a random variable like the predicted value of the closing price of gold in London on 31 October 2012.  Of course, we can pick any other quantity we like, and I’ll get to that in a moment.  But first, here is what a boxcar density looks like for some random variable x.


The boxcar says that with certainty (probability = 1), x will fall between xLow and xHigh  (more formally x >= xLow AND x <= xHigh).  So the length of the boxcar is simple (xHigh – xLow).  Since the height of the boxcar is uniform over the range of x, its value is the constant 1/(xHigh – xLow), and it is called the ‘likelihood’ of the boxcar density.

So we picked some markets of interest and submitted our highs and lows for them at a given date.  Scoring was simple.  If the market value fell within your boxcar, you were awarded a score based on the likelihood value of the boxcar that you predicted.  As you can see from the formula above, the shorter your boxcar, the higher your score.  If you think about it, that’s the reasonable way to assess the worth of any quantitative prediction.  Consider the relative worth of predicting gold to be between $1,500 and $1,550 on New Year’s Eve, and the prediction that it will then be between $1,000 and $2,000.  Your score will be zero if the actual value turns out not to be in your boxcar.


To make things concrete and illustrate the simplicity of it all (see above figure, click to enlarge).  I have extracted an example spreadsheet illustrating four competitors who have predicted the value of the SP500, Gold, Crude Oil, and 10-year Treasury interest rate on the given date.  The actual 31dec09 values are shown as are the predicted boxcar limits and resulting scores for each competitor.  The exact score value is calculated by taking the likelihood and multiplying it by the realized value of the predicted random variable (here the four markets).  Total score across the markets is just the sum of the individual scores with the ‘Leader’ indicated.

We published the intermediate scores of the competition to see how everyone was doing as the end date approached.  Our payoff was that the losers would pay pro-rata for a lunch or dinner attended by everyone and their guests. Lots of fun.

The Proposition.  So I propose that those who wish to participate state their predictions under the soliciting post.  I will do the math to report progress and tabulate the winner.  Each competitor would agree to send a check for a pre-specified amount to a 501c3 (tax deductible) recipient designated by the winner and in the winner’s name.  The winner will supply all the relevant recipient data.

To gauge interest, I offer the invitation for a $20 contribution to predict the first publication of the of a poll by Gallup on what fraction said Obama won the 3oct12 debate, AND the same poll’s fraction who favored Romney as the victor (we’ll leave out any ‘undecideds’).

Please feel free to suggest other such charitable prognostications we may have an interest in holding a competition - e.g. predict what fraction of voters will vote for Obama, Romney, ...  .  The only requirement is that we have a robust, accessible, and mutually accepted adjudication source.

[4oct12 update]

"So, who do you think won, honey?"

[8oct12 update]  The resulting scores for the competition are posted below.  All entrants are invited to send their checks to SESF's TechTest2012 Scholarship fund with much appreciation.

 

Comments

[George Rebane]

OK then, I'll start off and say that 44-49% will favor Obama, and 47-52% will favor Romney.

[Russ Steele]

I will bet $20.00 that the crickets win in a landslide, the noisy left will not take up your proposition. Regardless of who we think wins the Press will declare Obama the winner. Given this bias, 47-52% for Obama and 44-49% in favor of Romney. Not what I am hoping for but, the most likely outcome. Romney wins if he can just raise some doubts that maybe Obama is not the best choice among those that were Obama supporters. Romney does not need the winning knock out punch, which would make him the clear winner.

[Steven Frisch]

I'm in for $20

48-53 for Obama and 43-48 for Romney

No more major knowck-out punches for either side and I expect the race to tighten slightly in the final 2 weeks.

[George Rebane]

Most interesting that everyone so far has picked 5 point ranges, and SteveF's has the least overlap. Entrants so far are then -

GeoR 1011pm - Romney(47-52), Obama(44-49)
RussS 726am - Romney(44-49), Obama(47-52)
SteveF 752am - Romney(43-48), Obama(48-53)

[TomKenworth]

Romney and Ryan, quite a pair. One screams, "Midnight!" while the other yells, "Boxcars!" and those are just about the odds of them winning in this election...

[George Rebane]

TomK 936am - Soon we'll have a little competition in which you can put a little backbone into that prognostication puffery.

[TomKenworth]

George 10:25, the end result is all or nothing, the spread is interesting, but if you really believe Obamaists will blindly follow whoever offers the most handouts, then you probably will wind up seeing my odds in print. At 1 in 36 odds, 50% of the time Romney gets elected roughly once every 25 elections. Too many folks have figured out that the trickle faucet isn't run by Farrah folks at the top, and that the drips are dry by the time they reach them. When gambling, people will go where's there's at least a slim hope, hint or unreal promise of success. Romney 47&ed this election away.

[George Rebane]

TomK 328pm - Not sure how this comment responds to my post about scoring predictions in general, and joining in the described competition for next Wednesday's debate.

[TomKenworth]

Thought twas about the lection, not just the first debate. My bad.

[Paul Emery]

So here's a question to ponder

If Romney was such a great Governor and he's using his record as a showcase for his run for the Presidency why is he loosing by 24% in the state he was governor of? Not exactly a job recommendation from his former employer.

http://www.realclearpolitics.com/epolls/2012/president/ma/massachusetts_romney_vs_obama-1804.html

[George Rebane]

TomK 431pm - No "bad" at all, just a bit premature to start making predictions on the election. But we'll run one on it directly, and I look forward to your participation.

PaulE 1142am - with the celebration still in full swing at the Emery house, are you going to enter your predictions on Romney's and Obama's debate approval numbers?

[billy T]

Dr Rebane, here are some predictions. They really don't fit into the boxcar prognostications that can be gauged by polls, just results.

Within 6 months, little Timothy G will be shown the door whoever wins the Presidential Race.
The Bush tax cuts will not be passed in total. It will be a mixed bag of saving some of the Bush plan and throwing out others, by August 2013.
The payroll tax cut will not be extended. Period.
The debt forgiveness of those doing a short sale will no long be continued. Thus, if you buy a house for 300k and sell it in a short sale for 200k, you will be taxed on the 100k debt forgiveness as taxable income. Another Bush plan down the toilet.
The Fed will continue buying bonds, thus all risky bad paper will be bought up by the Fed. This will be known as "the place bad bonds go to die."

[George Rebane]

billyT 450pm - Good crisp calls. However, the Geithner one is a gimme since he already wanted to resign, and stayed only by Presidential plea. He is itching to resign soon after the election - like on 7 November.

[billy T]

Just warming up with slow balls, Dr. Rebane. Don't matter who wins the first debate. The media will say it is Obama by a mile. Even people who don't tune into the debate will say Obama kicked Romney's lily white behind.

I will come up with some bolder predictions in the coming days. It will be like that old TV show One Step Beyond which "is based on scientific fact."

[Paul Emery]

Easy one. Obama by 4 because of his likeability.

[George Rebane]

billyT 533pm - here we're actually not worried about what the media say, but what Gallup reports on the first poll after Debate1.

PaulE 621pm - please make give us the ranges for Romney and Obama that reflects that strong feeling (see my 853am for your competition).

[billy T]

Gallop poll. Hmmm. Well, the first one is on the economy I believe. Fair moderator. Obama without the prompter. Obama 46, Romney 50. 4% don't know. That is the polling up to 48 hours, not just overnight.

[Russ Steele]

billy [email protected] 07:45 Excellent question. I thought that Gallup used a three day rolling average, so for the a clean showing of the debate impact we need at least three days of polling? If I am wrong please send me in the right direction.

[George Rebane]

billyT 745pm & Russ 926pm - Mr Tozer, you picked only a single percentage for each, instead of a range like the rest of us. According to the described scoring algo you will surely win if Obama gets 46% OR Romney gets 50% of the polled who favor them (your score will be 'infinite'). However, you will also lose completely if neither of those numbers obtain, since you have specified two zero ranges (think of it as having no wiggle room). Please confirm that this indeed is your entry for this competition.

Our contest here will be adjudicated as described - by the first published post-debate Gallup poll that cites the two required percentages for Romney and Obama.

[billy T]

Dr. Rebane. I am just throwing out a prediction, not a parameter. I know the rules, thus I am not interested in winning. Unless, of course, money is taken out of a 501(c) and put in my pocket. Lets say the winner has to be paid out of my least favorite charity and put in my hot little hand. That would make it more interesting and fun.

[George Rebane]

billyT 324am - I understand that you are NOT entering the competition. My error; apologies.

[TomKenworth]

The candidates should be required to spend two hours a day on FB in a public conversation, where the speed and quality of the responses can be noted, with dual live feed showing exactly what websites or advisers they consult with before responding, as well as facial reactions, etc. Two or more computers in a house is quickly becoming the norm, so many would get a real intimate look at the candidates under pressure.

[George Rebane]

TomK 945am - not sure about the relevance of this comment here. Perhaps you meant it for 'Ruminations - 2oct12'?

[Steven Frisch]

Just confirming....just the three of us are in (George, Russ & Steve F.)

[George Rebane]

SteveF 541pm - correct.

[Steven Frisch]

I would give it to Romney so far by a solid 5-10%

[Steven Frisch]

I would still give it to Romney by a solid 5-10% at this point.

[THEMIKEYMCD]

Obama got owned. Here are the nights best tweets:

Bill Maher ‏@billmaher
i can't believe i'm saying this, but Obama looks like he DOES need a teleprompter

Keith McCullough ‏@KeithMcCullough
"Jim, you may want to move onto another topic" [email protected] on the economy

Joe Hagan ‏@joehagansays
Worse wedding anniversary ever? #obama #debate2012

@Heminator That wasn't a debate so much as Mitt Romney just took Obama for a cross country drive strapped to the roof of his car.

@NoelSheppard: Apparently Obama spent as much time preparing for this debate as he has reading national security briefings!”

[THEMIKEYMCD]

The.Best.Tweet.of.the.night

David Burge ‏@iowahawkblog
Poor guy, couldn't get it up on his anniversary.

[George Rebane]

SteveF 706pm - agreed.

ps. Steve, Russ, help me find the first Gallup poll for adjudicating that scores.

[Steven Frisch]

Of course you guys do realize that Mr. Romney just turned on several of his campaign stances right?

[George Rebane]

SteveF 813pm - didn't hear anyone on PBS or Fox accuse him of that, but are you saying that Romney cheated?

[Russ Steele]

Steven @ 08:13

Details, Details, Details. We just watched the body language version with the sound turned down. Romney +1, Obama -1 We are now watching the spin rooms. Luntz focus group was awesome! Biggest shift ever, according to Frank!

[Russ Steele]

[email protected]:09

I just signed up for the Gallup e-mail Alert for breaking results.

[Steven Frisch]

No I'm not saying he cheated, Im saying he shifted his position on several issues.

[Steven Frisch]

But I give this one to you guys....I think Romney won by about 8%......

[George Rebane]

SteveF 834pm - Sorry, I forgot to put a smiley face at the end of my 816pm.

Were we to do a postmortem on the debate's national scoring, I would say that Romney is in the 52-55% range, and Obama in the 41-47% range. Thoughts?

[Ryan Mount]

Rhetorically, Romney owned the night:

"I've got 5 boys. I'm used to people saying something that's not always true but just keep repeating it and ultimately hoping I'll believe it."

It's that kind of clever language that landed a serious punch tonight.

I have no idea how to bet on this because I'm quite dense. Can I just give money to some organization and be done with it? Steven, you can pick it for me.

[George Rebane]

RyanM 846pm - why don't you wait until SteveF wins before he picks your 501c3. This is self-serving because I will designate the SESF TechTest2013 scholarship fund as my 501c3, and I'd like your $20 in there too ;-)

[Ryan Mount]

Deal George. And a round of drinks for all of you guys on me. That's something I know how to do well.

[Steven Frisch]

I'm not going to win, I think George is. But lets wait until Gallup speaks and George declares a winner.

[George Rebane]

RyanM 851pm - you are truly a gentleman. I look forward to that, and I will buy the second round of libations.

Here was Gallup's pre-debate performance sentiments poll.
http://www.gallup.com/poll/157835/americans-predict-obama-better-job-debates.aspx

[George Rebane]

Well, this is turning out to be harder than I thought. So far Gallup is silent on last night's debate, and I thought they'd be right up there with one of the first results of public sentiment, since they published the pre-polling sentiments.

Googling 'debate poll' gives a snootfull of links to all kinds of sites that generally agree that Romney won. But if they cite poll results, then they're all over the map - ranging from high 20s to low 70s that Romney won. In some polls a big fraction thought it was a tie. And some outfits are doing multiple polls ranging from last night's quick reaction to today's more considered, and who knows how far that will go since their results change with the 'time late' of the poll.

But no Gallup poll yet. Help!

[Steven Frisch]

I noticed that there was a big chunk saying "It was a Tie", (although I have to wonder what debate they watched)....we await for Gallup.

[Michael Anderson]

I won't be playing in this arena (due to too many other current distractions) but I am enjoying the repartee. Thanks George.

My favorite line thus far:

"@Heminator That wasn't a debate so much as Mitt Romney just took Obama for a cross country drive strapped to the roof of his car"

Priceless...