Update: Scott Brown did win the election. Here is an article I wrote after his victory. Here is a visual glance at voter results by town.
From the twilight zone department: various polls put Scott Brown, Republican challenger for the special election for Ted Kennedy's former Senate seat in Massachusetts, within striking distance of the Democratic favorite, Martha Coakley. Public Policy Polling yesterday released a poll suggesting a statistical dead heat in the race, while Rasmussen released poll results on January 4th showing Coakley with a 50% - 41% edge. The Boston Globe, however, counters with a poll showing likely voters leaning 53% to 36% towards Coakley - more in line with historical results in Massachusetts.
Massachusetts' Voting History
The last Republican Senator elected from the Bay State was Edward Brooke, voted out after two terms in January of 1979. He was succeeded by Paul Tsongas, who served one term, then by the current Senator, John Kerry (also the 2004 Democratic Presidential Nominee). Ted Kennedy served as senator of Massachusetts from November 7, 1962 until his death on August 25, 2009. He was succeeded by Democrat Paul Kirk (by appointment), who will not run in the special election. Since 1979, the Bay State has only voted Democratic in senate elections. Since John Kerry took office in 1985, he and Ted Kennedy were the only Senators elected from Massachusetts.
How about Presidential elections? Since 1988, Massachusetts has voted for Democratic candidates for President. In that time, Massachusetts provided two of the Democratic candidates for President - the aforementioned John Kerry in 2004, and Michael Dukakis in 1988.
In the House of Representatives, Massachusetts sends ten members (since 1990, when the census resulted in the loss of one seat). The last Republican Representatives, Peter Blute and Peter Torkildsen, were voted out in 1997. I think you'll agree that Massachusetts is a strongly Democratic state.
Senate and Presidential Votes, by the Numbers
Let's go back to 1966, when Ted Kennedy first joined the Senate, and take a look at the voting trends in every Senate and Presidential election in Massachusetts. I got my presidential data from Dave Leip's Atlas of U.S. Presidential Elections, and my Senate data from Wikipedia.
- US Senate Election Results since 1962, Massachusetts
There you have it; 17 Senate elections since 1962 (when Ted Kennedy won the special election to replace brother John F. Kennedy in his seat vacated to become President). In those 17 elections, 2 Republican wins. In further analysis, results from 2002 will be thrown out since the state Republican Party didn't even field a candidate. In every other election outside of 1966 and 1972, the Democratic candidate took over 50% of the vote. The Presidential results tell a similar story.
- US Presidential Election Results since 1966, Massachusetts
The outliers in this chart are the two slight Ronald Reagan victories in 1980 and 1984, and the Perot candidacy in 1992 (in 'other'). Of special note, in 1972, Massachusetts was the only state to vote for Democratic candidate George McGovern over Republican candidate Richard Nixon.
Back to Scott Brown
So, how unlikely would it be for Scott Brown to pull off a Massachusetts Senate victory? The 2008 voter registration breakdown was 36.95% Democratic, 11.62% Republican and 50.75% unenrolled (with a few other minor parties representing the rest). Since 1962, with the 2002 outlier removed, the Democratic candidate for Senate has garnered an average 58.95% of the vote, with 38.78% going to the Republican and 2.27% to other candidates. In Presidential elections, the averages work out to 57.12% Democratic, 36.91% Republican, and 5.97% other. Combining Presidential and Senate elections (a slightly shady practice perhaps, but it gets me to 28 samples since Ted Kennedy took office), the numbers move to 58.16% Democratic, 37.98% Republican and 3.86% other. Since averages don't give a good sense of variance, it may be interesting to take the standard deviation and have a rough idea of the probability of certain results from the statistical data we have. Note that this is really just a rough guide and assumes a normal distribution of election results. Since you know the caveats I am offering, also note than any poll mentioned has to deal with similar issues (namely, inferring state-wide results and turnout from a small sample of population). One thing I can avoid: selection bias (one example: people who answer polling firms may be biased in a certain manner), since I am sampling actual voter turnout (but I digress).
In the elections mentioned above, the standard deviation in Democratic votes is 10.11%, in Republican votes it's 10.69%, and in 'Other' votes it's 5.68%. A similar analysis on election results since Kerry has been in office (1984, without the 2002 Senate election results - only 15 samples) yields slightly different numbers: 59.12% Democratic (7.36% std. dev.), 35.93% Republican (9.43% std. dev.) and 4.95% 'other' (6.60% std. dev.). And for all elections since 2000, except the 2002 Senate race, the numbers are 65.23% Democratic (4.99% std. dev.), 29.95% Republican (8.75% std. dev.) and 4.83% 'other' (5.38% std. dev.). Of course, since 2000 that leaves us with only 6 samples.
For our purposes, we will assume that the candidates represented by 'other' will take that average 4.95% of the vote in the special election. Therefore to see a Scott Brown victory, we need exactly half of the remaining voters plus one voter. We can therefore look for 47.525% of the vote for Brown, since that one extra voter will be insignificant in the percentage. In fact, let's look at an election that breaks down like this (mindful of how precise our percentages are: this is sloppy but educational) :
47.525% for Scott Brown, 47.525% for Martha Coakley, and 4.95% for 'other' challengers- perhaps most of it going to Joe Kennedy.
We need to find out the probability suggested by the normal distribution that Scott Brown receives more than 47.525% of the vote, and Martha Coakley receives less. Using an average of 58.16% Democratic votes and a 10.11% standard deviation, we find that there is a 14.64% chance that Coakley will get less than 47.525% of the vote. Similarly, we find that Brown has an 81.40% chance of getting less than 47.525% of the vote. Using our numbers since 1984, Brown's chances fall even further. The numbers imply an 89.06% chance his share of the votes will be less than 47.525%, and a 5.76% chance Coakley's share is less than 47.525%.
Normal Distribution vs. Votes
Numbers don't say much, considering votes are a measure of mood, turnout, enthusiasm, and any number of factors. It's impossible to put an exact number on the odds of an upset in the Senate special election. For sure, the odds are very low in a state as historically Democratic as Massachusetts. In a state that hasn't had a Republican senator since 1979 (and has probably become even more Democratic recently!), Brown's chances are very slim. However, his odds aren't impossible. A victory for Brown would take a perfect storm: high Republican enthusiasm combined with low Democratic enthusiasm; unenrolled voters in Massachusetts turning out to vote and leaning heavily to the Republican side, and possibly a third factor like a ridiculous amount of luck. It will be interesting to see this one play out.
I hope I gave everyone enough caveats, but feel free to argue my numbers. Using a normal distribution to model votes is disingenuous, I know, but I wanted to convince you that an upset in Massachusetts would be a huge deal, a monumental shift, if you will. Also, I wanted to write a political article which I didn't talk policy or suggest trends which would favor either side - just historical results. Also interesting are analyses from Five Thirty Eight and Pollster on the actual polls which prompted this article. Post your thoughts!
