Friday, November 28, 2008

New Toy

I just stumbled on this new gadget created by Doug Beeferman. It ranks Google search terms by state popularity and shows correlations of the search term with different variables. If you're curious about regional differences it is kind of a fun deal.

Here are a couple of interesting searches with a high MN ranking:

unemployment - MN ranked 7th in the USA for volume of people searching about unemployment.

quality of life - MN ranked 9th

ice fishing - the stereotype holds, Minnesotans are 2nd in searching about ice fishing.


You really can't draw large conclusions about the relationships, but it's a fun time waster.

Check it out: http://statestats.appspot.com

Thursday, November 20, 2008

Mator RT unveils a program that provides free parking under Gaviidae Commons during the weekends this Holiday Season. Talk about a bailout. No word if overnight parking is available. If so, I predict my car staying there many Saturday night. Just doing my part to keep the economy rolling. Everybody has to sacrifice.

Tuesday, November 18, 2008

Facebook is the New Rasmussen (in 2-4 years)

Just before the election for MN Senate, the Degausser pointed out that Franken had a handy lead in Google Search trends in Duluth that could translate to a win for him in the area. After doing a little bit of research into the emerging world of political analytics, I have seen the future. And man, is it scary/fascinating/scary.

Just a quick primer: analytics is the process of using internet data such as searches, page views, and unique visitors to identify trends. These data are being refined in the business world to better market products to buyers. In the political realm, these data will become the new polling numbers.

In 2006 mid-term elections, researchers looked at candidates Facebook entries and created models that equated supporters to voters. They found that challengers were more likely to use Facebook as an organizing tool and that it could potentially add 1%-3% to the candidate's total vote.

Analytics are just beginning to be used by political scientists and campaigns. Most of the data is scattered and more research needs to be done to discover total impacts, but in a few election cycles, the internet will become (or maybe it has already) a 24/7 polling place.

Just for fun, let's look at some data collected from Al Franken's and Norm Coleman''s web presence over the course of 2008.

NOTE: Data presented here are from January 1, 2008 to November 2, 2008.

I collected Google Search trends, Facebook supporters, Youtube Channel subscribers, and basic website visitor stats collected from compete.com to see if there were any interesting patterns happening.
Throughout the course of the election, search trends were pretty similar between Franken and Coleman. However, right before the election, more people started searching for Franken at rates far beyond Coleman's. The graph above represents average search volume from Jan - Nov. 2. Anything above 1 represents larger than normal search volume for this period. So, by Nov. 2, there was a 24% larger search volume for Franken than Coleman. It is unclear what affect, if any, these searches had on voting outcome.

Looking at the top 8 MN cities by search volume, Franken lead in each except Prior Lake.


Election Results

Franken Coleman
Rochester 48% 51%
Minneapolis 69% 19%
St. Cloud 54% 42%
St. Paul 61% 25%
Hopkins 57% 38%
Mankato 60% 39%
Prior Lake 38% 57%
Duluth 62% 37%

There appears to be a relation between final election results and the volume of searches.

Looking at search trends through the election season and comparing them to the average of the polls conducted through the same time show even more relation. The last data point for the poll line is the final election result.

One interesting point however, is that there is a statistically significant and positive relationship between search volume and poll results. The more search volume, the higher the poll numbers.





It's obvious that Franken had a larger web presence than Coleman, but it is unclear how this presence translated into votes. For now, look at the data and make your own conclusions.

Monday, November 10, 2008

All of the Floridian Drama, Without the Warm Weather

Minnesotans have backstage passes to one of the greatest political dramas since the 2000 Presidential race. Interestingly, this drama is seemingly more a product of weak candidates than a great partisan split. Recent articles in the Strib and on MPR suggest that many people who voted Obama did not cast a vote for any candidate in the Senate race. This non-decision means that either there was a mistake made in the marking of the ballot or there was simply no vote made. Either way, election officials are determining the intention of each ballot. Here’s some map pron via the AP that shows the concentrations of people that opted out of voting in the Senate race. St. Louis Co. leads the way. Way to go Northland, we’re finally #1 in something.

aleqm5jjt59oy3eyakbanr6cl6jl49z5uq

A good portion of the attention from this ‘no-vote paradox’ is being focused on St. Louis County. Coleman’s campaign recently challenged 100 ballots coming out of Mt. Iron that all went for Franken. Coleman’s campaign manager also called the recount results from the Iron Range ’statistical dubious’. Is there some trickery going on up north or is there some other explanation? Let’s do a cursory analysis (insert cheers or jeers).

Last week The Degausser wrote about how the Northland delivered heavily for Franken. St. Louis County, the County that the Coleman campaign is questioning ballot totals from, went to Franken by 22 points. However, the totals from last week to today have changed as election officials are certifying results. Look at how things have changed. Franken has gained 395 total votes from Lake, St. Louis, and Carlton Counties. Coleman actually lost 2 votes.

Votes Gained or Lost Since Nov. 5


Franken
Coleman
Barkley
Aitken
-5
-2
0
Carlton
54
0
0
Cook
0
0
0
Itasca
0
0
0
Koochiching
0
0
0
Lake
246
0
0
Saint Louis
100
0
0
TOTAL
395
-2
0

We can see how Coleman’s campaign manager would be upset, but what are the probabilities of evil shenanigans?

The precinct in question is Mt. Iron P1. According to the MN Secretary of State, there were 891 registered voters on Nov. 4 in this precinct. Here are the breakdowns:

Mt. Iron P1 Franken Coleman Barkley TOTAL
New 506 211 95 812
Old 406 211 95 712





New % 62% 26% 12%
Old % 57% 30% 13%

The election was never in doubt for Franken in this precinct, he was up nearly 2 to 1 before the re-count. Also it appears that that the 100 vote gain was simply human error in totalling rather than lost votes being found, which changes the probability of fraud.

For arguments sake, however, let’s assume the new votes were found votes. What would the probability that they would all be Franken votes? Here is what we need to know to do the calculation

  • The number of votes left (179) = Total registered voters (891) - Total votes cast (812) + 100
  • Changed votes = 100
  • Probability votes going to Franken = 57% (original win percentage)
  • Probability votes going to Coleman = 43%

The expected score out of these assumptions, according to a binomial probability equation is 102. This means that the 100 votes Franken got is actually less than the statistical expectation.

One thing to be aware of is that this assumes that the Mt. Iron P1 district has a normal distribution of partisanship. Considering the heavy Obama vote and long standing DFL affiliations in the area, partisanship is not split 50/50. Although the 100 vote windfall was a calculation error and not found votes, numbers suggest Franken would have fared better if the ballots were found in the trunk of somebody’s car and entirely within the realm of possibility given the political make-up of the precinct. So, yes, the 100 vote appearance is strange, but surprisingly, it is statistical aberration that did less damage to Coleman than odds would suggest.

Thursday, November 6, 2008

The Duluth News-Tribune is Wrong OR Why Forrum Communications Cannot be Trusted

I was reading my hometown paper just now and they are saying that Northland Dems dissed Franken. In it, they posit that the Northland didn’t deliver for Franken and that Barkley played spoiler. They also say that if just 600 more people would have voted for Franken, everything would be different. I disagree with their assesment.

The DNT suggests that because 77% of St. Louis County went for Obama and only 55% went for Franken that Franken failed. Comparing Obama to Franken is like comparing a Ferrari to a Yugo. No way is this an apples-to-apples comparison. Franken’s campaign never really got its shit together and is lucky to be where they are now. As the Degausser showed yesterday, he still managed to destroy Coleman in the Northland. This shouldn’t have happened and it is more a testament to people hating Coleman more than liking Franken.

If the DNT wants to blame somebody, which is retarded because Coleman shouldn’t have made it such a close race, they should blame Hennepin or Ramsey County voters. They voted less for Franken, had less turnout, and have a higher concentration of voters. There were more opportunities to win in the Metro-Area than in the Northland. Franken didn’t pull it out, even in a heavy DFL/Dem year.

If Ciresi was the DFL Senate candidate, he would have pulled more votes up North and probably would have won the election outright.

I have a sneaking suspicion that this article in the DNT is Forrum Communication’s way of saying that the Northland is moving more into a Republican/Independent mindedness. This has been Forrum Communication’s stated mission for the newspaper ever since they took control of it. Forrum Communications endorsed Coleman and owns 22 papers in MN. Moreover, their endorsement was made on the basis of one man’s (Steve McLister) opinion and forced onto the opinion pages of all affiliates. Could it be that Forrum Communication’s endorsement swayed enough voters for Coleman? Instead of blaming the Northland, the DNT should look at their own.

Wednesday, November 5, 2008

Why Franken Should Kiss Duluth’s Ass

Yesterday on the Degausser we suggested that MN’s Arrowhead region could tip in favor of Franken. Boy howdy did it.

Because of the impending recount, all results reported here are technically unofficial, but Franken carried the region by 18 points (33,560 votes).

For those of you metro-centric folks who refer to anything outside of Minneapolis/St. Paul as outstate, here’s a map of the Arrowhead region (shaded in red). The counties that make up this region are Aitken, Carlton, Cook, Itasca, Koochicing (Kooch for those in the know. Or, if you want to get fancy, combine it with Itasca to get Kootasca), Lake, and Saint Louis. The major city here is Duluth (my home town, woot!).

arrowhead-region_minnesota1

Anyway, this region is reliable DFL territory. However, due to Franken’s rocky start and lack of strong ties outside of the metro-area, it was up in the air as to how he would perform in the North Country. Apparently, Franken went over well. Here are the numbers:

Franken Coleman Barkley
Saint Louis 64,319 38,267 14,275
Lake 3,304 2,434 863
Cook 1,620 1,206 458
Carlton 9,517 6,099 2,555
Aitken 3,897 3,617 1,770
Itasca 11,544 9,264 3,182
Koochiching 3,091 2,845 721
TOTAL 97,292 63,732 23,824
Percent 52.6% 34.5% 12.9%

In Duluth, Franken won by more than 20 points and St. Louis County had the highest percentage of Franken voters than any other county in the state, including Hennepin. Duluth also had a whopping 85% turnout rate, which will probably be one of the highest in the country.

The Arrowhead region provided Franken with over 8% of his total votes. Given the turnout and wide margin that he beat Coleman by in this area, Franken owes his recount to Northeastern MN, and specifically to Duluth.

untitled8

Tuesday, November 4, 2008

The Importance of Being Analytic

Web analytics, the use of internet search traffic and page visit data, is just beginning to be used by campaigns to determine marketing efficacy. This practice is still in its infancy and the data is a little sketchy, but compiling data from the internet will be huge in future elections.

The candidates that can effectively use web analytics in the next election cycle (or this one, for that matter) will have a huge advantage over an opponent that cannot use the data. After the election, the Degausseer will attempt to unravel web trends of the MN Senate race to see if there was anything happening on the internet that could have evidenced the final outcome.

In the meantime, here is some quick info pr0n.

Charts show the google search indexes of Franken and Coleman through Oct 4 to Nov 2. Higher index means more searches. Again, since there is an error associated even with this data, searches are statistcally tied. Franken leads in internet searches by city, with more Duluthians searching for him (per capita) than any other city in MN.

Is Franken going to carry the Arrowhead region?

untitled11untitled7

Monday, November 3, 2008

Could Coleman Eke One Out?

The final round of Senate polling before the election was completed over the holiday weekend. While little kids were learning the finer points of bartering and Halloween tricks were being played (or turned), the rest of MN was getting sampled.

The results of the these final polls show a shift toward Coleman. There have now been a total of 49 polls conducted since last year. Although the results of the final few polls show that there is a statistical tie between the two candidates, there are a few data points that suggest Coleman may eke out a victory tomorrow.

First off, here is the line graph of all polls.



Coleman has come back from mid-October lows. Franken has hovered around 40% pretty consistantly.

Looking at the overall trends, Coleman has taken a minute lead, but because of variation in the data these trends are too close to call. The take away point, however, is that Coleman has made a come back and Franken has stagnated.



Looking at spread advantage (the difference between poll numbers), the trend line has broken in favor of Coleman for the first time since mid-September. Again, this is too close to call, but trends have shifted in Coleman’s favor.

Finally, looking at average highs and lows for all polls, Coleman has a 3-point advantage. This advantage is again within a statistical margin of error, so the race is still a toss-up, but considering the overall data trends, Coleman appears on the verge of eking out a victory.

Poll Averages .......High .....Low .....Total
Coleman..................48....... 40......... 44
Franken.................. 45....... 37......... 41

In races this close, campaign organization become the key to winning. The election will be decided on who has the best behind the scenes operation. Who knocked on more doors? Who had the most face to face contact with voters? Was it the GOP or DFL?

If you’re a Franken fan, you hope to God that Franken has hitched his wagon onto the organizational juggernaut that is Obama. The GOTV effort from that campaign may push enough DFL minded folks out to the polls to vote for Franken giving him the edge.

If you’re a Coleman fan, you hope that the storied GOP voter mobilization efforts of the last few presidential election cycles still work. MN is clearly going to fall in Obama’s favor, but will the GOP mobilize enough people to come out for Coleman? We will know shortly.

If you’re a Barkley fan, you are justifiably bitter. The political egos of Franken nd Coleman dominated an election season that should have been a policy wonk’s wet dream. Instead of measured responses to very serious issues, you got a high school pissing contest.

Sunday, November 2, 2008

Last Stop on the Straight Talk Express…

… is New Hampshire! (At least before the campaign does the obligatory return to the home state for photo-op vote and Election night party.)

As the Presidential Campaign winds down into its final 48, this bit of news struck me:

Tomorrow, McCain spends part of the day in Pennsylvania before heading to New Hampshire, where he’ll hold his final town hall of the campaign in Peterborough, a place that McCain regards as something of a lucky charm. He held his first town hall meeting there during his 2000 campaign and wound up there on the eve of his comeback primary victory earlier this year. It’s “where our campaign has been rescued and resurrected many times before,” Rick Davis, McCain’s campaign manager, told reporters yesterday.


Ah yes … you know they call it the comeback.

Q: Will it work?

A: Not a chance.

Despite the grand symbolism McCain is evoking with the return to the resurrection of his campaign during the 2008 GOP primary - which led to the splintering of the fundamentalists between Romney and Huckabee, and let McCain steal the moderates from Guiliani - but even his advisers aren’t buying the prospect of a sequel:

… privately, McCain and his aides are unsure if it will be as lucky this time around. Down by 13 points in the state according to poll released yesterday, McCain and his aides view his stop there as more nostalgic than game changing. Indeed, a senior McCain aide says the candidate and his advisers have gone back and forth in recent days about whether the stop was time well spent in the final hours of the campaign.


The best analogy I can come up with to explain the McCain campaign’s decision is to explain my own decision to draft Tom Brady of the Patriots in my fantasy league this season. The reason: Nostalgia. While the Patriots have become the Evil Empire of the NFL, I’ve had Brady on my team each of the past two seasons - both to my benefit as I performed admirably…nay, heroically with his help. As such, when he was an available late first round pick this year, against good logic, I took him because he’s been good for me. But this is no way to gamble - Brady went down and out for the season during the first game. It’s like betting money on the Vikings on the first game of the twice annual Vikings-Packer matchup, and even after they get trounced in the first game, going back to the trough and betting on them to win the second game - possible even going double or nothing out of state pride.

But it’s no way to run a Presidential campaign, and it leads me to believe that the McCain camp knows it’s over. This is for pride now. To re-visit the battlefield of bygone campaign victories, and be able to leave the field with your chin up. When McCain goes to New Hampshire, where he is 13 points down the day before the election, while Obama tries to run up the score in the West is as obvious a signal as they come.

For me, I’ll have the good fortune to be in Chicago on election night; gotta work my way down to Grant Park to catch a glimpse of history.

-Jeffrey Oak

Saturday, November 1, 2008

Studs Terkel - A Man For History - (1912-2008)

I have to use this forum to recognize an icon of contemporary American culture, literature, history, activism, etc., etc. Louis “Studs” Terkel died yesterday at the ripe age of 96.

Terkel will largely be remembered for writing contemporary accounts of historical events through the use of oral histories or interviews with actual participants. Such a style offers a refreshing contrast to historical interpretation offered by more academic or pop authors. Indeed, I was drawn to Terkel’s works by their ability to rip much of the romanticism and nostalgia from mainstream versions of historical events and recognizes events by how they affected, and were affected by, the many people that lived them. It also reduced the tendency to see individual participants as heros or otherwise to aggrandize the acts of an individual when reality, and legitimate historical analysis, begs for more context.

I first was introduced to Terkel via my grandfather’s bookshelf, where The Good War captured my interest in all things World War II. The Good War recognized something that historians had traditionally, and willingly, ignored in their approach to writing about history - that the further we remove ourselves from the time, events and people who make up the setting for historical events, the more we lose the ability to capture the human experience. And what is history, if not a recollection and retelling of the human experience?

More recently, pop historians have begun to emulate Terkel’s oral history style. Most notably Tom Brokaw’s The Greatest Generation essentially re-packaged The Good War, slapped a high profile name on the cover, and was published with the backing of a billion dollar publisher. The success of Brokaw’s Greatest Generation was buffeted by the fact that it came on the scene at a time when many of the subjects of the book were reaching their twilight years, and as many Baby Boomers sought a way to re-connect with dearly or soon-to-be departed parents. The same is essentially true of Ken Burn’s 2007 documentary The War.

Don’t get me wrong, I’m not trashing Brokaw or Burns here. In fact they confirm the importance of Terkel’s contribution - he legitimized of oral histories as a pedagogical method and established a mainstream art form that society craves.

Terkel’s personal story is also impressive for what it tells us about the values of our society and how they have changed in his lifetime. For example, in his Chicago Tribune obituary:

He attended the University of Chicago, where he obtained a law degree and borrowed his nickname from the character in the ” Studs Lonigan” trilogy by Chicago writer James T. Farrell. He never practiced law. Instead, he took a job in a federally sponsored statistical project with the Federal Emergency Rehabilitation Administration, one of President Franklin D. Roosevelt’s New Deal agencies. Then he found a spot in a writers project with the Works Progress Administration, writing plays and developing his acting skills.


WPA, New Deal, FDR - cornerstones of the foundation for LBJ’s Great Society and a fundamental change in the shape of government and society. Yet, imagine a politician of either party standing in Congress to support the creation of a “writers project” funded by a WPA-like institution today. The disdain with which many popular decision-makers have towards arts and academia is palpable. There are fewer and fewer avenues for a future Studs Terkel to develop her craft with the resources to support a family available to them.

Studs Terkel didn’t achieve literary respect and fame until much later in life. In fact, his first major work wasn’t published until he was 55.

The result was “Division Street: America,” published in 1967 to rave reviews and best-selling success. It told the stories, in their own words, of businessmen, prostitutes, Hispanics, blacks, ordinary working people who formed the unit of America and also the divisions in society, using Chicago’s Division Street as a prototype of America.


And only after that did Terkel get to the works that will be the hallmarks of his legacy:

It was a theme that Terkel would explore again and again, in “Hard Times,” his Depression-era memoir in 1970; in “Working,” his saga of the lives of ordinary working people in 1974; in “American Dreams; Lost and Found” in 1980; and “The Good War,” remembrances of World War II, published in 1985 and the winner of the Pulitzer Prize.


In the chronology of his life, it’s interesting to examine how he didn’t achieve his greatest personal and professional success until later in life. It may be debatable, and I hope I can participate in the debate, but Terkel’s work and contributions must be considered something akin to genius, in the same way other literary giants are credited for their works.

A recent New Yorker article by Malcolm Gladwell provides an interesting and analogous study of genius - Late Bloomers - and why some achieve greatness early in life, such as T.S. Eliot’s “Prufrock” at age 23, and some later in life, Robert Frost’s “Mending Wall” at 42.

What Gladwell concludes that those who reach success later in life:

his or her success is highly contingent on the efforts of others. … Late bloomers’ stories are invariably love stories, and this may be why we have such difficulty with them. We’d like to think that mundane matters like loyalty, steadfastness, and the willingness to keep writing checks to support what looks like failure have nothing to do with something as rarefied as genius. But sometimes genius is anything but rarefied; sometimes it’s just the thing that emerges after twenty years of working at your kitchen table.


Terkel certainly wasn’t toiling away at a kitchen table before he published at age 55, but his written works, the works that will have the greatest lasting impact were produced in the latter half of his life. But much like the history he wrote, his life success can be traced to the contributions and participation of others. The writing skills he developed in the federally subsidized New Deal programs, the interviewing skills he honed in thousands of interviews on his long running radio program, and what he learned from each of people he interviewed, and infinite unknown experiences, all contributed to his success. To this extent, Terkel’s approach to analyzing history - a history that is is complete only when it views events through the eyes of all the participants - is similar to how his own history would be written - it wouldn’t be complete without a review of all the perspectives.

Lets hope that Terkel’s death reminds us that much of the greatness of man cannot be attributed to rugged individualism, but a manifestation of the collective greatness that a committed society can produce.

-Jeffrey Oak