Relevant and even prescient commentary on news, politics and the economy.

You Can Fool Most of the People Most of the Time

At least on certain issues.

I’m not inherently a great pessimist, but with few exceptions each passing month for over a decade now has seen my optimism whither, at least a little.  So I can’t help but see the manure-colored lining in this otherwise rosy, fluffy cloud.

Steve Benen reports that according to the new NBC/WSJ poll, Americans trust Democrats more than Republicans on domestic issues, sometimes by large margins.  Here is a graph.  (As always, click to embiggen.)

Graph 1   Who Do You Trust?

But the causes of my pessimism are four-fold.  First, as Benen goes on to note, the same polling reveals that in the popular mind “Republicans have an advantage on the (sic) reducing the deficit, ‘controlling’ government spending, and national defense.”  Well, there’s three reasons for pessimism right there.  A) Reducing the deficit is an issue of exactly zero urgency, and attacking it now will certainly cause economic hardship, especially for those at the bottom. Further, Republicans have been huge debt increasers for decades, while Dems have not.  B) We absolutely do not have a spending problem.  We absolutely do have a revenue problem, as graph 2 plainly indicates.  I think the Republicans have become convinced of their own lies.

Graph 2   Federal Gov Current Recpts by GDP

C) From FDR through LBJ to BHO, Dems have been every bit as war-mongerish as their Rep counterparts; BHO has continued his predecessors war initiatives almost seamlessly;  and 9/11 happened on W’s watch.  This just makes me want to cry.

But I have a bigger list.  Second, a look a graph 1 reveals some disturbing details.  A)  Joe BeerCan must not connect “Looking out for the middle class,” Medicare,” “Health Care,” “Medicare,’ or “Social Security” with “Economy” or the results for those categories would line up better.  B) Considering Paul Ryan and the never-ending series of Republican contrived cliffs, scoring Dems only marginally better than Repubs on the economy is, all by itself, cause for despair.  C) As is the close call on taxes.

Third, and I’ve already alluded to this, there is almost no daylight between the two parties on foreign policy issues.  Still I have to give a slight nod to the Dems, based on practicality, because: John Bolton.

And last, though I firmly believe to the bottom of my heart that the Dems are superior on absolutely every issue, problem and question that might rise, they still aren’t that damned good.  Case in point: the new head of the Michigan Democratic party is a venture capitalist.  As Bill Maher sagely put it, while the Democrats have moved to the right, the Republicans have moved to the insane asylum.  They demonstrate this anew, almost every single day

The lessons of history and even a casual observation of the current failures of European austerity show that progressive policies are the clear and present necessity.  But even if we had strong Dem majorities, we still have Reaganite B. Hoover Obama in the White House, and a genuine progressive movement in congress the exact size and shape of Bernie Sanders. 

As one of my college professors put it long ago:  Booze is the only answer.

Polling Obsession

The 2012 Presidential is so close just so close that not only do different polls show different results but different averages of polls do.

OK it is clear that extremely fancy models such as the fivethirtyeight model which attempt to assign undecideds and use state data to estimate the national vote and vice versa are different (generally I don’t like fancy but Nate Silver has a track record).  But simple smoothers also give different results.

Partly it is the choice of polls to include yes/no on web based or partisan polls. is a simple average but they don’t include partisan polls or web based polls (and use self reported partisanship so Rasmussen is included but not PPP ooops).

But part of it is the smoothing algorithm.  both and use loess smoothers which fit a constant and a coefficient on time using weights which decline away from time t and report the fitted value for t.  I think this means they are too quick to extrapolate trends.  This means that one outlier can convince the computer that the polls have been trending for a candidate.

My example the odd case of TalkingPointsMemo and Michigan.  The program has Ohio leans Obama and Michigan a tossup.  This is due to one poll by Foster-McCallum (in serious contention for the bitterly fought prize for worst pollster).  Without them, the computer estimates that Obama is 3.9% ahead.  That is enough for TPM to call the state leans Obama and give more than 269 electoral votes in Obama leaning states.

Here is the graph with Foster-McCallum

Here is the graph without them

This is extreme, because Foster/McCallum had a Florida poll with an absurd sample which earned the coveted and very rare double asterix from TPM for not included in the average because of an editorial decision.

I think such decisions should be made pollster by pollster not poll by poll.  But totally aside from that a sensible smoothing algorithm shouldn’t be so sensitive to one poll when 10 October polls are available all of which polled after the first TV debate.

Notice how the poll pulls down estimates of the state of the race before it’s sample began.  This feature of the smoother makes it hard to see if a shift is due to an event.

I say even simpler is better than Loess.

Standard silly rounding for polls, 4th grade style fail

Lifted from Angry Bear Robert Waldmann’s Stochastic Thoughts:

All aggregators are doing funny things by rounding first then subtracting.  This is a comment on Singiser, but it could be on any aggregator.
In the Gallup Tracker likely voter subsample, Romney is up 1.2% (this link points to numbers which will change)  If one must round one should round to 1 not 2.

The standard report rounds Romney’s lead up to 2 % .
“NATIONAL (Gallup Tracking): Romney 49, Obama 47 (LV); “
In the IBD/Tipp tracker likely voter sample   Obama is up 0.7.  If one must round one should round to 1 not zero.

IBD/TIPP 2012 Presidential Election
Daily Tracking Poll

Day 4: Oct. 12, 2012
Obama: +0.7Obama 46.4% | Romney 45.7%
Read More At IBD:
The standard report rounds Obama’s lead down to 0%
“NATIONAL (IBD/TIPP Tracking): Obama 46, Romney 46”
The two average Romney up 0.25% not 1%
I often read “percents don’t add to 100 because of rounding”.  But somehow this is not OK when subtracting rather than rounding. 

update: Singiser notes that (with the standard silly rounding) Romney’s national poll lead for the day declined from 1.3% to 1%.  He notes that this is totally statistically insignificant and not worth wasting thousands of pixels to criticize as I do below.
It matters to people silly enough to try to back out daily results from trackers.  To be maximally silly I will use the rounded numbers released yesterday (actually day before yesterday here in Rome).
Gallup 10/11 minus Gallup 10/5 (silly rounded) comes up as a shocking Romney up by 1.4% (shockng as 10/5 should have been a great day for Romney) not an even more shocking 7%.
IBD/TIPP 10/11 minus IBS/TIPP 10/6  (silly rounded) comes out as Obama up 10.2  not 6.
The estimate for a normal day vs the two dread days after the debate is 4.4%  with half silly rounding not -0.5% with all silly rounding.  With silly rounding (both days for both pollsters) it looks as if the debate effect was permanent not a bounce. With silly rounding for the second latest day and not the latest day, it looks as if the debate effect was temporary.  This is all silly, because it is torturing the poor numbers to try to make them say something they don’t know, but the point estimates are not small.

Why can’t the Washington Post report on the Washington Post’s poll ?

The Washington Post had two articles on their latest poll. The first focused on the Presidential horse race. The second noted that US adults disapprove of congress and then went on to discuss other results. At the very end of the the second article this follows:

Obama’s proposal to raise taxes on millionaires to help close the deficit enjoys wide public support — three-quarters of adults, including majorities of independents, moderates, conservatives and Republicans, back it.

Among the few groups that don’t favor such tax increases are Republicans who strongly support the tea party movement; they oppose the proposal by more than two to one.

This isn’t news to anyone who pays attention to polls anymore, but it is more newsworthy than the observation that most US adults have noticed that Congress is not functioning. Importantly, opinion leaders don’t pay any attention to polls even when discussing public opinion.

It is widely argued that Obama has decided to fire up the base with populist proposals which will increase turnout of Democrats and liberals but reduce his support among moderates and independents. In fact, his populist proposal is supported by a majority of self identified Republicans and conservatives. Obama is moving towards the center of public opinion. He is also firing up the base*.

Also and less importantly:

Paul Kane’s and Scott Clement’s understanding of statistics is essentially as low as possible.

They wrote

Only 3 percent of Americans said they “strongly approve” of the performance of lawmakers on Capitol Hill — essentially as low as possible, given the poll’s margin of error of four percentage points.

That is they said that mathematical statistics proves that we can’t agree on anything. They definitely asserted that it is “essentially” impossible for 100% to agree on somethnig.

The problem is that pollsters have reported nonsense standard errors for so long that journalists have been convinced of something absurd.

In fact, the variance of the mean of a sample from a binomial distribution depends on the true probability — in this case the fraction of the population which strongly approve of the performance of Congress. To be modest, pollsters always present the highest possible standard error corresponding to an evenly divided population. To be honest, I think they report the largest plausible standard errors due to sampling alone to hide the fact that poll responses deviate from actual voting for reasons other than sampling error.

In any case the standard error corresponding to 3% is 100% times the square root of (0.03*0.97/(smple size) or roughly 0.55%. The convention is to report a number plus or minus 2 standard errors so 3% plus or minus 1.1%. This would be a 95% interval if the distribution were normal. Using the normal approximation, one can reject the null that the true fraction of strong approvers of Congress is zero at the 95% level.

Of course if one has any sense at all, one rejects that nul at the 100% level not the 5% level, since some people said they strongly approve of Congress. The normal approximation works very well even for fairly small samples so long as the true probability is close to 0.5. Obviously it doesn’t work whenever it gives an x% level which includes the hypothesis that no one in the population would say something which someone in the sample said.

But that is an advanced topic.

Next topic English. An obviously false statement is not made true by adding the qualifier “essentially.” The fact is that some US adults strongly approve of our Congress. This is appalling, but they really exist. The word “essentially” was used to assert that this mere fact is negligible. This contempt for mere facticity reminds me of Hegel (them’s fighting words where I come from).

Hegel did have a point. A historical movement can turn into its opposite. So the theory of statistics has become a way for some people to dismiss inconvenient data as “essentially” non-existent.

* More on taxes (my usual rant)

*I have long complained that the MSM failed to report the strong majority support for a more progressive tax code (revealed by dozens of polls on the question dating back to the early 90s). They are now reporting it — at the very end of articles. I think that under reporting of the strength of left populism is a systematic error partly caused by the self interest of opinion leaders.

Even now, many people (including the usually reasonable Charlie Cook) consider a proposal with majority support among conservatives the result of a decision to give up on winning the support of moderates. This is crazy.

I think part of what is going on is that people naturally approximate the infinite dimensions of opinion with one dimension left vs right. If you average over issues and people, the average US adult is to the right of Obama (he believes in evolution and global warming and is skeptical about the death penalty. It bothers him when he violates due process right. Etc). Even two dimensions are enough to find one where Obama is to the right of the average American. The average American is eager to soak the rich. Obama prefers soaking the rich to the alternative, but he’d really rather reform health care and regulate greenhouse gas emissions.

Another part of what is going on is that political reporters lose touch with the USA outside the beltway. In official Washington raising taxes on the rich above Clinton era levels was (until yesterday) an extreme left wing position. In the general population it is a centrist position. On this issue, the median voter is far to the left of the median congressman,the median political operative, and the median pundit.

Finally there is the “opinions on optimal re-election strategy differ — both sides have a point” problem. There are people who call themselves Democratic strategists who argue that running on a soak the rich platform is bad strategy (Penn-Schoen no not the PR firm the … yes the PR firm and the P stands for politicians). This approach would clearly cause increased turnout of liberals and Democrats. If one assumes they must have a point, one must assume that it will cause moderates and independents to vote Republican or make Republcians more likely to vote. Therefore one must conclude that moderates and independents will vote against a proposal supported by a majority of moderates and independents or that Republicans and conservatives will be outraged by a proposal supported by a majority of Republicans and conservatives. This is crazy. But there is the methodological a priori that both sides must have a point, so it must be true.