A Complete Set of Subjective Probabilities, Bayes’ Formula, Overconfidence and Over-reaction to new information

There is an approach to cognitive psychology and modelling beliefs and forecasts which is natural for economists and very misleading. The assumptions are that we have a complete set of subjective probabilities updated with Bayes’ formula, that  for any possible combination of events we have a belief about the probability that they will occur and that these probabilities are updated with Bayes’ formula so we divide the probability of a series of  events by the probability that those which we have observed occur. About a century ago, Keynes argued that this model is completely unrealistic in “A Treatise on Probability”.  In fact a moment of introspection should convince you that you do not have such a complete set of subjective probabilities. When it is attempted to elicit subjective probabilities, we feel forced to make a difficult choice. The elicited probabilities of different series of events are not.  Most of all Bayes’ formula is, to us, black magic. The simplest applications are difficult to impossible, and the implications are implausible.

The main appeal of the assumption of a complete set of subjective probabilities is that it is at least less absurd than the still conventional assumption of rational expectations in which the subjective probabilities are objectively accurate probabilities.

However, the assumption of a complete set of subjective probabilities updated with Bayes’ formula is also implausible and has inaccurate implications.

One is the implication of subjective overconfidence for reactions to new information.   If people are asked for a 90% interval, they give a 50% interval. If their beliefs are subjective probabilities (which are accurately elicited and which obey the laws of probability) this implies a subjective probability distribution which is more concentrated than the objective probability distribution (for example having lower variance).  That implies under-reaction to new information.

But the data show over-reaction to new information. Most clearly with excessive adjustments of predictions, so the best predictions is a weighted average of the latest prediction and earlier predictions.

It is possible to tell a story about beliefs and adjustments to new information provided on doesn’t assume a complete set of subjective probabilities. People have the thought that something is unlikely. If it occurs they are surprised.  They can choose between believing that they were completely wrong or believing that the world has changed so the old rules don’t apply. The second is less embarrassing. If the world has changed to the old rules don’t apply, then old data are not useful, so one should use only new data. This causes and over reaction to new data.

This argument raises an important concern.  If one is allowed to tell any story one pleases after seeing the data, then there is too little intellectual discipline, so one can fit any pattern yet make terrible forecasts. This is an unconvincing argument for assuming rational expectations. It is a more nearly convincing argument for assuming we have a complete set of subjective probabilities and use Bayes’ formula.  A solution is to use assumptions about psychology only if they are well supported by empirical data collected in experiments which support the assumption general rules which are not just identical to the behavior we are trying to explain (that is which imply valid forecasts and not just the ability to fit available data).  This remains an ambitious goal.

Fortunately my story has a testable implication. Sadly, I have a very high subjective probability that my hypothesis will be rejected by the data.  The story gives a high average adjustment of expectations to new data achieved by occasional very very high adjustments when we are convinced that the world has changed so old rules don’t apply.  This means that, most of the time, the adjustment will be too low.  This implies that the median adjustment will be too low.  This also implies that if forecasts are elicited from a set of people, the medial adjustment will be too low. I am quite confident that the prediction based on my hypothesis will be rejected by the data. Confident enough that I wouldn’t dare that resources be expended on an experiment.  Confident enough, that I wouldn’t even dare that available raw data be re-analyzed to test the prediction.  So I enjoyed typing it, but don’t think my thoughts are very useful.