Last week two British economists sent a letter to Queen Elizabeth in which they explained their failure to foretell the current financial crisis as a ‘failure of imagination.’ The same phrase has been used before, noticeably with people trying to explain the failure of various intelligence agencies in the US to foretell the attacks on September 11th, 2001.
It is a catchy phrase: ‘a failure of imagination.’ As if something mystical and creative simply failed to happen. Along the lines of ‘my Muse abandoned me,’ or ‘the Fates conspired against me.’ There is another name for it, one that is a more accurate descriptor: poor analysis. And an accompanying phrase should be associated with it: ‘poor leadership.’
Analysis is the process of using various specific processes to construct a logical, defensible argument to reach a conclusion. Analytics have certain key elements; in broad terms they are: facts, assumptions, an analytic approach or model, and a conclusion. Each of these requires honesty and intellectual rigor. There is a place for imagination, but it comes after you have completed all the appropriate analytic steps.
Whether you are engaged in intelligence or economic modeling or stock-market forecasts or designing a new wing for an aircraft or a host of other possibilities, predicting what is going to happen requires that you do several things.
First, you must use all the appropriate means of analysis available to, as best as possible, model what might happen. One set of answers isn’t enough, no matter how convenient they are.
Second, remember the limits of any particular model. As anyone who has studied analysis at even the most basic level remembers, every analytic model has strengths and weaknesses; each analytic model was developed for application to a certain set of circumstances and should not be applied to certain types of problems. Don’t try to make ‘the foot fit the slipper’ if it doesn’t fit.
Third, understand your assumptions and recognize that as your assumptions change, so do your answers. Understand all your assumptions; most analytic cases involve not only several explicit assumptions, but also a host of implicit assumptions. For major problems it is important to list your implicit assumptions and then identify which ones are based on hubris. Various government economists around the world are guilty of failing to do this vis-à-vis the current economic crisis, confident both that they had accurately accounted for all the possible permutations and that the various government ‘safety valves’ were adequate for coping with any perturbations. They were wrong.
Fourth, check your facts. Do you have all the relevant facts? Where you can’t get all the relevant facts are you adjusting your assumptions accordingly? When you get new facts do you re-run your analysis?
Fifth, regularly review your findings – with a clean slate. Are your assumptions still valid? Are there newly obtained facts that need to be included in the analysis? Is your model still correct? Is there a better model? This is where peer review really comes into play. Do others get the same answers you do? If so, are you engaged in Group Think? If not, why not? You need to answer both questions.
As for the role of imagination… Are bubbles in the economic sector new? No. In fact, they are routine and the subject of literally hundreds of books. Every student of economics has read about a wide range of economic bubbles where, due to a host of reasons, the price of a particular item or commodity rose far above what the market could bear. The story of the tulip bulb market of the early 17th century is a favorite example in economic history. Bubbles, and other such events are not new, just as the attack on the World Trade Center was not new in 2001. Not only had it been attacked before (1993), attacking it had been discussed in at least one book as early as 1979 (Terrorism: Threat, Reality, Response by Kupperman and Trent). The idea of crashing aircraft into things was not new either, beginning with the Kamikaze attacks of World War II, and including the 1994 Air France hijacking by Algerian terrorists in which the hijackers intended to crash the aircraft into several ground targets. Tom Clancy’s ‘Debt of Honor’ (written in 1994) includes a terrorist/Japanese nationalist crashing an airliner into the Capitol during the President’s address to Congress.
Counting on the ‘imagination’ of the analyst to think up new ways things might happen is as dangerous as counting on the Fates or a Muse. The fact is that there are wide ranges of historical examples that will usually suffice to point the analyst in new directions of inquiry if the analyst is rigorous. Sound analysis should involve not only the construction of complete analytic arguments based upon well defined facts and models, but also upon expanded assumptions, assumptions that can be crafted by working backwards from worst case scenarios.
This leads to a simple but profound change in perspective: one the one hand, the process that led the US into the economic situation we find ourselves in, analysis is weighed against risk. Leadership then weighed the benefits of taking the risk against the probability of something going wrong. Beginning under the Carter administration rules were changed to increase the availability of housing loans to those who otherwise would not qualify. The idea was that the US Government would insure the loans and, given the level of government oversight there was a low risk of things going horribly wrong. The Reagan administration took the position that the government oversight was not as capable as others were suggesting and the result of things going ‘bad’ was not worth the risk, and accordingly tried to tighten up on the government backed programs. This unraveled in the 1990’s and the ‘race’ to a bubble was on.
In simple terms, those in the government confused the probability of an event with the consequences of the event. If the members of the Carter administration had focused on consequences, and the possible consequences of the concept of government backed loans to people who would otherwise not qualify had been well discussed, it is possible that they would have realized that the long-term implications were in no way worth the risk and would never have started us down this road. Furthermore, sound analysis at such institutions as the Federal Reserve Board and the Congressional Budget Office should have, over the years, revisited these decisions and pointed out the consequences if the system faced a burst bubble. That they failed to make this case is an indictment against both the leadership and the analysts.
These kinds of events are not simply failures in analysis, they are failures in leadership. Analysts will only produce sound analysis if held to rigorous standards. When the leadership becomes sloppy or worse, partial to an answer irrespective of the analysis, the analysts will eventually lose their ‘edge.’ It is incumbent upon the leaders of any organization that is charged with trying to ‘see’ into the future to demand the maximum rigor in such analysis. The leadership needs to take the time to understand not only the strengths and weaknesses of the various analytic approaches, but the strengths and weaknesses of each element of the analysis, to include the analysts themselves. It is incumbent on the leadership to both insist on analytic excellence AND provide the means to ensure that excellence. That is not something left to the Fates or the Muses or ‘Imagination.’
It is a catchy phrase: ‘a failure of imagination.’ As if something mystical and creative simply failed to happen. Along the lines of ‘my Muse abandoned me,’ or ‘the Fates conspired against me.’ There is another name for it, one that is a more accurate descriptor: poor analysis. And an accompanying phrase should be associated with it: ‘poor leadership.’
Analysis is the process of using various specific processes to construct a logical, defensible argument to reach a conclusion. Analytics have certain key elements; in broad terms they are: facts, assumptions, an analytic approach or model, and a conclusion. Each of these requires honesty and intellectual rigor. There is a place for imagination, but it comes after you have completed all the appropriate analytic steps.
Whether you are engaged in intelligence or economic modeling or stock-market forecasts or designing a new wing for an aircraft or a host of other possibilities, predicting what is going to happen requires that you do several things.
First, you must use all the appropriate means of analysis available to, as best as possible, model what might happen. One set of answers isn’t enough, no matter how convenient they are.
Second, remember the limits of any particular model. As anyone who has studied analysis at even the most basic level remembers, every analytic model has strengths and weaknesses; each analytic model was developed for application to a certain set of circumstances and should not be applied to certain types of problems. Don’t try to make ‘the foot fit the slipper’ if it doesn’t fit.
Third, understand your assumptions and recognize that as your assumptions change, so do your answers. Understand all your assumptions; most analytic cases involve not only several explicit assumptions, but also a host of implicit assumptions. For major problems it is important to list your implicit assumptions and then identify which ones are based on hubris. Various government economists around the world are guilty of failing to do this vis-à-vis the current economic crisis, confident both that they had accurately accounted for all the possible permutations and that the various government ‘safety valves’ were adequate for coping with any perturbations. They were wrong.
Fourth, check your facts. Do you have all the relevant facts? Where you can’t get all the relevant facts are you adjusting your assumptions accordingly? When you get new facts do you re-run your analysis?
Fifth, regularly review your findings – with a clean slate. Are your assumptions still valid? Are there newly obtained facts that need to be included in the analysis? Is your model still correct? Is there a better model? This is where peer review really comes into play. Do others get the same answers you do? If so, are you engaged in Group Think? If not, why not? You need to answer both questions.
As for the role of imagination… Are bubbles in the economic sector new? No. In fact, they are routine and the subject of literally hundreds of books. Every student of economics has read about a wide range of economic bubbles where, due to a host of reasons, the price of a particular item or commodity rose far above what the market could bear. The story of the tulip bulb market of the early 17th century is a favorite example in economic history. Bubbles, and other such events are not new, just as the attack on the World Trade Center was not new in 2001. Not only had it been attacked before (1993), attacking it had been discussed in at least one book as early as 1979 (Terrorism: Threat, Reality, Response by Kupperman and Trent). The idea of crashing aircraft into things was not new either, beginning with the Kamikaze attacks of World War II, and including the 1994 Air France hijacking by Algerian terrorists in which the hijackers intended to crash the aircraft into several ground targets. Tom Clancy’s ‘Debt of Honor’ (written in 1994) includes a terrorist/Japanese nationalist crashing an airliner into the Capitol during the President’s address to Congress.
Counting on the ‘imagination’ of the analyst to think up new ways things might happen is as dangerous as counting on the Fates or a Muse. The fact is that there are wide ranges of historical examples that will usually suffice to point the analyst in new directions of inquiry if the analyst is rigorous. Sound analysis should involve not only the construction of complete analytic arguments based upon well defined facts and models, but also upon expanded assumptions, assumptions that can be crafted by working backwards from worst case scenarios.
This leads to a simple but profound change in perspective: one the one hand, the process that led the US into the economic situation we find ourselves in, analysis is weighed against risk. Leadership then weighed the benefits of taking the risk against the probability of something going wrong. Beginning under the Carter administration rules were changed to increase the availability of housing loans to those who otherwise would not qualify. The idea was that the US Government would insure the loans and, given the level of government oversight there was a low risk of things going horribly wrong. The Reagan administration took the position that the government oversight was not as capable as others were suggesting and the result of things going ‘bad’ was not worth the risk, and accordingly tried to tighten up on the government backed programs. This unraveled in the 1990’s and the ‘race’ to a bubble was on.
In simple terms, those in the government confused the probability of an event with the consequences of the event. If the members of the Carter administration had focused on consequences, and the possible consequences of the concept of government backed loans to people who would otherwise not qualify had been well discussed, it is possible that they would have realized that the long-term implications were in no way worth the risk and would never have started us down this road. Furthermore, sound analysis at such institutions as the Federal Reserve Board and the Congressional Budget Office should have, over the years, revisited these decisions and pointed out the consequences if the system faced a burst bubble. That they failed to make this case is an indictment against both the leadership and the analysts.
These kinds of events are not simply failures in analysis, they are failures in leadership. Analysts will only produce sound analysis if held to rigorous standards. When the leadership becomes sloppy or worse, partial to an answer irrespective of the analysis, the analysts will eventually lose their ‘edge.’ It is incumbent upon the leaders of any organization that is charged with trying to ‘see’ into the future to demand the maximum rigor in such analysis. The leadership needs to take the time to understand not only the strengths and weaknesses of the various analytic approaches, but the strengths and weaknesses of each element of the analysis, to include the analysts themselves. It is incumbent on the leadership to both insist on analytic excellence AND provide the means to ensure that excellence. That is not something left to the Fates or the Muses or ‘Imagination.’
No comments:
Post a Comment