Document created: 31 December 03
Air University Review, January-February 1972
R. H. Blythe, Jr.
In an article published in the Air University Review for May-June 1971,
Major General Glenn A. Kent wrote regarding the future of analysis as a factor
in decision-making. His views deserve the careful attention of analysts and
decision-makers, since they reflect the thought of someone who has legitimate
claim to both titles. From his position of vantage,
Is this incipient decline coming about because decision-makers feel that analysis in general does not contribute to better decisions? Do they feel that experience and judgment unperturbed by the findings of analysis provide a better basis for decisions? In which case, analysis is unnecessary. Or is it a feeling that good analysis can be useful but the analytical work decision-makers have been receiving is not good? In the latter case, the cure is not to give up the analytical effort but rather to take steps to ensure its quality.
Uncertainty is what causes decision-making to be difficult. If facts were universally available, and if we all were gifted with the ability to foresee the future, we could all be chief executives. The uncertainties that plague decision-makers can be recognized as belonging to one of three general classes:
a. Uncertainty as to what our objectives are or should be.
b. Uncertainty as to what our opponent will do.
c. Uncertainty about the state of nature; i.e., what is fact and what is
fancy about the opportunities that physics, chemistry, and biology present to
Modern analysis procedures are especially directed toward decision-making in the face of these kinds of uncertainty; and I believe that reasonable decision-makers and reasonable analysts will agree that good analysis can be a useful adjunct to good decision.
Let us therefore abjure further attempts to fix the blame for past errors in decisions and instead consider the problem of how to increase the probability of good analysis being supplied to a receptive decision-maker.
In the preceding sentence I should like to have said, “How to ensure” rather than “how to increase the probability of” good analysis. But analysts and decision-makers are both fallible human beings whose performance and outputs are necessarily subject to some risk of failure.
General Kent has pointed out clearly the two common shortcomings of analysis:
He does not specifically use the word “incompetence.” Rather, he refers to several manifestations of incompetence, such as wrong objectives, inadequate models, and poor presentations, both written and oral. I have chosen to subsume all of these under the term “incompetence.”
What can be done to eliminate bias? If an analyst allows himself to be a party to presenting a biased analysis, there must be a reason. Most probably the reason is that he expects to enjoy some reward, or at least to avoid censure, by presenting an analysis that he expects will please someone who is in a position of authority over him. Such authority is established by the organizational structure and its rules. To avoid its being applied to frustrate an analyst’s objectivity, this authority should be assigned to someone who is not a decision-maker and presumably has nothing to gain one way or the other from the results of the analysis.
The product of the working analyst and the quality of his performance should be evaluated by an analyst higher in the organizational structure and not by someone who has a parochial interest in the outcome. Then, although the analyst’s professional output is still for the benefit of the decision-maker, it is not subject to his influence. The independence and objectivity of the analyst can be ensured by imposing rules that do not provide the decision-maker an opportunity to advance, retard, or otherwise affect the career of the analyst.
There remains the problem of how the highest-level analyst in an organization—the one who is ultimately responsible for the work of his unit—can himself be protected from the blandishments of “pleasing the boss” and transmitting this pressure, as through a conduit, to the working analyst. Organizationally, there is not much that can be done to help this person. It is of some value that he be given a sufficiently high grade, with commensurate prestige in the organization, that he can meet the ultimate decision-maker eye to eye and not cringe. The point of attachment of the analyst’s unit to the overall organizational structure suggests a possible effect on this problem, but this factor is of more apparent than real importance.
Probably of greater importance than any of the relationships indicated by organizational structure are the rules under which the analyst operates. The organizational rules may take the form of simple, unexpressed policy, or they may be prescribed in formal regulations. Informality is the more desirable, but formal rules may be necessary to promote objectivity and intellectual freedom and, indeed, to provide protection from capricious and unreceptive decision-makers. Such rules, whether written or merely understood, allow the analyst direct access to any decision-maker he deems to have need for the results of his analysis. He must be free to say whatever is necessary in presenting his analysis. To aid him in this, he may be permitted to operate under the disclaimer that his remarks, written or oral, do not necessarily reflect the official or authorized views of his commander. Even if called upon by an authority higher than his own command, he must be free to present results which may be at variance with his command’s official position, again under the provision of the disclaimer. The analyst cannot be totally unfettered, however; he must, for example, be restricted by any and all security regulations. Finally, it can be said with some confidence that in addition to the protection afforded by any of the provisions just discussed, the analyst will be most benefitted by a tough skin and a tender conscience.
Next, how can competence be achieved? The hiring and firing of the analysis staff should be left up to the senior analyst. Competent analysts are available; they can be attracted by a combination of adequate salary and the opportunity to work with other competent analysts. The importance of the second condition should not be underestimated. Let an analysis unit get a reputation for hiring incompetents and it becomes impossible to attract the competent.
Having so handily disposed of the analyst’s shortcomings, what can be said about the decision-maker? Again, the problems divide naturally into two general classes, which are counterparts of those described for analysts:
a. A tendency to establish a firm position prior to completion and
consideration of an analysis.
b. Lack of the training necessary to understand and particularly to assess the findings of an analysis.
The first of these is a major factor in tempting an analyst to reach a biased result. An unprotected analyst who is aware that a position has already been taken and who is also sensitive to the responsibility of feeding his family is sorely tempted to make his analysis support the pre-established position. This is an unfortunate solution for both parties. The analyst has suppressed or glossed over the information that could have led to a useful result, and the decision-maker has developed an unjustified sense of confidence in his untested judgment. But in the case of decisions by government, the biggest loser is the public itself, which must suffer for the bad performances of both.
In some cases it may be necessary or justifiable for the decision-maker to take a firm position without benefit of prior analysis. If the decision-maker then calls upon the analyst to conduct a study, there must be a clear understanding that the analyst is under no compulsion to produce a document which supports the decision. If analysis does not support the decision, then the decision-maker is free to seek other means of obtaining a supporting statement. (But he should avoid the plight of the drunk who must seek the lamppost for support rather than illumination.) In effect, the analyst should stand in the position of a certified public accountant. The CPA’s seal of approval is on the statement of financial accounting, but it does not necessarily apply to claims made by management in the body of the annual report to stockholders. Similarly, the seal of the analyst should go only on studies executed without reference to prior decisions. Generally, it would be helpful to identify those parts of a document which have been produced as a result of analysis work.
It is not my intention to decry the uses of advocacy. In fact my position is somewhat different from the generalization by General Kent. He states that “it is probably permissible, although somewhat dangerous, for analysts to be allowed to take a position.” He goes so far as to believe that “the analysis (the study itself) should not contain conclusions and recommendations.” I believe this dictum should not be universally applied, although there are instances in which the analyst could avoid trouble by heeding it. However, avoidance of trouble should not be the analyst’s main concern. Rather, he should be concerned that in making a decision the results of his analysis be given their proper weight. In some cases the contribution he can make will be very small because of important factors he could not take into account or because of lack of data that pressures of time prevented his assembling. On the other hand, when he believes that his analysis does adequately deal with the important factors, he should be a strong advocate of the position to which his analysis points.
If he refrains from advocacy on the grounds that it would be a violation of his scientific detachment, can he escape a share of the castigation his decision-maker will suffer if he adopts the wrong course because the analyst spoke too softly? Is not the analyst remiss if, after completing a competent analysis, he fails to persuade his decision-maker that the analytical results are worth adopting and supporting? Of course, the analyst could find some small solace in the post-decision lament, “See, boss, I told you so.” But this would be scant comfort to a sensitive and competent analyst who had just witnessed the man he is supposed to advise being sliced into small pieces by a better-prepared adversary, perhaps one whose position is intrinsically weaker than his own. Certainly, a physician should reproach himself if his patient suffers because the doctor failed to persuade him to follow the prescription necessary for a successful therapy.
What can be done to prevent or at least discourage decision-makers from taking positions prior to analysis from which subsequent retreat is embarrassing? And, related to this matter, what can be done to help the decision-maker in better comprehending the results of an analysis? Education and on-the-job evolution seem to be the only available remedies. The processes which a man goes through in making up his mind about some difficult problem cannot be prescribed by regulation, but decision-makers can be educated so that they can better understand the significance of an analysis. Perhaps it might even be possible to establish educational facilities that would enable staff officers to become familiar with at least the rudiments of modern decision-making theory. If a good course of study could be developed and enough officers exposed to it, then surely the overall quality of Air Force decision-making would tend to improve.
In this connection the analyst has the responsibility to speak and write clearly in terms that the decision-maker can understand. As decision-makers increase their formal training in analysis and as analysts strive harder to avoid the jargon of their trade, analysis will be more useful and decisions more likely to be effective.
No cure for the ills on the analyst’s side of the house has been suggested except in general terms. Hence a pragmatic solution is suggested:
My earlier remarks about the need for better comprehension of analyses by the decision-maker will not be formulated into a specific recommendation, as this is best left to someone from that side of the house.
The problem addressed here is how to make use of modern quantitative analytical procedures to increase the probability that decision-makers will make good decisions nearly always and best decisions as often as possible. General Kent, speaking from the viewpoint of a decision-maker, pointed out two classes of occupational diseases of analysts: bias and incompetence. In this follow-on opinion I recognize the existence of these ills and in fact believe that at times they have reached epidemic proportions. But I have pointed out that decision-makers are exposed to two parallel classes of illness: preconceived positions and inability to comprehend analytical procedures. I have suggested some pragmatic measures to curb the spread of the diseases which may affect the analyst, and I have offered some ideas of a more general type for helping the decision-maker. It seems reasonable to conclude that if these disease-prevention measures can be put into effect, the result will be better decisions.
Air Force Base,
Richard H. Blythe, Jr., is Director of Analysis, North American Air
Defense Command. He has served with the Operations Analysis units since 1951
and as Director since 1957. Prior to joining the Air Force, he worked with the
U.S. Forest Service and the Federal Trade Commission. He is a graduate of
The conclusions and opinions expressed in this
document are those of the author cultivated in the freedom of expression,
academic environment of
Home Page | Feedback? Email the Editor