Document created: 2 September 03
Air University Review, May-June 1975

The private sector is not the only place where money is ^"tight." In the federal government, the Congress is faced with determining how to allocate funds among a multitude of agencies and departments, each of which represents some form of public need. Our Congressmen more and more resemble a board of directors forced by corporate shareholders to demand an efficient operation and increased return on investment from management. This demand, in the federal system, is passed on principally to the executive branch, where most of the public need is transformed into expenditures. During the past decade, a special target has been the Department of Defense. 

The public--and therefore Congress--seems to accept massive expenditures for defense when the need is obvious, that is, when the nation is threatened and the threat is recognized. When the need is no longer urgent in the eyes of the bill-payer, however, there is an understandable clamor for reductions. The need may still exist, but other requirements appear, at least, to have greater priority. It is this reordering of priorities that has put DOD on the defensive and generated the climate in which we now find ourselves. Briefly put, military managers must find more efficient ways to reallocate and use what resources they are provided.

Military preparedness programs are being viewed now more than ever before as an integral part of the broader concept of national security. As a result, the expected benefits from specific military programs are being compared directly with the expected benefits from non-military programs for purposes of making resource allocations.¹

The practical advantages are clear. Finding the least costly or most beneficial alternative, when viable alternatives are available, can assure the purseholders that management is truly attempting to get most from each dollar. Even when the choice is not the least costly, simply having considered all the costs benefits of each choice is comforting to both the decision-maker and those who provide the wherewithal. One practical advantage, then, is an increased capability to justify a selected proposal or at least maintain the current position (which may very well be one alternative). All other advantages are derived from this kind of capability. The decision-maker is reasonably assured that he has been given a chance to review all the feasible ways to get the job done, that all relevant costs of each route have been evaluated, and that the pluses and minuses involved in each case have been considered.

The essential thing is the comparison of all the relevant alternatives from the point of view of the objectives each can accomplish and the cost which it involves; and the selection of the best (or a "good") alternative through the use of appropriate economic criteria.²

Economic analysis is simply the process of formulating a basic structure or methodology for a systematic evaluation of problems of choice. Given an objective, an economic analyst identifies alternative ways to reach the objective and then determines the costs and benefits of each alternative. The analysis product should provide management with an orderly and comprehensive presentation of the essential elements of each alternative and thereby assist the decision-maker. In reaching that point, economic analysis structures informal thinking and hopefully avoids unfounded "gut" decisions, surfaces hidden assumptions and their implications, and provides a convenient and effective means of communicating the considerations behind a recommendation.³

Program evaluation is essentially the same process except that it addresses an existing program to determine whether a change is appropriate. In this article, whenever "economic analysis" is used, the reader should assume that program evaluation is included, as an extension of the basic technique.

This kind of capability surely is worth having at any level in military management or in the private sector. DOD is committed to it, and the Air Force is pushing for service-wide application. In the field, at major command level and below, however, use of economic analysis so far is spotty. This writer believes the opportunities abound and that an effective economic analysis program truly is essential at major command level. Once established there, applications at lower levels will surface or increase in proportion to the number of managers who have seen economic analysis work, realize that it is not complicated nor simply a statistician's tool, and demand its use.

It cannot be stated too frequently or emphasized enough that economic choice is a way of looking at problems and does not necessarily depend upon the use of any analytic aids or computational devices.⁴

Surprisingly, in view of the aura of sophistication surrounding the term, economic analysis does not need to be complicated to be effective. A viable capability requires only a few basics: a problem (a clearly defined need), a source of standard or generally acceptable cost factors, maybe a calculator, and a format or procedure to follow.

It is true that economic analysis involves a variety of statistical techniques. They contribute to the basic capability and include the tools that generate the cost and planning factors used in quantifying each alternative, provide a uniform basis for comparison, or test the validity of the analysis. But these are the technician's tools and no more essential to understanding and using economic analysis than being able to build a computer is necessary to making effective use of its capability.

The uninitiated might reasonably assume a viable economic analysis program would demand an economist, a mathematician or statistician, maybe an operations analyst. Their disciplines would contribute much to the technical end of economic analysis. Of course, such expertise may be necessary at an advanced stage. What is really needed, however, is someone with initiative and imagination. His or her facility with statistics need be no more than what is required to handle college algebra. An economist, mathematician, or statistician without a healthy imagination is generally much less desirable than an English major who can apply relatively simple algebraic equations, provided he or she can approach a problem of choice in an orderly, logical manner.

Along with initiative and imagination, the ideal economic analyst must be able to work effectively with people. In generating inputs to an economic analysis, as in any problem-solving situation, the analyst must rely on the cooperation and contributions of others.

As for the technical aspect, eventually a graduate mathematician or statistician will be useful as improved and unique cost and planning factors and more detailed or complicated analyses become necessary. Personnel well versed in the command mission and its technology are also essential. Since some continuity will always be valuable, civilian employees are possibly the best source of both kinds of expertise.

A well-rounded capability, however, requires occasional disruption: new ideas and the flexibility to go with them. This by no means implies that experienced civilian analysts are unlikely to generate new approaches and new applications, but it does reflect this writer's conviction that a change in faces provides ideas. That alone is worth any temporary loss in momentum. It is occasionally even worth reinventing the wheel. The obvious way to bring in new personnel now and then is to have some military authorizations. And there is another way: perhaps idealistic but not necessarily counterproductive is a program for career progression wherein a productive and aggressive civilian employee makes his or her contribution and moves on to a promotion elsewhere.

Recognition of accomplishments is another key factor--not simply acknowledgment of a job well done but the kind of recognition that motivates doing the very best within one. It means putting the individual's reputation on the line, putting his or her name on the analysis, trusting him or her to carry the results of an effort up through the chain of command. Let the analyst brief the results, defend the findings, answer the questions, and reap the benefits of exposure. At the same time, his or her supervisor must he prepared to support the analyst and accept criticism of what is surely more than an individual effort.

The importance of an effective and imaginative staff in a program for economic analysis and program evaluation is understandable. No less important is the environment in which these people work. This is, true, of course, whether we are discussing the analyst or those who participate in other ways.

Environment. To be most productive and imaginative, people must operate in an atmosphere that is conductive to originality. Essential to creating this atmosphere is the encouragement of open discussion with all concerned. This is an atmosphere wherein the boss truly wants to know when he or she is wrong and the staff feels free to say so. It also is an atmosphere in which mistakes are made and accepted as a natural part of the analytical process. It is most certainly a productive and dynamic environment, a place where past successes are not enough and the status quo is almost always questioned.

The prerequisite for this kind of climate is a healthy share of self-confidence on the part of the person in charge of the program as well as others' confidence in the capability and professionalism of the analyst. The end result can be a genuinely effective group of analysts who in turn impart their enthusiasm to the "customers" with whom they work.

Training. Assuming the embryonic economic analysis function eventually will need additional statistical capability, training requirements must be identified. Even if the staff is well qualified academically, the short courses available at the Air Force Institute of Technology or in the Air Training Command will aid immeasurably in orienting individuals toward Air Force problems. Next best, in order of preference, are courses offered for the same reasons by our sister services, by the Civil Service Commission, and by civilian colleges. Intermediate and advanced courses should be held in abeyance until there is some evidence that the basic talents have been put to good use.

An excellent list of training courses is provided conveniently in a Directory of Training, Films, Publications, and Models on Defense Economic Analysis and Program Evaluation, compiled in 1974 by the Defense Economic Analysis Council.⁵

Formal training in economic analysis and other analytical techniques will prepare the analyst, and materials such as those cited in the notes at the end of this article will provide background and a lot of buzz words for management. Most formal training and published materials, however, really only reach those who must know or sincerely want to know the subject. What about those persons in any organization who harbor an almost inborn distrust of "statistics" and "analysis"? These are the functional technicians and managers who must be converted if economic analysis is to be accepted routinely as a valuable tool. They also are the people whom at least one major command is teaching to use economic analysis methodology in working problems of choice. The process of application, at "working group" level, will be described briefly later; but how these technicians and middle managers come to know economic analysis is appropriate at this point.

The Air Force Communications Service (AFCS) economic analysis program includes a very practical workshop series that puts the theory, principles, and application of economic analysis into "real world" terms, specifically, communications problems. The workshops are conducted at command headquarters and in the field at numbered air force level by a team of one or two analysts and sometimes a representative from another staff activity. The team attempts to sell economic analysis--so far very successful in AFCS--in introductory briefings to the senior staff, to get its support as the two-day workshop begins. In the workshop, theory and principles are covered in less than a morning. The true learning process begins with an economic analysis of a problem taken from the command files. The problem always involves something for the planners, engineers, programmers, logisticians, and representatives of manpower and personnel, who make up the majority of the attendees, for their expertise is essential to economic analysis in AFCS. The prob1em is real, and the students relate to it and interact accordingly. Economic analysis is no longer something "they" do; "they" (the statisticians or analysts) just help with the stubby-pencil work.⁶

Equipment. Initially, all that is needed is standard office furnishings and a calculator for each analyst. Electronic calculators are speedy and quiet and therefore desirable. Office calculators should be equipped with a paper tape readout and be capable of performing most statistical functions. A small, battery-operated calculator for use on the road and in meetings should be available for at least part of the staff.

When the economic analysis function is well under way, a remote computer terminal with access to the command data bank should be considered. The major advantages are savings in analyst time and an increased capability for generating unique outputs, using standard software or programs developed in-house. This is a major step and naturally will require either additional training for someone or hiring someone with the needed computer expertise.

Cost Data. Cost and planning factors can be and are provided in tables and other sources, usually by higher headquarters. At lower levels of command, for example at major command level, there may be a need for unique factors. If the expertise exists, such inputs can be developed by the using organization, or they may be developed for the user. At the beginning of a command program, however, most if not all of the factors needed in the crawling stage are usually available. This depends, of course, upon the complexity of the proposed investment or existing program being evaluated. Common sense suggests it is only practical to start with something simple, gain experience, identify additional needs, and then get into developing unique requirements.

Caution. Statistical data--costs included-often are afforded more credence than they deserve. No matter how authoritative the inputs and methodology may appear, most cost estimates, for example, are little more than guesses. Unfortunately, many data collectors, analysts, and managers tend to lose sight of this and certain other points.

Not everything that should he considered in an economic analysis can be quantified. Even when some elements might otherwise be quantified, pertinent data may not be available. And whatever data are available may be subject to a variety of interpretations by the users. Such limitations apply to both empirical and future data--both costs and benefits.

The analyst and those who use the product should always bear in mind that it is dangerous to focus on specifics. It is much more realistic to consider a range of values whenever possible and to think in terms of relationships.

The essential elements of an economic analysis are (1) establishing the objective, (2) selecting alternatives, (3) formulating assumptions, (4) estimating the costs and its of each alternative, (5) comparing alternatives, and (6) testing alternatives under conditions of uncertainty.⁷

Generally, the process is similar to that of the scientific method of solving problems much seems clear enough, but whoever is responsible for establishing a program to do these things will have some very practical and valid questions.

When should an economic analysis be performed? Who should participate? Who should be responsible for its completion? What things should be considered? What sources should be used? How detailed should the product be?

Economic analysis techniques can be used anytime there is more than one way to accomplish a given objective. The process is a logical development in the evaluation of investment proposals and ongoing programs and is related to some more familiar analytical tools. Cost analysis is primarily a budgeting tool. It tells how many dollars are required to accomplish some objective or to determine whether a given proposal is worth further consideration and additional effort, but it does not assist in choosing among alternatives. Cost comparisons (a formal process done in accordance with Department of Defense Instruction 4100.33) are undertaken to deal with accomplishment of a given task by in-service civilian versus commercial lease, where the government alternative has been identified, possibly through economic analysis. Economic analysts and program evaluation are broader in scope than cost analysis or comparative analysis because they examine all alternatives and employ an evaluation of benefits. It should be apparent from these brief definitions that economic analysis techniques and other types of cost work are not mutually exclusive.

The key question in determining the appropriateness of an economic analysis is: "Does the potential impact of the decision make all this worthwhile?" As the time of the participants is usually pretty costly and sometimes any delay in reaching a decision can be at least equally costly, it is important to establish some criteria. At the beginning, this can he a simple dollar threshold keyed to the cost of the programs of most concern to management. Of course, any given program--regardless of its cost--may be of sufficient concern to the commander or senior staff to warrant a full-blown economic analysis. Also, there surely are points in between when the use of some of the procedures and methodology of economic analysis will be useful in helping management reach a decision. Criteria are only guidelines, and practical judgment should be used in every case to determine whether economic analysis is appropriate and its cost justified.

Although the need for economic analysis may be recognized in Congressional hearings and at the Pentagon, the term itself may frighten many of those whom it is intended to assist. Economic analysis and related techniques suffer from association with the Whiz Kid atmosphere in the DOD of the 1960s. "Systems analysis," "cost-effectiveness," and similar terms aroused suspicion if not antagonism. This must be overcome at all levels. The practical way to do so is to demonstrate applications at the level one is trying to reach and simplify the procedures and methodology to the greatest extent possible.

Participation. There probably are many ways to implement this advice, but AFCS has found that getting everyone involved works pretty well. First, get the commander and senior staff behind the program to a point where they habitually assess a proposal or ongoing program in terms of cost and benefits, asking "Is this the least costly or most beneficial way to go?" Second, insure that middle management is well aware of its bosses' interest. Third, show those who will be working the problem how to respond.

AFCS uses "working groups" extensively at both major command and intermediate levels. Representatives of the staff--planners, programmers, engineers, logisticians, and people from manpower and personnel--meet to resolve problems, formulate positions, and implement programs. Normally part of program implementation, although it can occur earlier, economic analysis is used by AFCS in determining the best way to provide a given communications-electronics-meteorological service or in evaluating the current methods of doing things.

Responsibility. The economic analysis program in Arcs is a responsibility of the Comptroller, who in turn has passed the authority for policy and implementation to the Management and Cost Analysis staff. Economic analyses and program evaluations, per se, are performed by a committee (working group) of mostly noncomptroller types. Responsibility for the product is vested in the functional manager or someone to whom the functional manager has delegated authority to accomplish whatever is necessary to meet project or program objectives. Normally, it is this "program manager" who has the ball. Someone from Management and Cost Analysis would assume responsibility if the analysis were concerned with a strictly comptroller application or if the office of primary responsibility were not clearly identified.

The point here is that whoever is most concerned with meeting the objective should have responsibility for the economic analysis product.

Process in Brief. The working group is given the objective or otherwise identifies it, then in an interactive exchange determines facts, assumptions, costs in terms of people and material, and alternatives. Comptroller representatives contribute their expertise in the cost and statistics areas. They provide the latest cost factors or develop those otherwise unavailable, generally act as consultants to assure the chairman that a comprehensive analysis is made, and provide whatever statistical help is needed.

The product is normally a complete, well-structured, accurate, and coordinated economic analysis. Seldom are issues raised above the working group level. The commander or other decision-maker is presented with an array of alternatives and a recommendation. Each alternative is shown with its initial investment cost and the cost of operation and maintenance throughout the system's life cycle, all in terms of present value; and the uniform annual (average) cost of each alternative is compared to all the others. This does not mean that the lowest cost automatically determines what is recommended. Quite often the benefits or a simple lack of limitations will actually drive the working group to recommend an alternative that is "most beneficial" and not "least costly."

The working group procedure briefly described here may not be the best approach for other organizations. Whatever the program’s structure and operational processes, the key elements listed the beginning of this section and shown here as applied in AFCS are necessary to make sure the facts are provided in a form that will assist and not confuse or degrade the decision-making effort.

This article is not intended to provide a litany covering all aspects of each step of the economic analysis process. One source, the DOD Economic Analysis Handbook, ⁸covers the entire process very well, and the reader should refer to it.

Uncertainty. The final step, however, is worth special mention. Since the costs of the future operations, and therefore the relationship of alternatives, are estimates and subject to uncertainty, it is always to question their validity and consequently their impact upon the findings of an economic analysis. Furthermore, the benefits and limitations of specific alternatives are often difficult or impossible to quantify. For these reasons, the frequently performs more than one type of analysis to determine the degree of impact on the outcome of a change in some important element.

Sensitivity analysis is one test. For example, as the DOD-directed discount factor used in calculating present values--and hence the uniform annual cost--is subject to frequent criticism,⁹ various discount factors can be applied to see just how significant the Defense figure really is. Another technique is contingency analysis, in which either a relevant criterion or a major assumption is changed. A fortiori analysis would be applicable where intuitive judgment favors a specific alternative, but analysis indicates it would be a poor choice. In this situation, any major uncertainties can be resolved to the advantage of the favored alternative, to determine if the results would change significandy.¹⁰

Format An economic analysis during its formulation will be quite detailed or at least appear so to the individual responsible for pulling all the pieces together. Another practical advantage of the working group approach, then, is the opportunity to draw on the individual and collective expertise and labor of functional representatives, while a largely independent review of the product by a cost analyst will assure that the final product is properly documented and statistically accurate. The analysis, in spite of many inputs, need not be detailed if it is presented in an outline or summary format and properly documented (so details may be filed and the analysis successfully defended if necessary). Relatively simple formats are suggested, for example, in Air Force Regulation 178-1; DOD is more concerned, however, with application than appearance. "There is no reporting system involved. If you don't like the suggested formats, develop your own."¹¹

Closing the Loop. Feedback is important throughout these processes. A truly professional and competent analyst will not he satisfied to develop procedures and methodologies that simply work. His or her product, of course, must be good enough to withstand the scrutiny of management and critics alike. Additionally, the cycle should generate actual costs and benefits that can be used to update the data hank and thereby improve earlier estimates, procedures, and methodology. What may be successful and fruitful one time may he inappropriate and useless when viewed in relation to empirical data. Feedback requires both added effort and a willingness to accept correction or at least improvement. This is simply a matter of closing the loop. It involves cost tracking and "post-expenditure analysis," another name for program evaluation.

Once an economic analysis has been completed in the planning and programming phases, it's absolutely necessary to track the subsequent decisions and results through budget formulation and execution.¹²

An economic analysis should reflect bath the environment in which it is developed and the prerogatives and criteria of the decision-maker it is intended to assist.¹³ The output, then, is a product that provides information to the deciding authority in a given decision-making situation by (1) illustrating the nature of the trade-offs between costs and service provided or mission performance and (2) summarizing background factors and nonquantitative considerations influencing the situation. What the decision-maker gets is a presentation of data relating the cost of a proposal to the benefits expected. This information will be used to decide which way to go, to support that decision if it must be justified to some higher authority or in order to obtain the necessary funds, and to provide a point of departure for future evaluation of the project or program.

Economic Analysis and program evaluation are concerned with the basic problem of economic choice. Economics is the science that deals with the rational allocation of scarce resources. Analysis refers to a process of systematic investigation. Economic analysis, then, is a conceptual framework for systematically investigating problems of choice; it generally means a kind of "pre-expenditure analysis." Using the same techniques, program evaluation is essentially "post-expenditure analysis."¹⁴

The products of economic analysis, within DOD, are intended to assist management in making rational and supportable decisions. The process is not meant to make the Defense manager's job or that of his staff more sophisticated. Neither is the process itself unnecessarily complicated and time-consuming. It can be, of course, but the key to having a truly practical and useful economic analysis program is to keep it simple and operate as much as possible within existing capability.

The preceding paragraphs have discussed significant elements of what is in fact a successful but still developing economic analysis program at major command level. The need was emphasized. The "who" and "what" were discussed As for application, many of the specific were not provided because this tends to bore or even unnecessarily frighten those from outside the comptroller field.

The intent was to motivate nonusers of economic analysis to try it The potential for returns that far exceed the investment is tremendous: mission-essential programs approved on the basis of rational thinking instead of emotionalism or whatever; less costly ways of operating and maintaining forces and support activities, identified and "sold"; assurance that decisions are based on logic as well as experienced judgment and not just intuition.

Economic analysis is not complicated. It is worthwhile, and it works.

Notes

1. Charles G. Leathers, ''Resources Available for Defense," lecture at Defense Resource, Management Education Center, Monterey, California, 11 July 1974.

2. Charles J. Hitch and Roland N. McKean, The Economics of Defense in the Nuclear Age (New York: Atheneum, 1966), p. 118.

3. Economic Analysis Handbook, Office of the Assistant Secretary of Defense (Comptroller), Washington, May 1971, p. 2.

4. Hitch and McKean, p.120.

5. Directory of Training, Films, Publications, and Models on Defense Economic Analysis and Program Evaluation, Department of Defense, Defense Economic Analysis Council, Washington, May 1974.

6. Major John W. Mather, "Implementation of the Economic Analysis Program in the Air Force Communications Services," master of arts research paper, University of Oklahoma, 1974, pp. 22-26.

7. Economic Analysis Handbook, pp.2-9.

8. Ibid.

9. Major Richard Zock, "Resource Management, Economic Analysis, and Discounting in the Department of Defense," Air University Review, XXIV, 2 (January-February 1973), 34-35.

10. Economic Analysis Handbook, pp. 8-9.

11. Colonel Vincent J. Klaus, "Why DOD Needs Economic Analysis and Program Evaluation," Commanders Digest, vol, 15, January 3,1974, p. 13.

12. Brigadier General Leslie R. Sears, Jr., quoted in "AMC Works on 'Program Evaluation,'" Commanders Digest, vol.13, January 3, 1974, p. 5.

13. Economic Analysis Handbook, p. 9.

14. Klaus, p. 13.

Contributor