Document created: 8 September 04
Air University Review, November-December 1971

A Combat Crew Production Function

A production function describes how certain inputs can be combined to produce a given output. Because of the widespread belief that a quantifiable measure of military output does not exist, there has been little effort to estimate the parameters of such a function for military personnel. I suggest, however, that an acceptable proxy for this output could be the Operational Readiness Inspection scores generated by various units of our operational commands. These scores give a measure of the effectiveness we might expect from our units and personnel if they were ever put to the test in an actual wartime situation. Actually, this is the rationale behind the Operational Readiness Inspection (ORI).

Let us take the Strategic Air Command as an example. A SAC crew’s primary mission, in case of war, is to put its bombs and missiles on the target. The ORI bombing scores give a relatively objective measure of how SAC crews would perform this function. I am not arguing that these scores provide the full measure of an individual’s output; certainly there are other factors that one would wish to consider. However, a crew’s superior performance in this particular area does enhance SAC’s primary mission: to deter aggression. These scores, therefore, do provide a first-order measure of output.

This article describes the preliminary results of a feasibility study in the SAC ORI-Personnel subarea. The question which it attempts to answer is, “Are SAC ORI bombing scores randomly generated, or can we determine some basic input factors that significantly influence their values?” In other words, is there a significant relationship between a crew member’s background and how well he performs on a given ORI? Specifically, what type of returns do we obtain from such factors as experience in the aircraft, length of service, age, and educational level? Additionally, are qualitative ratings such as Officer Effectiveness Reports and crew designations (Ready, Senior, or Select) good indicators of how well the individual will perform on the ORI?

the data base

The sample period covered by this study ran from July to December 1967. During this period 30 SAC B-52 Strategic Wings participated in the Olive Pit Express simulated wartime mission as part of either an ORI or a Bar None exercise. The two exercises are identical except that the ORI is monitored by an Inspector General team whereas the Bar None exercise is not. All scores recorded during these flight missions were transmitted to Headquarters SAC and later provided for the study by the SAC Inspector General and Director of Training.

The individual SAC wings then provided lists of all crew members who participated in the Olive Pit Express exercise, so that personnel background data on these individuals could be extracted from the master file by the office of the Director of Personnel Planning, DCS/Personnel, Headquarters USAF. These data were mated with the crew ORI scores to provide the sample observations.

The decision whether or not to include a particular crew in the sample was predicated on two requirements. The first requirement was that the crew had flown the low-altitude phase of the mission and recorded bomb scores during that phase. The second requirement was that all large crew scores had resulted from crew error and not from materiel failure or unknown causes. Both these requirements were imposed to provide a more consistent measure of crew output. The data mating process, subject to these requirements, provided 387 observations, or “crews over the target,” for the sample analyzed in this study.

A list of the variables included in the sample is shown in Table 1. The qualitative variables take the value 1 or 0 and are used to classify the data by groups. Such classifications can often increase the sensitivity of the analysis, as will be shown later.

One qualification must be put on the results: Absolute bomb scores are considered security information and cannot be given in an unclassified article. For this reason whenever the results are presented graphically, one score is designated by the letter X, and other scores are shown as incremental differences from this base score. When the results are presented algebraically, the constant term in the equation is denoted by the Greek alpha, a. These procedures disguise the absolute scores obtained by the crews while still permitting one to observe the relationships between the scores and input variables—the subject of this study.

crew personnel characteristics

The high correlations between certain personnel background variables, such as total flying time, commissioned service, and age for each crew member, made it impossible to determine the independent influence of these variables with any degree of confidence. As an example, the correlation matrix for the aircraft commander variables is presented in Table 2. In situations such as this, the most effective variable for explaining the desired relationship must be chosen to stand for the common factor, which in this case could be labeled “experience.” For this study that variable is total flying time.

Another rather interesting finding of the study was that ORI scores were more sensitive, in a statistical sense, to the personal background of the aircraft commander (AC) than to that of the copilot (CP), radar navigator (RN), navigator (NAV), or electronics warfare officer (EWO). This may be surprising to some who suppose that scores would be most sensitive to the personal background of the radar navigator, the crew member who actually does the bombing. Apparently the aircraft commander sets the pace for the whole crew, whose effectiveness may be highly dependent on his leadership ability. One reason for this finding could be that the aircraft commander’s background is usually representative of the background of the entire crew. The correlations presented in Table 3 give credence to such an argument. The experience levels of the individual crew members are positively correlated. Except for the correlations between the aircraft commander and navigator and between the navigator and electronics warfare officer, all correlation coefficients are statistically significant beyond the 99 percent confidence level. In general, the more experienced aircraft commanders have the more experienced crews. For these reasons, the aircraft commander’s background data were used to represent the entire crew.

sensitivity results

On the Olive Pit Express exercise, four low-altitude simulated bomb drops were recorded. Ideally the average score should be used as an output variable, but tests using each individual score and the average indicated that the first score recorded was the main discriminator among crews. The variance around the mean score for the first target was approximately five times that of the others. Therefore, the variance of the first score dominated in the variance of the average, so that each gave essentially the same results. In addition, it was noted that the same bombing method was used on the first target, whereas the method varied considerably on the other targets as crews evaluated their equipment status after the first simulated bomb drop and changed to alternate bombing modes. Therefore, for reasons of both sensitivity and consistency, the first score rather than the average score was chosen as the measure of output in this study.

As previously mentioned, a number of qualitative variables were used to classify the scores in various categories. If there is a significant difference between the mean scores for each category, the analysis can often be improved by holding these mean effects constant in the regression equation. The remainder of this section describes the results of these sensitivity tests.

Aircraft model. The various B-52 models have slightly different bombing equipment, and if these differences significantly affect the scores, these effects should be taken into account when evaluating personnel effectiveness. An analysis of variance test indicated that the scores recorded by one model were significantly higher than those of other models. Therefore, this effect was held constant in the regression equation estimated in the following section. For security reasons it is disguised in the constant of that equation.

Aircraft commander’s source of commission. An analysis of variance test indicated there were no statistically significant differences between scores classified by the aircraft commander’s source of commission.

Substitute crew member. Seventeen percent of the crews in the sample had a substitute for one of the five officers on the crew. Contrary to the belief that substitutes adversely affect crew coordination, there was no significant difference between the scores of crews with and without substitute members. Perhaps the explanation lies in SAC’s highly standardized crew procedures.

ORI versus Bar None. There was no significant difference between the scores recorded by crews flying the ORI exercise monitored by the Inspector General team and those flying the unmonitored Bar None exercise.

Crew designations. There were significant differences between mean scores recorded by crews with different designations—Ready, Senior, or Select. This indicates there was a qualitative difference between the crews included in the sample. These qualitative differences were incorporated in the general model described in the following section, and their significance is explained there.

combat crew production function

Regression analysis was used to estimate the parameters of the combat crew production function. Ideally, this technique takes into account the interaction between input factors, and it attributes to each its marginal contribution to output. Sometimes, when the correlations between a set of input variables are extremely high, as between total flying time, commissioned service time, and age, it is impossible to separate the individual influence of each input. The technique is normally successful, however, in handling lower correlations, and confidence can be placed in the estimated parameters as long as the T tests are significantly high.

As an example, total flying time was negatively correlated with educational level (Table 2), since a number of the longer-time crew members entered service through the aviation cadet program, for which a college degree was not required. More recent entries had college degrees whether they entered through a service academy, ROTC, or OTS program. As might also be expected, total flying time was correlated with the qualitative crew variables because, in general, the Select crew members had more flying time than Senior and Ready crew members. In both cases, however, it was possible to break out the separate contributions of these inputs.

The most prominent factors in explaining crew bomb scores were the qualitative ratings of the crew (Ready, Senior, Select) and the aircraft commander’s experience and educational level. The estimated production function, using total flying time for the experience factor, is presented in Table 4. The T statistics for each of the estimated coefficients are presented in the right column and are based on the null hypothesis that the coefficient is equal to zero. All tests are significant beyond the 95 percent confidence level.

The R² statistic indicates that only 20 percent of the variation in bomb circular errors (CE) was explained by this function, but this is not unusual for a relationship estimated by cross-sectional data drawn from an operational environment. The fact that 80 percent of the CE variation is still unexplained does not render the function useless. The T statistics indicate that bomb CE’s are responsive to changes in the included explanatory variables, so that the estimated model can be used to examine policy alternatives.

The production function defines a three-dimensional surface with output (bomb CE) a function of the two inputs, flying time and educational level. The level of this surface is shifted vertically depending on whether Ready, Senior, or Select crews are being considered. This concept is rather difficult to visualize unless some of the factors are held constant at their means and the influence of the others is expressed in a two-dimensional graph, as in the following subsections.

Crew designations. When a crew first attains combat status, it is given the crew designation Ready. After the crew has performed well, the wing commander, taking into consideration such distinct empirical data as daily bombing scores and navigation proficiency, will promote the crew to Senior rank. The Select crews are then picked from the Senior crews who have performed in an outstanding manner.

The qualitative differences between the three crew categories can be depicted by holding the effects of the flying-time and educational-level variables constant at their means and adding these effects to the constant a₁. The bomb CE’s then become a function of only the differences between crews:

This equation defines the step function depicted in Figure 1. In this and each succeeding figure, one must remember that bomb CE reflects the miss distance from ground zero. Therefore lower scores are desirable.

Figure 1 shows that there is a significant difference between the mean scores by crew category, particularly between the Ready crews and the other two classifications. The wing commander’s qualitative ranking system can be verified on the basis of ORI and Bar None scores—not surprising since his selection process incorporates past performance indicators. Conversely, the crew designation provides an insight into the expected performance of a crew on the simulated wartime mission.

Experience factor. The independent effect of experience can be obtained by holding the qualitative crew differences and educational levels constant at their means and observing the production equation as a function of only the flying-time variable:

This equation defines a cubic relationship and is plotted in Figure 2. The bomb CE’s decrease until reaching 4500 flying hours and then begin to rise at an increasing rate. In addition, at approximately 2200 flying hours a point of inflection is reached, where the curve begins to bend upward, from increasing to decreasing returns to experience.

It is interesting to note that the shape of this particular function conforms closely to that of the traditional production function visualized in economic theory. If the vertical axis in Figure 2 is translated so that higher values are desirable, the production function takes the form shown in Figure 3. It shows increasing returns from additional flying hours in area A, decreasing returns in area B, and negative returns in area C. This particular production function is complicated, however, by the fact that it takes time to gain experience and move out the horizontal axis.

A regression equation, using commissioned service time, gives essentially the same form as that for flying hours. This is to be expected, since the two variables are highly correlated. A compatible commissioned service scale has therefore been included in Figure 2 below the horizontal axis. It shows that the minimum bomb CE occurs at 4500 flying hours or 11 years of commissioned service. The average aircraft commander has somewhat less flying time but more commissioned service than this.

The increase in bomb CE’s depicted by the right portion of the curve is interesting because it indicates that there may be a time beyond which it is undesirable to keep a pilot on a combat crew. It should be emphasized, however, that the solid experience curve is based on a weighted average of Ready, Senior, and Select crews. It does not necessarily imply that an older Select crew member will receive higher error scores than a younger Ready crew member. Both the qualitative and quantitative aspects must be taken into consideration. The Select curve falls below the weighted average, and the Ready curve above it. Portions of these curves are shown by the dashed lines in Figure 2. These curves do imply that within each category the younger crews are more effective.

One could argue that the rise in CE’s results from a qualitative change in the mix of the force through time as the more effective crew members advance to command and staff positions. However, this is a continuing phenomenon, and it would be inappropriate to deny advancement to the most proficient individuals just to keep the average quality of the crews up—unless, of course, aircrews are to be recruited on a career basis, as some people advocate. Barring this, we must still cope with the rising CE.

One might also question whether the flying time mix of B-52s and other aircraft types might not affect the slope of the experience curve. The high correlation between B-52 flying time and total time made it impossible to determine these separate effects with any degree of confidence. An analysis of the data, however, showed that approximately 40 percent of each aircraft commander’s total time had been in the B-52. The graph in Figure 2 reflects this mix.

Educational level. The final factor influencing output in the production function is educational level. Holding the other factors constant at their means gives the following production equation:

where Z is the educational level code. This reciprocal relationship is rather interesting because when plotted it takes the form shown in Figure 4. Positive returns result from higher education, but the incremental returns are not as great at higher levels of education as they are at lower levels. This would appear reasonable for aircraft crew members. The educational range for SAC aircraft commanders ran from high school graduate to master’s degree.

The actual plot of CE’s against educational level is given in Figure 5. (When comparing Figures 2 and 5, one should note the difference in the vertical CE scale.)

Officer Effectiveness Report

Beyond isolating Select crews, the Officer Effectiveness Report index gave little indication of how a crew member would perform on the ORI and Bar None exercises. Of course, the Select crews scored lower bomb CE’s, but the incremental differences between scores and OER’s by crew type were not highly correlated. This is illustrated in Figure 6, using the Ready crew as a base, where D stands for the difference in OER’s and CE’s between two crew ranks.

It would appear that Select aircraft commanders’ OER’s are out of proportion to their performance on the simulated wartime mission, but a statement such as this must be qualified by other considerations. First, the OER index reflects a number of other factors in addition to performance on simulated wartime exercises. Leadership potential, in particular, is highly weighted. Second, Select crews normally have the additional function of instructing and evaluating other crew personnel in the organization. All these factors are weighted in the OER index. Considering this, it is not surprising that the OER index fails to provide a particularly good forecast for performance on the simulated wartime exercises.

This article outlines the results of a study designed to determine the feasibility of estimating the parameters of a military production function. The output measure is the bomb scores generated by SAC B-52 crews on a simulated wartime mission. These scores are influenced by both qualitative and quantitative factors.

Qualitative factors are necessary because there are distinct differences between individuals that cannot be measured by the normal quantitative variables. The wing commander’s crew rating (Ready, Senior, or Select) provides the best input to describe this effect. This rating provides a valid estimator of a crew’s performance on the simulated wartime mission.

Holding these qualitative differences constant, one finds that two of the aircraft commander’s background factors, which can be quantified, also affect the crew’s output. The first of these is an experience factor which can be represented by either total flying time or total commissioned service. Using either of these variables, when one estimates the traditional production function, he finds first an area of increasing returns to experience, then decreasing returns, and, finally, negative returns.

The second influential factor in the aircraft commander’s background is his educational level. The estimated relationship shows that we experience positive returns with higher levels of education throughout the sample range. The incremental returns, however, are greater at lower levels of education than at higher levels.

The results of this study do raise some interesting questions about the possible structuring of our rated force, and they should be investigated with further analysis. For instance, on the curve in Figure 2, the optimal experience level appears to be at approximately 4500 flying hours or 11 years’ commissioned service. Beyond this point, effectiveness begins to decrease. At about 6700 flying hours or 16 years’ commissioned service, effectiveness has decreased to a point equal to that of a new aircraft commander. Would this be the ideal time to remove a pilot from combat crew duty? Not necessarily. It might be somewhat before or even after this time. Ultimately, it will probably depend on how much importance is placed on an increase in bomb CE, which in turn depends on such factors as weapon lethality, bombing pattern, target vulnerability, and the quality of target intelligence.

Costs also are important as well as rather complicated. Cost determination will depend on how the individual is utilized after he is removed from crew duty. If he is placed in a nonrated position that contributes very little to the flying portion of the Air Force mission, then his high training cost must be amortized over a shorter span of time. In addition, another pilot must be found and trained to replace him, necessitating a trade-off between an increase in force cost and an increase in bomb CE.

More than likely, however, the rated individual performing in a nonrated capacity still contributes, at least in part, to the flying mission of the Air Force, making it rather difficult to figure the amortization of his training cost. For example, how much of his flying is productive in the sense that it supports the unit mission and how much is unproductive, or straight proficiency flying? If this individual is placed in a nonflying position that requires a rated background, his training cost can still be amortized over the original period. The problem then becomes a question of how many of these command and staff positions are available. Surely there will not be enough to cover the increased demand if officers are arbitrarily removed from combat crews after a given number of flying hours or years of commissioned service.

One proposal that has intuitive appeal is to rotate a number of these officers who are not programmed for command and staff positions through flying jobs that are decreasingly rigorous. For example, if it is agreed that combat crew duty in B-52s is a rather strenuous activity, there may be a point at which it would be desirable to rotate rated personnel into less rigorous but still primarily flying jobs. According to this proposal, a hypothetical career pattern might take a pilot through fighters or bombers, and then transports. Before such a career pattern could be recommended, however, additional studies—such as this—are needed to determine the sensitivity of effectiveness to flying time or length of service for each type of flying activity.

The curve of Figure 3, showing the decrease in bomb CE’s with increasing education, must also be evaluated with caution. For example, it appears that a decrease in bomb CE’s of approximately 1000 feet could be expected as the force moves from one of high school graduates to one of college graduates. But there is both a social cost and a military cost involved in procuring a more highly educated rated force. The decision-maker must decide whether the increase in effectiveness is worth the higher costs. It must also be emphasized that the results presented in this article pertain only to primary aircrew members and not to personnel occupying higher command and staff positions—positions in which the returns to education may be considerably higher. If a substantial number of these positions are to be filled by members of the rated force, education takes on added emphasis.

The questions outlined above indicate how valuable well-defined personnel production functions could be as an aid in structuring the military forces. The main purpose of this study has been to determine the feasibility of estimating the parameters of such a function. It does appear that reliable statistical results can be obtained in areas in which a quantifiable measure of military output does exist. Additional and more detailed research is needed now.

 United States Air Force Academy

This research was originally described in Air Force Academy Technical Report 69-1 dated September 1969.

Lieutenant Colonel Herman L. Gilster (USMA; Ph.D., Harvard University) is Tenure Associate Professor of Economics and Management, USAFA. He was Air Training Officer, USAFA, 1955-57, and has been at the Academy since 1963 except for a year as Chief, Tactical Analysis Division, Hq Seventh Air Force, PACAF. His articles have been published in Papers in Quantitative Economics (University of Kansas Press), in Air University Review, and as USAFA Technical Reports.