In those circumstances where technical uncertainty is present and experimental development is required, I often see two approaches to experimentation. The first approach is referred to as the "One-Factor-At-A-Time" or "OFAT" approach and the other is a Statistically Designed Experiment or "DOE". The DOE is preferred for reasons that will be discussed. Unfortunately, the OFAT approach is most often used in industry. Within the definition of SRED is the idea that scientific research or experimental development is commensurate with a systematic investigation or search. Within the context of this article, I shall seek to demonstrate that a systematic investigation is superior to a search and show how a DOE is likened to a systematic investigation while the OFAT approach is nothing more than an inefficient search. The broad based acceptance, in industry, of DOEs as an approach to conducting a systematic investigation will accelerate technological advancement.

An approach often used in scientific research, experimental development, and engineering studies is to identify an important factor to study versus a response of interest while ignoring the other factors by keeping them constant. Once the single factor has been studied, recommendations specific to that factor and its effect on a response of interest are made and then the researcher moves onto the next most important factor and repeats the study again. This approach continues until an acceptable solution is found. This technique to experimenting is referred to as the "one-factor-at-a-time" or OFAT approach. Rarely does the OFAT approach yield a solution after studying only one factor. Several iterations are required and often do not result in an optimal solution. The OFAT approach is used without reservation in many investigations, especially by those who have not been exposed to statistically designed experiments. By comparison to statistically designed experiments the OFAT approach has the following disadvantages:

1. It requires more runs for the same precision in effect estimation.
2. It cannot estimate some interactions.
3. The conclusions from its analysis are not general.
4. It can miss optimal settings of factors.

We will examine each of the four points mentioned through an experiment using both the OFAT approach and a 2³ factorial experiment to study a physical system of interest. The factors have been identified as A, B, and C having the coded levels "-1" and "+1" where the coded levels represent two physical settings that must be changed by the experimenter.

A (-1 and +1)              B (-1 and +1)              C (-1 and +1)

OFAT Experiment

The first step of the OFAT approach requires that we change one of the factors while fixing the others. Factor A is thought to be the most important. As such, factors B and C will remain at their standard conditions (B +1 and C = +1). The two levels of A "+1" and "-1" are compared. The experimental trails along with their result are shown. A lower result (29) is better than the higher result (40) and occurs when Factor A is set at "-1".

First Experimental Runs: Changing Factor "A"

The result of this OFAT experiment yields a factor effect for A = 40 - 29 = 11.

The next factor of importance is B. By fixing A = -1 from our first experimental and C = +1 the two levels of B at "-1" and "+1" are compared. Here, B=-1 is chosen as it gives a smaller results as shown in the table. The two levels of B "+1" and "-1" are compared. The experimental trails along with their results are shown. A lower result (17) is better than the higher result (29) and occurs when Factor B is set at "-1".

Second Experimental Runs: Changing Factor "B"

The result of this OFAT experiment yields a factor effect for B = 29 - 17 = 12.

Finally, the remaining factor, C, is examined. The two levels of C are compared with A=-1 and B=-1 which were based on the previous experimental runs. The level C=-1 is chosen since it results in a smaller result.

Third Experimental Runs: Changing Factor "C"

The result of this OFAT experiment yields a factor effect for C = 17 - 11 = 6.

Based on the results of this experiment, setting factors A=-1, B=-1, and C=-1 yields the lowest result of 11.

We will now compare the OFAT approach to a 2³ factorial experiment. A 2³ is a factorial experiment that studies "3" factors having "2" levels for each of the factors being studied. The experimental trails along with the results are shown in the following table.

Experimental Matrix for 2³ Design and Resulting Data

This factorial experiment yields results lower than the finding obtained in our OFAT experiment. In fact, one of the results is substantially lower. To explain why the factorial experiment is superior, we will use a graphical illustration. The following illustration defines the experimental region. The corners of the cube represent each of the experimental treatments and the results that were obtained in our 2³ factorial experiments. The dark circles represent the OFAT experimental conditions.

2³ Experimental Space

Notice that the OFAT experiment only visits four corners of the cube as indicated by the dark circles. Since the cube graphically depicts the experimental spaced define by these three factors and each of their two levels, the OFAT missed 4 other conditions where lower results were observed.

To explain the first disadvantage, notice that at each level of factor A there are four observations (11, 25, 17, 29). In fact, each of levels of the remaining factors has four observations. As such, each of the experimental trails in our OFAT experiment would require 4 replicates to have the same level of precision (i.e. variance) as our 2³ factorial experiments. Recall that in the 2³ design, each factorial effect is of the form "+1 average" minus "- 1 average" where the average for each of the levels of a factor is the average of four observations. In order to get the same precision for the estimation of a factor effect from our OFAT experiments we would need to collect 4 observations at each of the four dark circles shown in the geometric representation. As such, the OFAT experiment requires 16 experimental trails as suppose to the 8 required in the 2³ factorial experiments. As such, the factorial experiment is much more efficient at estimating the factor effect than an OFAT experiment. Since we did not replicate the OFAT experimental conditions, the factor effects computed may not be precise. To illustrate, let's compute the factor effects using the data from the factorial experiment.

Factor A

Average A+ = ( 2 + 9 + 37 + 40 ) / 4 = 22

Average A- = ( 11 + 17 + 25 + 29 ) / 4 = 20.5

Average Effect of A = 22 - 20.5 = 1.5

Average B+ = ( 25 + 29 + 37 + 40 ) / 4 = 32.75

Average B- = ( 11 + 17 + 2 + 9 ) / 4 = 9.75

Average Effect of B = 32.75 - 9.75 = 23

Average C+ = ( 17 + 29 + 9 + 40 ) / 4 = 23.75

Average C- = ( 11 + 25 + 2 + 37 ) / 4 = 18.75

Average Effect of C = 23.75 - 18.75 = 5

The factor effects for both the OFAT and factorial experiment are shown. The computed OFAT effects differ. The effect of A is over estimated, the effect of B is underestimated, and the effect of C is close to the effect estimated by the factorial experiment.

Another disadvantage of the OFAT approach stems from the inability of the OFAT experiment to estimate interactions. While subject matter experts are good at identifying the import factors that have large single effects before the experiment, they are usually not as successful at identifying the important interactions effects.

The data in the table shows how we compute the multiplicative or interaction effect between two factors, something an OFAT approach cannot accomplish. To compute the factor effect of the A x B interaction we must multiply the levels of factors A and B as illustrated in the following table.

Computing A x B Interaction Effect

Average AB+ = ( 11 + 17 + 37 + 40 ) / 4 = 26.25

Average AB- = ( 25 + 29 + 2 + 9 ) / 4 = 16.26

The resulting A x B interaction effect is 26.26 - 16.25 = 10. We often call this a two factor interaction. There are other two factor interactions that may be computed, they are the A x C and B x C interaction effects. They are both computed using the same procedure to compute the A x B interaction effect. The effect for both two factor interactions is;

AC = [ (11 + 25 + 9 + 40 ) - ( 17 + 29 + 2 + 37 ) ] / 4 = 0

BC = [ (11 + 29 + 2 + 40 ) - ( 17 + 25 + 9 + 37 ) ] / 4 = -1.5

There is one more remaining interaction effect, the A x B x C interaction or the three factor interaction. Its effect is computed by multiplying the signs of A x B x C. The effect of the interaction is shown below.

ABC = [ (17 + 25 + 2 + 40 ) - ( 11 + 29 + 9 + 37 ) ] / 4 = -0.5

Notice the AB interaction effect is large compared to A, B, and C respectively. The remaining interaction effects (AC, BC, and ABC) are small. Let's see how our knowledge of the A, B, C, and AB effects can assist us in identifying the optimal condition. We will use as our frame of reference the centre of the cubic which represents the overall average of our experiment or the mid point between +1 and -1. We called this the "0" point which is 11 + 17 + 25 + 29 + 2 + 9 + 37 + 40 = 21.25. The table below shows the half effect which is simply the difference between the either level and the "0" point.

Based on the results in the table and using the overall average or "0" point as our starting value, we can use the half effect to determine the optimal condition where lower is better. Setting A to "-1" will result in a change from 21.25 to 20.50 (21.25 - 0.75 = 20.50). By setting B to "-1" will result in a change from 20.50 to 9 (20.50 - 11.5 = 9). Now setting C to +1 will result in a change from 9 to 6.5 (9 - 2.5 = 6.5). Finally, setting AB to "-1" would result in a lower value. However, AB is predetermined by the existing levels of AB which are both "-1" resulting in an AB condition that is "+1". Since the half effect of A is small, setting A to "+1" and keeping B at "-1" will result in an AB condition that is "-1". By setting A to "+1" results in an increase from 6.5 to 7.25 (6.5 + 0.75 = 7.25). Now taking the AB interaction into account the final result is 7.25 to 2.25 (7.25 - 5 = 2.25). The experimental conditions corresponding to A=+1, B=-1, and C=-1 equals 2.0. As shown, our prediction is off by 0.25, only because we did not take into account the other interactions. However, our prediction based on the half effects identified the optimal settings, something our OFAT experiment failed to do. 

The interactions discussed, especially the AB interaction contribute to the optimal settings and shows that if the search path in the OFAT approach is not chosen properly, it can miss the optimal setting. The disadvantage of the OFAT approach is rooted in the fact that it does not conduct a systematic investigation over the experimental region. This is clearly seen from the geometric representation and the fact that when a factor is being studied, the levels of the other factors are fixed. This type of approach to experimentation does not simulate the reality of a physical system where the factor levels rarely remain constant. In factorial designs, each factor effect is computed over all possible level combinations of the other factors. This is not true for the factor effects computed in the OFAT experiment and explains why they differ from the factorial experiment. As such, if two experimenters are studying the same physical system their OFAT approach may different even if they use the same factors and levels. Experimental conditions will differ simply by studying the factors in a different order. Consequently, a significant factor effect in the current experiment may not be reproduced in another study if the latter involves some level of combinations not included in the current experiment. As such, the results will differ and put in question the validity of their experiments.

In summary, a statistically designed experiment collects the required information with the least expenditure of resources and assures replication by different experimenters. They are the only efficient means to conducting a systematic investigation.