The APC-criterion for the Analysis of Screening Experiments

Abstract

This thesis introduces an AIC-type model selection criterion, called the APC-criterion (All Possible Comparisons criterion) for analysing a variety of screening experiments.The primary focus is on the orthogonal 2-level designs including the Plackett-Burman designs (PBD) where the effects of k factors can be analysed in only n = k + 1 runs (n is a multiple of 4). The proposed APC procedure has the advantage that it can be designed to maintain strong control of the false positive rates at a specified level for three different definitions of error rate: individual error rate (IER), experiment-wise error rate (EER), and false discovery rate (FDR). The criterion can be used to analyse data from any two-level designs, that are orthogonal or nearly orthogonal. Extensive simulation studies were carried out to demonstrate the procedure of APC-criterion for a wide range of experimental scenarios. The APC-criterion was identified as an effective analysis method in terms of (i) controlling IER, EER or FDR at the specified level while (ii) delivering high values of screening power and accuracy as long as the effects are big enough.The performance of commonly used AIC and BIC criteria and their modified versions was compared with the APC-criterion for a variety of scenarios using saturated PBD’s. The classical criteria are powerful, since they tend to select bigger models in most cases. However, these criteria are less effective for analysing screening experiments, since their observed IER, EER and FDR were found to be higher than that for the APC-criterion. The error rates for some of the modified versions, i.e., ACIc, mAIC,modAIC and modBIC were found to be comparable with the APC-criterion for certain experimental scenarios. The advantage of APC-criterion is that it controls the error rates in all cases with unreplicated two-level orthogonal screening experiments, which is not possible with commonly used model selection criteria. This thesis further explored the scope of applying the APC-criterion for two-level balanced non-orthogonal designs (NOD’s). The extensive simulation study revealed that the modified procedure for NOD’s controls the IER, EER and FDR at the specified levels when the design is D-optimal or at least the amount of aliasing is not too large. The procedures of APC-criterion for orthogonal and non-orthogonal designs were adapted with the two step approach of Miller and Sitter (2005) and applied to carry out an initial study for analysing folded-over 12-run PBD in the presence of few 2-factor interactions. Finally, the R-package “APCanalysis” which implements the procedure of APC-criterion in R was described and demonstrated for orthogonal and non-orthogonal designs using constructed examples and real data sets.