Power of a test for assessing interlaboratory consensus of nominal and ordinal characteristics of a substance, material, or object

A concept of the consensus among different laboratories participating in an interlaboratory comparison, classifying a substance, material, or object according to its nominal and ordinal (i.e. categorical) characteristics, is devised using decomposition of the total variation of the laboratory responses. One of the components of the total variation is caused by the between-laboratory differences, and the second—by conditions associated with the applied experimental design (for example, temperature of test items, technician experience, etc). This decomposition is based on the recently developed two-way CATANOVA for nominal variables and two-way ORDANOVA for ordinal variables. The consensus is tested as hypotheses about homogeneity, i.e. insignificance of the corresponding components of the total variation. The consensus power is taken to be the power of the homogeneity test. A methodology for evaluation of the consensus power and corresponding risks of false decisions versus the dataset size of categorical characteristics obtained in an interlaboratory comparison is detailed. Examples of evaluation of the power and risks are discussed using previously-published datasets of an interlaboratory comparison of identification of weld imperfections, and an examination of the intensity of the odor of drinking water. An example of computer code in the R programming environment is presented for the power calculations in the case of nominal variables, using a chi-square distribution. A newly developed tool for ordinal variables, an Excel spreadsheet with macros, which is based on Monte Carlo draws from a multinomial distribution, is also available.