Conformance probability in the assessment of Calibration and Measurement Capabilities

We argue that the assessment of the Calibration and Measurement Capabilities, CMCs, by means of the results of a Key Comparison is a bona fide exercise of conformity assessment, and as such should be treated, using the appropriate tools, including risk assessment. This position contrasts with the current practice, in which acceptance or rejection of a CMC claim are based on the normalised error. We show that, behind this seemingly unique acceptance criterion, different decision rules - guarded acceptance, simple acceptance and guarded rejection - exist in reality, depending on the characteristics of the comparison. This variety of decision rules impairs the fairness of the current equivalence arrangement. We suggest that the conformance probability should be the key parameter to be considered in the assessment of a CMC claim. Using a suitable Probability Density Function, PDF, for the measurand, we calculate the conformance probability for the possible scenarios, and show that using the current acceptance criterion the conformance probability can attain unacceptably low values. Therefore, we maintain that the current acceptance criterion is ambiguous and inadequate, and suggest to rather adopt a criterion based on the calculation of the conformance probability and the establishment of a minimum threshold for acceptance. We demonstrate our proposal by applying it to a practical case and to a fictitious example in mass metrology.