Skip to main content

Analysing the results (4): Paired comparisons: Tukey's test

There are a variety of techniques for analysing the significance of the variation between groups (in this case washing machines). We used the Tukey test, but others are equally applicable. The worksheet "Tukey - monaural quality whole" in the Excel spreadsheet of results gives an example analysis.

This is the case of the variation in overall quality, and we have previously seen a graphical representation of the results:

Washing machine chart

We take the data in pairs and see if A is significantly different from B, A from C, A from D etc.

For example, the difference between the means of A and B is 2.42. This mean is compared to a test parameter (0.888 in this case). As the difference in the means is greater than the test parameter, then we can see A is significantly different from B (as might be expected from the graph).

Forming the test parameter:

ParameterValueNotes
k5

Number of kettles.
This is the k-value in the studentized range distribution table

Degrees of freedom49

Number of washing machines multiplied by the number of subjects -1
You will find it in the ANOVA table made by Excel

Mean Square Error MSE0.493You will find it in the ANOVA table made by Excel
q (Studentized range upper quantiles)4This is read from the studentized range distribution table
N, Number of washing machines10 
Test parameter0.888q*sqrt(MSE)/SQRT(N)

We have to test every pair of washing machines against this test parameter. The spreadsheet shows how this can be set up most efficiently. For these tests we find that A is significantly different from B C D and E and B is significantly different from C D and E. This is normally expressed by a line plot where a line is drawn under any subset of adjacent means that are not significantly different

part of questionnaire, purchase influence question

This shows that there are three groups in this case: A : CDE : B, which is consistent with what might be guessed from the plot above. Looking back at the question this refers to:

another question

It shows that the sound of washing machine A is most likely to be associated with an expensive product, and the sound of washing machine B is most likely to be associated with a less expensive product. Washing machines C D and E lie somewhere in between.

For other cases, the interpretation is more awkward. Take the case of binaural, draining cycle, quality:

Line chart for binaural, draining, quality

These results show that machines: CA ABE and BED are grouped. So we can say that CA are among the best sounding, and ED are among the worst sounding. The position of B is more ambiguous because it is grouped with both the worst and the best, so it is difficult to draw any conclusion about that washing machine.