Multitrait Multimethod is an approach to examining construct validity developed by Campbell and Fiske (1959). It organizes convergent and discriminant validity evidence in a matrix, enabling comparison of how the measures relating to the same construct relate. This conceptual approach has influenced experimental design and measurement theory in psychology, including applications of structural equation models.
Multimethod analysis involves collecting measures of the same trait from multiple sources, such as self-report and peer report. Measures are then correlated to form a correlation matrix, which can be used to judge the strength of a construct. The correlations in the matrix can also be compared to see if they are consistent across methods (convergent validity). If the results are similar between different measures, then the construct being measured is probably reliable and valid. If the results are highly inconsistent between measures, then the construct may be unreliable and invalid.
The MT-MM method was developed to examine convergent and discriminant validity, which are important in determining the quality of psychological tests. The matrix is a table that displays the correlations between pairs of traits or sets of measures, in a way that allows for the separation of correlations between different traits measured by the same method from those between different methods.
In the example shown in Figure 1, the MTMM matrix shows that the pro-vs.-counter and neutral argumentation percentage scores from two studies correlate with each other and with the critical thinking ability score from the same study. This is a strong indication of convergent validity. In addition, the MTMM matrix also shows that the pro-vs.-counter argumentation percentage scores from the two studies correlate with the critical thinking ability score from the same studies, but not with the creative thinking ability score. This is a good indication of discriminant validity.
When analyzing the MTMM matrix, it is important to look at the proportion of correlations in the validity diagonals. Correlations in the heterotrait-heteromethod block should be lower than in the monomethod-monomethod blocks. This indicates that the correlations between measures are due to the methods and not the trait being measured. Similarly, it is important to look at the heterotrait-monomethod triangles. If these are smaller than the MTMM triangles, then the results indicate that the differences between the measures are due to the traits being measured.
In conclusion, the MTMM matrix is an important tool for assessing construct validity. However, in its purest form, it requires a fully cross-validated design, which is not practical for many applied research contexts. Fortunately, newer forms of analyzing MTMM data, such as structural modeling, can help researchers use the matrix for more general purposes, including evaluation of the reliability and validity of tests. Hopefully, with continued efforts in this direction, the MTMM matrix will become an integral part of the standard methodology for psychologists who study individual differences. This would help ensure that the results of our work are attributable to the constructs we intend to measure, rather than the idiosyncrasies of our measurement techniques.