Author(s): Kuncel, Nathan R., Sarah A Hezlett & Deniz S. Ones
Published: 2004 in Journal of Personality and Social Psychology
URL to article
Research Focus Area: Grading systems that work and are meaningful in the 21st century especially for colleges and employeers, and for topics like PE, arts, and health
Abstract:
How is it that many people believe that the abilities required for success in the real world differ substantially from what is needed to achieve success in the classroom? Perhaps the fact that tests and measures are often developed for particular settings (e.g., educational vs. occupational) has perpetuated this myth. The main purpose of the current study is to evaluate whether a single test of cognitive ability that was developed for use in educational settings is predictive of behaviors, performances, and outcomes in both educational and occupational settings. We first conduct a series of meta-analyses to establish that the Miller Analogies Test (MAT; Miller, 1960) assesses cognitive ability. We then report meta-analyses examining the validity of the MAT for predicting multiple criteria in academic and work settings, including evaluations of career potential and creativity. The results address the theoretical question of whether a single cognitive ability measure is valid for predicting important criteria across domains. In this article, general cognitive ability and g are defined as the underlying trait that leads to the well-documented positive intercorrelation observed between measures of cognitive behaviors. The phenomenon of g has been shown to have important, domain-general relationships with knowledge, learning, and information processing, and the general thesis of this article is that tests of general cognitive ability or g are predictive of success in academic and work settings, regardless of the setting for which they were developed. Although our thesis and findings may surprise some readers, it was our a priori expectation that the MAT would be a valid predictor of a wide range of academic and work criteria, as well as creativity and career potential. Our prediction was based on the enormous literature that unequivocally demonstrates the existence of a general factor of cognitive ability and its broad importance as a predictor of numerous life outcomes (for reviews, see, Brand, 1987; Gottfredson, 2002). Therefore, this study builds on and contributes to the substantial body of research already supporting the nomological network in which the construct of g is embedded.
Research Question(s):
Is the Miller Analogies Test a valid predictor of a wide range of academic and work criteria? Is it a valid predictor of creativity and career potential? Are the abilities needed in academic settings applicable to work settings as well?
Methods:
Literature Review, meta analysis
Setting:
graduate school admission and moderate to high complexity industry jobs in the united states
Key Findings:
- Any test that assesses g will have predictive value in both academic and work settings, regardless of the setting for which they were developed
- Existing evidence supports the existence of a single pervasive general factor in cognitive ability (as opposed to a number of distinct truly separate abilities)
- g, a measure of general cognitive ability,
- is defined as the underlying trait that leads to positive intercorrelation between measures of cognitive behaviors
- has been shown to have important domain-general relationships with knowledge, learning and information processing
- g holds a high level of cross-domain predictive value, meaning it is positively correlated with learning, affecting not only classroom performance, but job performance as well
- existing literature demonstrates that the validity of g for predicting job performance is moderated by the complexity of the job
- the researchers propose that the nature and determinants of academic performance are similar, although not fully identical, to the nature and determinants of job performance
- The Miller Analogies Test
- Is a timed 100-item test composed entirely of analogy items dependent on the test taker’s ability to reason with the sciences, vocabulary, literature, arts, and history
- Has been used since 1926 for admissions decisions into graduate schools as well as hiring/promotion decisions for moderate- to high-complexity jobs in industry
- has a generally positive modest relationship with degree attainment (though the validity is likely to be moderated by other factors)
- is a generalizably valid predictor of work performance criteria, such as:
- job performance
- counseling performance
- educational administration performance
- membership in a professional organization
- as well as graduate student performance, potential, and creativity
- the validity was at least as high for work criteria as for school criteria
- The MAT was a valid predictor of seven of the eight measures of graduate student performance, five of the six school-to-work transition performance criteria, and all four of the work performance criteria
- At the same time, however, the pattern of correlation between the MAT and individual criteria also supports the importance of more specific abilities
Implications:
- Findings of this study refute the common misconception that intelligence at work is completely different from intelligence at school. Instead, performance should be understood as a function of the motivated application of acquired declarative (i.e., job knowledge) and procedural knowledge (i.e., skill)
Limitations:
- Due to homogeneity of the groups studied the estimates obtained in this study are underestimates of the relationship between g and performance if the full range of talent in the U.S. population had been considered.
- the analyses are based on sample sizes that range from rather limited to very large