Open Access Repository for Measurement Instruments

AkkordeonDetailedScaleViews

The Hagen Matrices Test (HMT)

  • Author: Heydasch, T., Schnaedter, S.
  • In ZIS since: 2020
  • DOI: https://doi.org/10.6102/zis293_exz
  • Abstract: The Hagen Matrices Test (HMT; Heydasch, 2014) is a web-based figural matrices test primarily measuring induction, reasoning, and fluid intelligence according to the Cattell-Horn-Carroll (CHC) model ... more of intelligence (Schneider & McGrew, 2012). The test comprises 20 items and is so far available in German, English, and Spanish and usually takes less than 30 minutes. It is recommended for non-clinical adults aged between 18 and 65, preferably with above-average cognitive abilities and for group testing only.Reliability, convergent, divergent, and criterion validity have been attested for the German version. The quality criteria have so far only been checked for the German version.Example items can be found online (https://ww2.unipark.de/uc/HOT_preview/?a=zis&b=&c=preview). The HMT can be requested via e-mail to the author for non-commercial research purposes. less
  • Language Documentation: English
  • Language Items: German (source), English, Spanish
  • Number of Items: 20
  • Survey Mode: Survey Mode: Web-based
  • Processing Time: About 25 minutes: 5 minutes for the instruction and about 20 minutes to take the test.
  • Reliability: internal consistency (Kuder-Richardson-Formula 8) = .78; retest = .75
  • Validity: evidence for convergent, divergent, and criterion validity
  • Construct: fluid intelligence, reasoning, induction
  • Catchwords: intelligence, mental ability, measure, test
  • Item(s) used in Representative Survey: no
  • URL Website: https://www.fernuni-hagen.de/hot-project/
  • Status of Development: validated
  • Original Publication: https://doi.org/10.13140/RG.2.2.31433.75361
    • Instruction

      The instruction is available in English, German, and Spanish. It starts with (in English):

      “Dear participant, this test is about finding rules in abstract patterns and completing them in a logical way. Each task shows an incomplete jigsaw puzzle. The patterns you will see follow rules which may apply to a row, a column or a diagonal. They may apply to the figure as a whole or to parts of it only. They may involve addition, subtraction, the alignment of figures or single components. Only one of the eight pieces given is the correct one required to complete the design. It is your task to select the piece which completes the jigsaw puzzle. Each task needs to be completed within 2:00 minutes.”

      To ensure that the task is understood by all participants, two sample matrices are presented. Solutions as well as underlying patterns are explained.

       

      Items

      Each of the 20 items with increasing difficulty comprises 3 x 3 incomplete matrices: One of the nine parts in the matrix is missing and must be identified by recognizing the underlying rules of the given pattern.

       

      Response specifications

      Eight potential solutions to complete the matrix are presented below it. Only one solution fits to the pattern of the incomplete matrix and is, thus, the correct one.

       

      Scoring

      Correct solutions of the items are coded with 1, incorrect solutions and missed items regardless of the number are coded with 0. After completion of the test, the number of correctly answered items and the corresponding percentage are given. In addition, an IQ score and the 90% confidence interval of the IQ score are computed. To classify the personal results correctly, the following general IQ scale should be taken as a basis:

      The IQ shows you how good your test result was compared to the main population:

      IQ < 70 à result far below average

      IQ < 85 à result below average

      IQ in between 85 and 115 à average result

      IQ > 115 à result above average

      IQ > 130 à result far above average

       

      Application field

      The HMT is a web-based intelligence test. It may be applied by researchers (e.g. psychologists, economists, sociologists) to samples with average or high intelligence to survey non-clinical adults aged between 18 and 65 (ceiling effects are unlikely but floor effects may arise with frustration in samples of lower cognitive ability). The test was validated in samples with an age range of 17 to 57 years. It should be used for group comparisons and for correlative studies (sample sizes N > 50) and not for individual diagnostics. The average duration of the test is 25 minutes: 5 minutes for the instruction and about 20 minutes to take the test (M = 24.4 min, SD = 12.60 min).

       

    According to Deary (2012, p. 348), “intelligence predicts important things in life.” In particular, the relevance in job-related fields and education had been disclosed even decades ago. In an early paper, Harrell and Harrell (1945) demonstrated the association between intelligence and specific occupations. Additionally, intelligence is highly correlated with job performance and job training success, which was proven in a meta-analysis by Salgado and colleagues (2003). In another meta-analysis, Poropat (2009) had shown significant associations between intelligence and academic success.

    Intelligence must be understood as the mental ability to learn and solve problems. In other words (Gottfredson, 1997, p. 13):

    Intelligence is a very general mental capability that, among other things, involves the ability to reason, plan, solve problems, think abstractly, comprehend complex ideas, learn quickly, and learn from experience. It is not merely book learning, a narrow academic skill, or test-taking smarts. Rather, it reflects a broader and deeper capability for comprehending our surroundings “catching on”, “making sense” of things, or “figuring out” what to do.

    The Cattell-Horn-Carroll Model (Schneider & McGrew, 2012) allows us a more detailed look at the theoretical structure of intelligence. It distinguishes general intelligence (see also Carroll, 1993), as well as abilities on the broad (e.g., Reading & Writing Ability, Comprehension-Knowledge, and Fluid Reasoning) and narrow level (e.g. of Fluid Reasoning: Induction, General Sequential Reasoning, and Quantitative Reasoning).

    According to Carroll (1993), figural matrices primarily measure induction (which is the narrow level of Fluid Reasoning): The test taker’s task is “to inspect a set of materials and from this inspection induce a rule governing the materials, or a particular or common characteristic of one or more stimulus materials, such as relation or trend” (p. 211). Schneider and McGrew’s (2012) definition of induction is quite similar: It is “the ability to observe a phenomenon and discover the underlying principles or rules that determine its behavior” (p. 112). Accordingly, we conclude that the HMT is a test of induction. Induction, in turn, is defined by Schneider and McGrew (2012) as a narrow ability. Further, it is the core of the broad ability of fluid reasoning (Schneider and McGrew, 2012). Thus, the HMT is a test of induction as well as of fluid reasoning. One may now argue that there are plenty of well-designed, validated, and established tests of intelligence and the HMT would be redundant. However, this is the core point of the benefit: The aim of the HMT was to develop a web-based, economic, and valid alternative which is not very different from other tests using matrices, but: it is free of cost.

     

    If not otherwise indicated analyses were computed with SPSS (IBM).

     

    Item generation and selection

    The development of the HMT was part of the doctoral dissertation of the first author. Items were generated and selected in three pilot studies conducted from 2009 to 2011. In a fourth study the results of which are presented below the final version of the HMT was validated. The matrices were designed by using different operations: horizontal and/or vertical addition or subtraction and/or varying the positions of separate elements, i.e. rotation or movement. The original version was introduced in German. A professional translator translated the original version into English and Spanish. Because of the non-verbal tasks and existing samples without back translation or other procedures.

     

    Samples

    The sample for the validation study of the HMT was recruited in the year 2011 and consisted of 1,339 B.Sc. psychology students from the University of Hagen, a German distance learning university. 76% were female. The mean age was 32.2 years (SD = 8.97). Participants were recruited via email and the university’s online-studies webpage. They received course credit for their participation. Initially, 3,405 students registered. Several steps of data cleaning were performed. 1,384 participants abandoned the questionnaire at some point. Of the remaining 2,021, 119 participated multiple times and were excluded. Another 563 students were removed because they had taken part in at least one of the preliminary studies so that the final sample only included first-time participants. All tests were performed online except for the Intelligence Structure Test 2000 R (I-S-T 2000 R; Liepmann et al., 2007) which was administered as proctored paper-and-pencil test.

     

    Item analyses

    To determine the dimensionality of the HMT, a parallel analysis (PA; Horn, 1965; based on O’Connor, 2000) (with 9,000 data sets using principal component eigenvalues), the minimum average partial (MAP) test (Velicer et al., 2000, based on O’Connor, 2000), the comparison data (CD) technique (Ruscio & Roche, 2012, using R 2.15.1) and the scree test (Cattell, 1966) were performed. All four analyses suggested a two-component solution. Hence, a principal component analysis (PCA) with two predefined factors was conducted. The two factors explained 46.1% of the variance, 35.1% were described by the first factor. All loadings on factor 1 were greater than .30. The loadings on factor 2 decreased continually from a = .53 (item 1) to a = -.59 (item 19). The correlation of the loadings on factor 2 with item difficulty was r = .85 (p < .001) so that factor 2 was interpreted as a “spurious” difficulty factor (see McDonald & Ahlawat, 1974). Factor 1 represents reasoning, the fundamental ability needed to solve matrices.

     

    Item parameters

    Table 1 shows the item difficulties and item-total correlations. The first two items can be considered as very easy (p > .80). Seven items demonstrate a strong difficulty p < .20. The mean difficulty was M = .37 (SD = .26, Range .10 to .88). The correlation between item position and item difficulty was r = -.94 (p < .001). The item-total correlations ranged from rit = .19 to rit = .50. The mean number of correct responses was M = 7.43 (SD = 3.38).

     

    Table 1

    Item difficulty (p) and item-total correlations (rit)

    Item

     

    p

    rit

    1

     

    .88

    .30

    2

     

    .84

    .36

    3

     

    .66

    .38

    4

     

    .67

    .33

    5

     

    .65

    .45

    6

     

    .55

    .28

    7

     

    .56

    .38

    8

     

    .37

    .19

    9

     

    .25

    .34

    10

     

    .29

    .42

    11

     

    .24

    .29

    12

     

    .29

    .31

    13

     

    .21

    .38

    14

     

    .16

    .21

    15

     

    .15

    .48

    16

     

    .13

    .39

    17

     

    .17

    .27

    18

     

    .16

    .50

    19

     

    .12

    .30

    20

     

    .10

    .45

    Note. N = 1,339. The standard deviation of dichotomous item is [p(1-p)]½.

     

     

    Objectivity

    The HMT is a web-based test. Instruction, administration, scoring, and interpretation of the HMT are computer-based and thereby highly standardized. Accordingly, application, evaluation, and interpretation of the HMT can be deemed highly objective.

     

    Reliability

    The internal consistency calculated with the Kuder-Richardson Formula 8 (Kuder & Richardson, 1937) was rKR8 = .78 and the retest-reliability was rtt = .75. The mean test-retest interval was M = 78 days (SD = 123) with a range of 5 to 388 days. This indicates that the HMT is an adequate test for group-level analyses.

     

    Validity

    To attest convergent, divergent and criterion validity, several analyses were performed. To verify convergent validity, correlations with other intelligence tests were calculated. Divergent validity was assessed by correlating the HMT to measures of non-cognitive personality traits.

     

    The highest correlations with other measures of intelligence (see Table 2) were found for the reasoning ability measures from the I-S-T 2000 R (Liepmann et al., 2007). An overview of the correlations between the HMT and other intelligence measures to attest convergent validity are shown in Table 2. According to Cohen (1992) correlations of r = .10 can be considered as small, r = .30 as medium and r = .50 as large effects.

     

    Table 2

    Correlations between the HMT and intelligence measures

    Variable

       N

        R

     

    KR20

    I-S-T 2000 R

    **91

     

     

     

     

     

     

    Reasoning

     

    f.57***

     

     

    .93

     

     

     

    Verbal

     

    f.34***

     

     

    .77

     

     

     

    Numeric

     

    f.50***

     

     

    .93

     

     

     

    Figural

     

    f.51***

     

     

    .80

     

     

     

    gf

     

    f.53***

     

     

     

     

     

    Knowledge

     

    f.38***

     

     

    .85

     

     

     

    Verbal

     

    f.24*

     

     

    .69

     

     

     

    Numeric

     

    f.34***

     

     

    .65

     

     

     

    Figural

     

    f.39***

     

     

    .69

     

     

     

    gc

     

    f.30**

     

     

     

     

     

    Memory

     

    f.28**

     

     

    .82

     

    10MT

    **65

    f.45***

     

     

    .77

     

    ISI

     

     

    1332

     

     

     

     

     

     

    Vocabulary

     

    -.04

     

     

    -

     

     

    Word fluency

     

    -.08**

     

     

    -

     

     

    Numeric

     

    f.30***

     

     

    -

     

     

    Spatial

     

    f.23***

     

     

    -

     

     

    Memory

     

    -.06*

     

     

    -

     

     

    Perception speed

     

    f.01

     

     

    -

     

     

    Reasoning

     

    f.19***

     

     

    -

     

     

    Musical

     

    -.04

     

     

    -

     

     

    Physical bodily-kinesthetic

     

    -.06*

     

     

    -

     

     

    Interpersonal

     

    -.13***

     

     

    -

     

     

    Intrapersonal

     

    -.12***

     

     

    -

     

    Note. KR20 = Internal consistency according to the Kuder-Richardson Formula 20 (Kuder & Richardson, 1937); I-S-T 2000 R = Intelligence Structure Test 2000 R (Liepmann et al., 2007); 10MT = 10-Minute Test (Hilbig & Musch, 2010); ISI = Inventory of self-estimated intelligence (Rammstedt & Rammsayer, 2002); gf = fluid intelligence factor; gc = crystallized intelligence factor.

    * p < .05. ** p < .01. *** p < .001.

                       

     

     

    Correlations with personality traits appear in Table 3. Positive and negative affectivity (measured with the Positive and Negative Affect Schedule; PANAS; Krohne et al., 1996; based on Watson et al., 1988), all Big Five dimensions (measured with the Big Five Inventory; BFI; Lang et al., 2001; based on John & Srivastava, 1999), and narcissism (measured with the Narcissistic Personality Inventory; NPI; Schütz et al., 2004; based on Raskin & Hall, 1979) were not correlated with the HMT. There were some significant but small correlations with certain facets of the HEXACO Personality Inventory-Revised (HEXACO-PI-R 100; Lee & Ashton, 2018) and the Personality-Adjective Scales (PASK5; Brandstätter, 2010, 2014). All in all, these results indicate discriminant validity.

     

    Table 3

    Correlations between the HMT and personality traits)

    Variable

     N

      r

      α

    PANAS

     

    587

     

     

     

    Positive Affectivity

     

    f.01

    .87

     

    Negative Affectivity

     

    -.06

    .88

    BFI

     

    406

     

     

     

    Extraversion

     

    f.02

    .88

     

    Agreeableness

     

    f.05

    .79

     

    Conscientiousness

     

    f.00

    .85

     

    Neuroticism

     

    -.04

    .89

     

    Openness

     

    -.08

    .83

    HEXACO-PI-R 100

    694

     

     

     

    Honesty-Humility

     

    f.01

    .82

     

     

    Sincerity

     

    f.00

    .70

     

     

    Fairness

     

    f.01

    .76

     

     

    Greed Avoidance

     

    f.02

    .79

     

     

    Modesty

     

    f.01

    .67

     

    Emotionality

     

    -.09*

    .80

     

     

    Fearfulness

     

    -.12**

    .64

     

     

    Anxiety

     

    -.03

    .70

     

     

    Dependence

     

    -.06

    .71

     

     

    Sentimentality

     

    -.05

    .69

     

    Extraversion

     

    -.05

    .85

     

     

    Social Self-Esteem

     

    f.01

    .70

     

     

    Social Boldness

     

    -.03

    .68

     

     

    Sociability

     

    -.11**

    .66

     

     

    Liveliness

     

    -.04

    .74

     

    Agreeableness

     

    f.00

    .83

     

     

    Forgivingness

     

    f.01

    .71

     

     

    Gentleness

     

    f.00

    .63

     

     

    Flexibility

     

    -.06

    .50

     

     

    Patience

     

    f.03

    .73

     

    Conscientiousness

     

    -.02

    .79

     

     

    Organization

     

    -.07

    .67

     

     

    Diligence

     

    -.04

    .70

     

     

    Perfectionism

     

    f.06

    .66

     

     

    Prudence

     

    -.02

    .57

     

    Openness to Experience

     

    f.04

    .75

     

     

    Aesthetic Appreciation

     

    -.06

    .63

     

     

    Inquisitiveness

     

    f.11**

    .65

     

     

    Creativity

     

    f.00

    .55

     

     

    Unconventionality

     

    f.08*

    .42

     

     

    (Altruism)

     

    -.08*

    .57

    PASK5

     

    505

     

     

     

    A

    Warmth

     

    -.10*

    .63

     

    B

    Reasoning

     

    f.21***

    .56

     

    C

    Emotional stability

     

    f.04

    .81

     

    E

    Dominance

     

    -.02

    .52

     

    F

    Liveliness

     

    -.05

    .59

     

    G

    Rule-consciousness

     

    f.01

    .47

     

    H

    Social boldness

     

    f.00

    .72

     

    I

    Sensitivity

     

    -.02

    .70

     

    L

    Vigilance

     

    f.00

    .47

     

    M

    Abstractedness

     

    f.01

    .56

     

    N

    Privateness

     

    f.00

    .12

     

    O

    Apprehension

     

    -.06

    .70

     

    Q1

    Openness to change

     

    f.12**

    .74

     

    Q2

    Self-reliance

     

    f.03

    .53

     

    Q3

    Perfectionism

     

    f.02

    .69

     

    Q4

    Tension

     

    -.07

    .77

    NPI

    576

    -.02

    .83

    Note. PANAS = Positive and Negative Affect Schedule (Krohne et al., 1996); BFI = Big Five Inventory (Lang et al., 2001); HEXACO-PI-R 100 = 100-item HEXACO Personality Inventory-Revised (Lee & Ashton, 2018); PASK5 = Personality-Adjective Scales (Brandstätter, 2010, 2014); NPI = Narcissistic Personality Inventory (Schütz et al., 2004).

    * p < .05. ** p < .01. *** p < .001.

                   

     

     

    Besides convergent and divergent validity, criterion validity was investigated by calculating correlations with high school grades and the grade point average. Additionally, the correlation with the statistics grade in the B. Sc. Psychology studies was computed. Results are shown in Table 4. The HMT correlated with study grades, high school school-leaving qualification, and high-school grades in mathematics and biology. This indicated criterion validity.

     

    Table 4

    Correlations between the HMT and indicators of academic achievement

    Variable

      N

    r

     

    High schoola

     

     

     

     

     

    SLQb

    637 (118)

     

    f.15***

     

     

    GPAc

    645 (118)

     

    f.19***

    (.34***)

     

    Mathematicsc

    641 (118)

     

    f.27***

    (.45***)

     

    Englishc

    639 (118)

     

    f.07

    (.21*)

     

    Germanc

    641 (118)

     

    f.00

    (.08)

     

    Biologyc

    626 (114)

     

    f.12**

    (.35***)

     

    Artsc

    610 (113)

     

    -.01

    (.12)

    B.Sc. Psychology

     

     

     

     

     

    GPAc

    255

     

    f.25***

     

     

    Statisticsc

    140

     

    f.36***

     

    Note. GPA: Grade point average. SLQ: School-leaving qualification. Results in parentheses were computed on a homogenous subsample of participants younger than 24 years (M = 21.67, SD = 1.14) who all had the same school-leaving qualification (Abitur). a Participants who did not have a German high-school degree were excluded because of the diverse international coding of degrees. bSpearman (1904) correlations. cGrades were recoded so that positive correlations indicate that higher HMT scores occurred with better grades.

    * p <.05. ** p < .01. *** p < .001.

     

     

    Descriptive statistics

    N = 1,339 participants took the test, the mean number of correctly answered matrices was M = 7.43 with a standard deviation of SD = 3.38.

     

    Further quality criteria

    The HMT is an economic test because it is relatively short with a mean duration of less than half an hour. It is web-based and free of cost so that it is a useful tool for group analyses especially in the context of psychological research. Most items are quite difficult, so that it is recommended to use the HMT only with samples of at least average, or even better, above-average intelligence to avoid frustration.

    Men (M = 8.37, SD = 4.26, N = 347) solved approximately one more matrix than women (M = 7.11, SD = 3.38) with a difference of MΔ = 1.26 (d = 0.34, t = 4.98, df = 507.45, p < .001, N = 987). Other studies on sex differences in intelligence tests measuring induction with figural matrices tests found similar results (Irwing & Lynn, 2005).

    In addition to gender effects, age effects were detected as well. The association was r = -.12 (p < .001, N = 1,333) indicating that younger participants solved more items correctly. Both, gender and age effects can be considered substantial but not unusual for that kind of test.

     

     

    Further literature

    Heydasch, T., Haubrich, J. & Renner, K.-H. (2020). The Short Version of the Hagen Matrices Test (HMT-S). A 6-Item Induction Intelligence Test (T. Heydasch, Trans.). methods, data, analyses, 7, 183e-205e. https://doi.org/10.12758/mda.2013.021

    Dr. Timo Heydasch, University of Hagen, Department of Work and Organizational Psychology, Universitaetsstr. 33, 58097 Hagen, Germany, E-Mail: Timo.Heydasch@fernuni-hagen.de