Open Access Repository for Measurement Instruments

AkkordeonDetailedScaleViews

CCS-Based 7-Item Traditionalism Scale (CCS-T-7)

  • Author: Partsch, M. V., Danner, D., & Chernyshenko, O.
  • In ZIS since: 2018
  • DOI: https://doi.org/10.6102/zis261
  • Abstract: The CCS-Based 7-Item Traditionalism Scale (CCS-T-7) conceptualizes and measures traditionalism as facet of Conscientiousness. The CCS-T-7 has been used and evaluated in online surveys. The items have ... morebeen developed in English and translated into German, French, Spanish, Polish, and Japanese. The scale allows a mostly reliable and valid measurement in the six countries. Furthermore, the RMSEA suggests metric measurement invariance for the scale. less
  • Language Documentation: English
  • Language Items: English (source), German, French, Spanish, Polish, Japanese
  • Number of Items: 7
  • Reliability: Mc Donald’s omega = .63 to .79; Cronbach’s Alpha = .62 to .79
  • Validity: evidence for convergent construct validity
  • Construct: traditionalism, conscientiousness
  • Catchwords: traditionalism, conservatism
  • Item(s) used in Representative Survey: yes
  • Status of Development: tried, standardized
    • Instruction

      Please rate how much you agree with each of the following sentences.

       

      Items

      Table 1

      Items of the CCS-Based 7-Item Traditionalism scale

      No.

      Item

      Polarity

      1

      I have the highest respect for authorities and assist them whenever I can.

      +

      2

      Even if I knew how to get around the rules without breaking them, I would not do it.

      +

      3

      I believe that people should be allowed to take drugs, as long as it doesn’t affect others.

      -

      4

      I support long-established rules and traditions.

      +

      5

      People who resist authority should be severely punished.

      +

      6

      In my opinion, all laws should be strictly enforced.

      +

      7

      When working with others I am the one who makes sure that rules are observed.

      +

       

      The items in German, French, Spanish, Polish and Japanese are available in Appendix 1.

       

      Response specifications

      Each of the 7 items was presented with the five response categories “strongly disagree,” “disagree,” “neither agree nor disagree,” “agree,” and “strongly agree.”  

       

      Scoring

      All items are answered on a five-point scale ranging from 1 = strongly disagree to 5 = strongly agree. Item 3 is reverse-keyed and thus has to be recoded (6 – raw score). The traditionalism scale score is computed as the unweighted mean or sum score of all 7 items. For the handling of missing responses no general standards exist. As a crude rule of thumb we suggest to aggregate individual data to the mean or sum score only if not more than 20% of item answers (k = 1) are missing, assuming that factor loadings in the sample at hand are homogenous.

       

      Application field

      The CCS-T-7 is an economical instrument suitable for measuring traditionalism in large-scale assessment. It was designed for use in written questionnaires and applied in a web-based format. The scale can be used to assess traditionalism in adults of all ages and levels of educational attainment in the USA, Germany, France, Spain, Poland, and Japan.

    The CCS-T-7 conceptualizes traditionalism as a facet of Conscientiousness. In a study aimed at developing a comprehensive lower-order taxonomy of Conscientiousness, Roberts, Chernyshenko, Stark, and Goldberg (2005) identified traditionalism as one of six facets of this personality dimension. People who score high on traditionalism "tend to comply with current rules, customs, norms, and expectations; they dislike changes and do not challenge authority" (Roberts et al., 2005, p. 122). Integrating traditionalism into the Big Five framework, these authors found that it correlated not only with Big Five Conscientiousness but also (negatively) with Big Five Openness to Experience.

     

     

    Item generation and selection

    The items of the scale were drawn from the Chernyshenko Conscientiousness Scale (CSS; Chernyshenko, 2003; Hill & Roberts, 2011). Based on Roberts et al.'s (2005) comprehensive taxonomy of conscientiousness, Chernyshenko (2003) developed a pool of traditionalism items from which Hill and Roberts (2011) derived a 10-item traditionalism scale. We in turn obtained the CCS-T-7 scale from Hill and Roberts’ (2011) 10-item scale by dropping two items to simplify the scale for low-literacy adults and dropping a further item to achieve an acceptable model fit of a unidimensional measurement model. The items of the scale were then translated from English into German, French, Spanish, Polish, and Japanese based on the multistep team translation approach TRAPD (Harkness, 2003). In each country, two independent professional translators translated the scale from English into the target language. Then an expert on the traditionalism construct reviewed those translations and finalized them in exchange with the professional translators. Scale development and translation were coordinated by an OECD expert group on non-cognitive skills.

     

    Samples

    The scale was administered in two online surveys conducted by the OECD: First in the USA and then in Germany, France, Spain, Poland, and Japan. The initially collected samples were quota samples based on gender, age, and education according to the country-specific distributions. Table 2 shows sample sizes and sociodemographic characteristics after data cleaning and case exclusion.

    The data doesn’t comprise any missing values on the CCS-T-7 items, the sociodemographic variables, and the BFI-2 scales used for construct validation of the scale. It comprised very few missing values on the cognitive ability scale used for further scale validation. To deal with those missing values we applied list-wise deletion (see Table 5).

     


     

    Table 2

    Sample characteristics

     

    Country

     

     

    USA

    Germany

    France

    Spain

    Poland

    Japan

    Sample Size

    1190

    1538

    1324

    1384

    1455

    1243

    Gender [% Female]

    58.2

    49.4

    57.0

    50.3

    51.3

    51.4

    Age [M(SD); [Range]]

    41.7(13.2);

    [16 – 65]

    43.8(12.5);

    [18 – 65]

    42.3(13.1);

    [16 – 65]

    41.7(11.9);

    [18 – 65]

    40.2(13.0);

    [18 – 65]

    43.9(12.4);

    [18 – 65]

    Educational Level

     

     

     

     

     

     

    Primarya (%)

    0.8

    0.7

    1.1

    2.2

    3.5

    0.0

    Secondaryb (%)

    24.0

    52.7

    55.5

    25.4

    44.1

    36.0

    Post-Secondaryc (%)

    37.0

    13.1

    32.4

    27.2

    11.1

    20.3

    Tertiaryd (%)

    38.2

    33.5

    11.0

    45.2

    41.4

    43.7

    Note. Educational levels are derived from ISCED 2011: a Less than primary or primary education (ISCED 0+1); b Lower secondary or upper secondary education (ISCED 2+3); c Post-secondary non- tertiary or short-cycle tertiary education (ISCED 4+5); d Bachelor, Master, Doctoral or equivalent education (ISCED 6+7+8).

                     

     

     

    Item analyses

    We investigated the factorial structure of the items using multi-group confirmatory factor analysis (CFA). Because of the big sample size we used ML as estimator but checked that fit indices and standard errors estimated with MLR didn’t deviate severely. We used a unidimensional measurement model developed for the United States specifying one general traditionalism factor. In all countries, factor loadings and item intercepts were freely estimated, whereas the variance of the latent traditionalism factor was set to 1. The structure of the measurement model is sketched in Figure 1 below. While the chi-square test proved significant c2(84) = 755.384, p ≤ .001, more suitable fit indices indicated an acceptable to good fit of the multi-group model, with RMSEA = .077, CFI = .931, and SRMR = .036 (Hu & Bentler, 1999; Schermelleh-Engel, Moosbrugger, & Müller, 2003; Schweizer, 2010). The model parameters are shown in Table 3.

     

     

    Figure 1. Measurement model for the traditonalism construct. Factor loadings were estimated freely, whereas the variance of the latent traditionalism factor was set to one. Error terms are not displayed.

     

    Item parameter

    Table 3 shows the standardized factor loadings and the unstandardized item intercepts of the multi-group structural equation model. The squared standardized factor loading indicates the item variance explained by the latent factor. An unstandardized item intercept indicates the difficulty level of an item (the lower the intercept the more difficult the item). On average, the standardized factor loading was .52 which suggests that the items reflect the latent traditionalism variable well. Most factor loadings were >.30. The third item (“I believe that people should be allowed to take drugs, as long as it doesn’t affect others”) revealed loadings >.30 in Germany and Spain, but smaller loadings especially in Poland and Japan. This suggests that this item is less associated with traditionalism in these countries. Alternatively, the reverse-keying might have provoked a slightly different response behavior for this item than for the other six items.

     


     

    Table 3

    Standardized factor loadings and unstandardized item intercepts of the six countries

     

     

    Loadings

    on traditionalism factor

     

    Item intercepts

    Item

     

    US

    DE

    FR

    ES

    PL

    JP

     

    US

    DE

    FR

    ES

    PL

    JP

    1

     

    .63

    .51

    .58

    .55

    .53

    .54

     

    3.90

    3.04

    3.58

    3.72

    2.77

    2.93

    2

     

    .52

    .41

    .50

    .34

    .55

    .39

     

    3.60

    3.14

    3.30

    3.28

    3.15

    3.19

    3

     

    .35

    .21

    .35

    .24

    .14

    .10

     

    3.56

    3.58

    3.64

    3.54

    3.45

    4.13

    4

     

    .69

    .53

    .59

    .64

    .41

    .59

     

    3.70

    3.47

    3.52

    3.49

    3.47

    3.20

    5

     

    .64

    .69

    .66

    .67

    .42

    .50

     

    3.04

    3.12

    3.34

    3.25

    2.37

    2.76

    6

     

    .78

    .72

    .74

    .76

    .57

    .59

     

    3.32

    3.46

    3.45

    3.28

    3.31

    3.13

    7

     

    .59

    .52

    .58

    .41

    .54

    .54

     

    3.42

    3.29

    3.49

    3.11

    3.32

    3.01

    Note. US = United States of America (N = 1190), DE = Germany (N = 1538), FR = France (N = 1324),

    ES = Spain (N = 1384), PL = Poland (N = 1455), JP = Japan (N = 1243).

     

    Objectivity

    The scale is applied in a self-administered online assessment. The written instructions, the scoring rules, and the reference statistics ensure the objectivity of the measurement.

     

    Reliability

    We calculated Cronbach’s alpha and McDonald’s omega (McDonald, 1999) to assess the reliability of the manifest scale score (mean or sum score). Cronbach’s alpha is reported because it is commonly used, even though it is an unbiased reliability estimate only for at least essentially τ-equivalent measurement models in which all factor loadings are restricted to be the same. McDonald’s omega is reported because it is more appropriate for τ-congeneric measurement models that may show a heterogeneous structure of factor loadings. The coefficients displayed in Table 4 are >.70 in the USA, Germany, France, and Spain which suggests sufficient reliability of the manifest scale score for group studies (Danner, 2015). In Poland and Japan, Cronbach’s alpha and McDonald’s omega of the mean or sum score are >.60 which is described as sufficient for exploratory research (Hair, Black, Babin, Anderson, & Tatham, 2006).


     

    Table 4

    Cronbach’s alpha and McDonald’s omega of traditionalism mean or sum score

     

    USA

    (N = 1190)

    Germany

    (N = 1538)

    France

    (N = 1324)

    Spain

    (N = 1384)

    Poland

    (N = 1455)

    Japan

    (N = 1243)

    Cronbach’s alpha

    .79

    .70

    .76

    .70

    .62

    .63

    McDonald’s omega

    .79

    .70

    .77

    .71

    .63

    .64

     

     

    Validity

    To examine the convergent construct validity of the traditionalism scale, we calculated correlations (based on the manifest scale scores) between traditionalism and the Big Five Inventory 2 (BFI-2; Soto & John, 2017), which measures the Big Five personality domains Extraversion, Agreeableness, Conscientiousness, Negative Emotionality, and Open-Mindedness. The BFI-2 items were translated from English into the other five languages together with the CCS-T-7 items. Furthermore, we explored the correlations between the traditionalism scale and age, educational level, and cognitive abilities (ICAR verbal reasoning test; The International Cognitive Ability Resource Team, 2014), which were also included in the OECD surveys. The data collection within the OECD surveys didn’t allow for a hypothesis-guided investigation of the scale’s discriminant validity or criterion validity, which should be caught up on in a study with a more flexible data collection. The correlations are shown in Table 5. According to the guideline of Gignac and Szodorai (2016) we considered correlations of .10 as small, .20 as typical and .30 as relatively large.

     


     

    Table 5

    Manifest correlations of traditionalism scale with Big Five domains and further variables

     

    US

    DE

    FR

    ES

    PL

    JP

    Extraversion

    .23***

    .12***

    .15***

    .23***

    .15***

    .07*

    Agreeableness

    .31***

    .23***

    .25***

    .23***

    .29***

    .32***

    Conscientiousness

    .37***

    .29***

    .39***

    .34***

    .33***

    .21***

    Negative Emotionality

    -.24***

    -.14***

    -.12***

    -.13***

    -.14***

    -.02

    Open-Mindedness

    -.06*

    .04

    -.04

    .06*

    .10***

    .13***

    Age

    .07*

    .04

    .18***

    .06*

    .14***

    .02

    Level of Education

    -.04

    -.05*

    -.05*

    .01

    .00

    .00

    Cognitive Ability

    -.26***

    -.07**

    -.15***

    -.07**

    -.11***

    .02

    Note. US = United States of America (N = 1190), DE = Germany (N = 1538; different N for correlation with Cognitive Ability NCogA = 1536), FR = France (N = 1324; NCogA = 1323), ES = Spain (N = 1384), PL = Poland (N = 1455; NCogA = 1452), JP = Japan (N = 1243; NCogA = 1242). Kendall’s tau was computed for Level of Education; Pearson correlations were computed for all other variables. * p < .05, ** p < .01, *** p < .001.

     

    As expected, there were large correlations between the traditionalism scale and the Conscientiousness scale in most countries. Only in Japan, the correlation was only typical. There were also typical to large correlations between traditionalism and Agreeableness in all countries. The BFI-2 Open-mindedness scale displayed null correlations or small correlations with the traditionalism scale across countries. Traditionalism was conceptualized as a lower order facet of Conscientiousness. Thus, the positive association of the CCS-T-7 with the BFI-2 Conscientiousness scale supports the convergent validity of the measure. The positive association between the CCS-T-7 and BFI-2 Agreeableness might be mainly due to the Agreeableness facet of Respectfulness. The null correlations or small and partly positive correlations between the CCS-T-7 and BFI-2 Open-Mindedness are not in line with earlier findings (e.g. Roberts et al., 2005), but can be explained by the item selection of the CCS-T-7 that operationalizes traditionalism mainly through rule-obeying and the respecting of authorities. Hence, scoring high on the CCS-T-7 might not preclude also scoring high on the BFI-2 Open-Mindedness facets of Aesthetic Sensitivity, Intellectual Curiosity and Creative Imagination.

    We also found a typical to large negative correlation between the CCS-T-7 and cognitive abilities in the USA. In Germany, France, Spain, and Poland there were small negative correlations between traditionalism and cognitive ability.

     

    Descriptive statistics (scaling)

    Table 6 shows reference statistics of the traditionalism scale for each of the investigated countries. As can be seen, the scale scores were approximately normally distributed around the scale midpoint.

     

    Table 6

    Reference values for each country

    Country

    N

    M

    SD

    Skewness

    Kurtosis

    USA

    1190

    3.51

    0.68

    -0.32

    3.22

    Germany

    1538

    3.30

    0.59

    -0.22

    3.94

    France

    1324

    3.47

    0.65

    -0.25

    3.36

    Spain

    1384

    3.38

    0.57

    -0.07

    3.34

    Poland

    1455

    3.12

    0.53

    -0.22

    3.57

    Japan

    1243

    3.19

    0.47

    -0.28

    4.51

     

     

    Further quality criteria

    We investigated the measurement invariance of the scale across the US, German, French, Spanish, Polish, and Japanese sample. We specified and compared configural, metric, and scalar invariance with multi-group structural equation models. In the configural model, we specified the same factor structure for each country, which suggests that the basic structure of the construct is equivalent across countries. In the metric model, we additionally restricted factor loadings to be identical in all countries, which suggests that the latent factors have the same meaning across countries. In the scalar model, we additionally restricted item intercepts to be identical in all countries, which suggests that the difficulty of the items is identical across countries, too. We evaluated measurement invariance based on differences in the χ2, CFI, RMSEA, and SRMR between the four models. According to Chen (2007), a difference of ΔRMSEA ≤ .015, ΔCFI ≤ .010, and ΔSRMR ≤ .030 between the configural model and the metric model suggests metric invariance and a difference of ΔRMSEA ≤ .015, ΔCFI ≤ .010, and ΔSRMR ≤ .010 between the metric model and the scalar model suggests scalar invariance. Table 7 shows the fit indices of the configural, metric, and scalar traditionalism models and the level of measurement invariance they each imply. Moreover, the results of the chi-square difference tests between the models are displayed.

     

    Table 7

    Model fit for the different levels of measurement invariance across the six countries investigated

     

    Δχ2 (df)

    CFI

    RMSEA

    SRMR

    Configural invariance

    -

    .931

    .077

    .036

    Metric invariance

    206.278 (30) ***

    .913

    .074

    .051

    Scalar invariance

    2964.735 (30) ***

    .614

    .139

    .112

    Invariance level implied by each fit index

    Configural

    Configural

    Metric

    Metric

    Note. *** p < .001.

     

    As shown in Table 7, the root mean square error of approximation (RMSEA) and the standardized root mean square residual (SRMR) imply metric invariance of the traditionalism scale, whereas the comparative fit index (CFI) and the chi-square difference tests imply only configural invariance. The chi-square difference test is usually considered as too strict, which is why we give less weight to its results in the determination of the appropriate level of measurement invariance. Unlike the RMSEA and the SRMR, the CFI computes the model fit based on a comparison of the specified model with a model assuming no intercorrelation between the items – the so-called independence model. The better the fit of the independence model, the more the CFI declines with the level of invariance.  As shown above in the section Item analyses CFI and RMSEA imply only acceptable model fit for the configural multi-group model specifying a unidimensional measurement model. This implies a certain degree of independence between the items. Thus, the strong decline in CFI from the configural model to the metric model is due to the limited fit of the unidimensional measurement model. RMSEA however even slightly improves from the configural model to the metric model implying that a metric model fits the data not substantially worse than a configural model. Therefore, we decided to give less weight to the CFI and to accept metric measurement invariance for the traditionalism scale. Metric measurement invariance allows for the cross-national comparison of the strength of correlations between the latent variable and other variables (e.g., correlations with an external criterion) but not of (1) the latent group means and (2) the manifest group means (as reported in Table 6).

     

     

     

    Melanie Partsch, GESIS, PO Box 12 21 55, 68072 Mannheim, E-Mail: Melanie.Partsch@gesis.org .