Open Access Repository for Measurement Instruments

AkkordeonDetailedScaleViews

The Perceived Political Self-Efficacy (P-PSE) Scale

  • Author: Bromme, L., Rothmund, T., & Caprara, G. V.
  • In ZIS since: 2021
  • DOI: https://doi.org/10.6102/zis309_exz
  • Abstract: Caprara et al. (2009) criticized existing measures of internal political efficacy for not taking into ac-count psychological theories of self-efficacy and for the resulting low construct validity. As ... morean alter-native, they presented a ten-item measure called Perceived Political Self-Efficacy (P-PSE) Scale. Based on social cognitive theory, it adopts a psychological understanding of self-efficacy and cap-tures the phenomenon in a more systematic and complete manner than previous measures of inter-nal efficacy. We translated the P-PSE scale to German and tested it in a German national quota sample, using quotas for age, gender and education (N = 1025). We provided evidence on the scale’s construct validity (by testing its correlations towards related constructs) and on its criterion validity (by regressing political participation propensity on the P-PSE score). The scale explained ΔR2 = 26% of people’s propensity for political participation over and above sociodemographic varia-bles, and ΔR2 = 12% over and above previously existing measures, demonstrating its incremental value. We also tested cross-cultural measurement invariance towards an Italian sample, establishing configural, as well as partial metric and scalar invariance. In addition, we validated a four-item short version of the scale, which proved to be similarly valid as the full version. We argue, that these two measurement instruments provide a more adequate way of assessing internal political efficacy for research in German-speaking countries. less
  • Language Documentation: English
  • Language Items: English (source), German, Italian
  • Number of Items: 10
  • Survey Mode: CASI (Germany), PAPI (Italy)
  • Processing Time: 60 seconds (long form); 30 seconds (short form)
  • Reliability: (long form: 10 items) McDonald’s omega = .91; (short form: 4 items) McDonald’s omega = .84
  • Validity: evidence for construct and criterion validity
  • Construct: Internal political efficacy
  • Catchwords: Internal political efficacy, Self-efficacy, Social cognitive theory, Political participation, Translation, Construct validity
  • Item(s) used in Representative Survey: no
  • Status of Development: validated
  • Original Publication:

    https://doi.org/10.1186/s42409-020-00013-4

    • Instruction

      German: “Nun geht es um die Fähigkeiten, die nötig sind, um politische Handlungen auszuführen. Ich fühle mich in der Lage...“

      Italian: “Di seguito sono elencate alcune situazioni nelle quali un cittadino può trovarsi nell’esercizio dei suoi diritti di partecipazione all’attività politica. La preghiamo di valutare quanto lei ritiene di essere capace di affrontare queste diverse situazioni. Risponda, utilizzando la scala riportata di seguito.”

       

      Items

      Table 1

      Items of the German and Italian version of the Perceived Political Self-Efficacy (P-PSE) Scale

      No.

      German

      Italien

      1

      Meine eigene politische Meinung offen zu bekunden, auch in einem offensichtlich feindseligen Umfeld.

      Dichiarare la sua opinione politica apertamente, anche nei contesti decisamente ostili

      2

      Mich zu vergewissern, dass die politischen Repräsentanten, für die ich gestimmt habe, ihre Wahlversprechen halten.

      Fare sì che i rappresentanti politici per i quali ha votato onorino gli impegni presi con l’elettorato

      3

      Werbung für politische Bewegungen zu machen, die ich gut finde.

      Farsi promotore di iniziative pubbliche a sostegno dei programmi politici che lei ritiene giusti

      4

      Persönlichen Kontakt mit Abgeordneten oder Mitarbeitern der Regierung aufzunehmen und zu pflegen.

      Intrattenere rapporti personali con rappresentanti degli organi di governo nazionale

      5

      Die Wahl von Anführern einer politischen Bewegung entscheidend zu beeinflussen.

      Svolgere un ruolo decisivo o comunque rilevante nella scelta dei dirigenti dei movimenti politici ai quali si sente vicino o fa parte

      6

      Eine effektive Öffentlichkeitskampagne für eine politische Bewegung durchzuführen, mit deren Zielen oder Überzeugungen ich übereinstimme

      Svolgere un’efficace azione di propaganda per il movimento o lo schieramento politico di cui condivide le ispirazioni e i programmi

      7

      Aktiv Wahlwerbung für politische Kandidaten zu machen, denen ich vertraue.

      Contribuire attivamente all’elezione dei candidati politici in cui ripone la sua fiducia

      8

      Freunde oder Bekannte erfolgreich zu informieren und zu mobilisieren, um ein politisches Programm zu unterstützen, von dem ich überzeugt bin.

      Promuovere azioni efficaci di informazione e mobilitazione negli ambiti in cui opera (lavoro, amici, famiglia), a sostegno dei programmi politici in cui crede

      9

      Eine bedeutsame Menge Geld zu sammeln, um damit eine politische Bewegung zu unterstützen.

      Raccogliere consistenti somme di denaro a sostegno delle attività del movimento politico del quale fa parte

      10

      Die Mittel zu nutzen, die mir als Bürger zur Verfügung stehen, um das Tun der politischen Vertreter kritisch zu überwachen

      Utilizzare i mezzi che lei possiede come cittadino per vigilare criticamente sull’operato dei suoi rappresentanti politici

      Note. The short version consists of the Items 3, 4, 8, and 10

       

      Response specifications

      The P-PSE items were administered using a five-point Likert agreement scale with only the extreme categories labelled (German: stimme überhaupt nicht zu (1) and stimme vollkommen zu (5); Italian: per nulla capace (1) and del tutto capace (5)).

       

      Scoring

      The scale-level P-PSE score is the arithmetic mean of the ten (or four) single-item scores. In cases of item non-response, the arithmetic mean might be distorted due to differences in item difficulties. We thus suggest the use of imputation, listwise exclusion, or full information maximum likelihood estimation (in structural equation modelling) to deal with item non-response.

       

      Application field

      Internal political efficacy refers to “an individual’s perception of her/his abilities to execute political actions […]” (Sohl, 2014, p. 42). Research on it has not yet come to a consensus about how to measure the construct - many of the previous measures have been criticized for several reasons (e.g., Bandura, 1997, pp. 483–484; Morrell, 2003, p. 595). Largely neglected by scholars studying political efficacy, SCT (Bandura, 1991) has offered a psychological and systematic perspective on self-efficacy beliefs. Based on this theory, Caprara et al. (2009) created the P-PSE scale as a new measure of internal efficacy, which - constructed in terms of capabilities related to relevant participation behavior - offers an arguably more content valid alternative to the established measures of internal efficacy. We translated and validated the scale for the use in German samples via an online survey. Analogous to Caprara et al. (2009), the results confirm the reliability and construct validity of the translated scale. Analyses of measurement invariance revealed that the translated scale yields the same factorial structure, as well as partial metric and scalar invariance compared to the original scale by Caprara et al. (2009). Regarding the most important external criterion - political participation propensity - the scale surpasses the established internal efficacy measures, thereby attesting to its potential value for the study of political behavior. In addition, a four-item short version of the scale resulted in similar results - though with a small decrease in internal consistency - and hence offers an economical alternative especially suited for the application in large surveys.

    Political efficacy is defined as “an individual’s perceived ability to participate in and influence the political system” (Yeich & Levine, 1994, p. 259). It is usually conceptualized as two-dimensional, with interrelated but distinct dimensions (Balch, 1974): External political efficacy refers to the belief that the political system is responsive to citizens’ demands (Balch, 1974, p. 24); internal political efficacy refers to “an individual’s perception of her/his abilities to execute political actions […]” (Sohl, 2014, p. 42). Latter has been argued to be an important psychological predictor of political behavior (Bandura, 1997; Campbell et al., 1954, p. 187). Empirical research supports this prediction in regard to different forms of political behavior, such as participation in elections (Gallego & Oberski, 2012, p. 437), political protest (Chang & Chyi, 2009), and other forms of political action (e.g., Krampen, 1990; Vecchione & Caprara, 2009). Assumptions about the psychological underpinnings of self-efficacy beliefs can be drawn from the social cognitive theory (SCT) by Bandura (1991). The theory assumes, that people are capable of exercising control over their actions through self-reflection and self-regulation.

    In a review of definitions, Sohl (2014) illustrated how the exact meaning of the concept of internal efficacy differs between studies. She identified three different components that are regularly used to define internal efficacy: (1) the perception “that one can exert influence (affect political outcomes)” (p. 36-37), (2) the “perceived ability to […] execute political actions” (p. 37), and (3) a perception of “understanding politics/ the political system” (p. 37). We argue, that a focus on the second component (i.e., internal efficacy as a perceived ability) is the most useful, because (a) it best separates internal efficacy from related concepts, like external efficacy (e.g., Balch, 1974; Cohen, Vigoda, & Samorly, 2001), political awareness (Zaller, 1992), and political sophistication (e.g., Luskin, 1987); and (b) it matches with the psychological concept of perceived self-efficacy from SCT (Bandura, 1991, 1997). While the importance of the concept of internal political efficacy is undisputed, there has been controversy on how to measure it appropriately (for an overview, see Morrell, 2003, pp. 591–595). Most researchers use internal efficacy measures based on the National Election Study (NES) scale introduced by Niemi et al. (1991). Caprara et al. (2009) identify two conceptual weaknesses in those traditional measures of internal efficacy, which potentially limit their predictive validity. First, although self-efficacy is a psychological construct, with an extensive body of psychological theory and research surrounding the topic (for an extensive review, see Bandura, 1997), the NES scale has not been constructed using the theoretical insight into the psychological nature of the concept. Second, the NES items address a relatively small set of politically relevant skills, with a focus on understanding (rather than participating in) the political process (for a similar critique of the NES scale see Bandura, 1997, pp. 483– 484, and Caprara & Vecchione, 2017, p. 287). Therefore, Caprara et al. (2009) constructed the P-PSE scale.

    Item generation and selection

    Caprara et al. (2009) constructed the P-PSE scale, which is based on the guidelines that Bandura (2006) created for the construction of domain-specific self-efficacy measures: Its items should be formulated in terms of capability judgements, their content should represent all tasks relevant for what is considered successful behavior, and the items should include varying levels of task difficulty. Applying these principles to the political domain, Caprara et al. (2009) suggested two main sets of skills necessary to successfully participate in a representative democracy: (1) to voice and effectively promote one’s own political opinions and (2) to execute control over elected officials. From these skills, they deduced ten items referring to concrete tasks of political participation, for each of which respondents are asked to rate their capability of mastering it (see Table 1). In addition, Vecchione et al. (2014) suggested a short version of the P-PSE scale, by selecting a subset of four items (3, 4, 8, and 10), which they argue to adequately represent the content of the full scale.

    Since the original scale was published in English, we used the English version as source instrument for the translation, thereby ensuring that future translations to other languages can be based on the same source instrument. After one German native speaker translated the scale to German, we reviewed the scale in a three-person team. The team consisted of two Germans and one English native speaker, all of whom are fluent in the other language, and who - as suggested by the Best Practice Guidelines for Cross Cultural Surveys - unite different levels of discipline expertise (Survey Research Center, 2016, p. 245). Following the guidelines, we aimed to “keep the content of the questions semantically similar; keep the question format similar within the bounds of the target language; [and] retain measurement properties, including the range of response options offered” (Survey Research Center, 2016, pp. 233–234). In order to achieve these goals, we based our translation on an asking-the-same-questions-and-translation approach (ASQT; Survey Research Center, 2016, p. 234), and tried to stay as close to the content of the original items as possible. As in the original instrument, the translated version asks about respondents’ perceived capability to execute different political activities in the introduction. By instruction, each item displays a capability judgment (“I feel capable to…”), which is to be rated on a five-point agreement scale. The translated items are displayed in Table 1.

     

    Samples

    The online survey was conducted in October 2016 by the professional sampling agency Respondi (www.respondi.com). The sampling process followed a plan with representative quota for the German adult population regarding age, gender, and formal education. All respondents declared their informed consent before starting the survey and received a financial incentive for their participation by the sampling agency. After exclusion of careless responders (cf. Meade & Craig, 2012) and listwise exclusion of missing values (N = 64), a total of N = 1025 cases was used for analysis. The sample consisted of 51.7% females vs. 48.3% males. The mean age was 51.7 years (SD = 16.5). Of the participants, 36.7% reported low levels of formal education (‘Hauptschule’ or no degree at all), 30.5% reported medium levels (‘Realschule’), and 32.8% reported high levels (‘Abitur’ or ‘Fachabitur’). The sample distributions closely approximated population parameters, even after the exclusion of careless responders and missing value.

    In order to assess cross-cultural invariance of the scale, we used an Italian sample from Caprara et al. (2009, Study 1), which was kindly provided to us by the authors. The data were collected via face-to-face questionnaire in Italy in 2008. The participants were recruited by psychology majors, who conducted the interviews as part of a course assignment (for more details, see Caprara et al., 2009, p. 1006). All respondents participated voluntarily. After listwise exclusion of missing values (N = 30), the Italian sample had a total of N = 1654 valid cases. Although a convenience sample, distributions were diverse regarding gender, age, and formal education: The sample consisted of 54.4% females vs. 45.6% males. Respondents’ mean age ranged from 19 to 89 years (M = 44.7; SD = 17.6). Of the respondents, 20.7% had concluded elementary or junior high school, 55.6% had concluded high school, and 23.8% had achieved some university degree.

     

    Item analyses

    We tested unidimensionality of the scales for the German sample using confirmatory factor analysis (CFA) with one latent factor. All data analysis was conducted in R version 3.5.1 (R Core Team, 2017) via RStudio version 1.1.456 (RStudio Team, 2015). CFA was conducted using the lavaan package (version 0.6-3; Rosseel, 2012) with diagonally weighted least squares estimation due to the items’ ordinal level of measurement (Kline, 2016, pp. 257–258). As recommended by Hu and Bentler (1999), we assessed model fit by jointly considering the comparative fit index (CFI; acceptable fit > 0.95) and standardized root mean-square residual (SRMR; acceptable fit < 0.08). Both indices corroborated the hypothesis of a single latent factor for the full scale (CFI = 0.993; SRMR = 0.048), as well as the short scale (CFI = 0.999; SRMR = 0.022; see the Table 2 for an overview).

    Table 2

    Confirmatory factor analysis (CFA)

    Model

    χ² (df)

    CFI

    RMSEA [90% CI]

    SRMR

    Decision

    CFA 1: Full scale

    103.58* (35)

    0.993

    0.044 [0.034, 0.054]

    0.048

    Accept

    CFA 2: Short scale

    4.34 (2)

    0.999

    0.034 [0.000, 0.078]

    0.022

    Accept

    Note. N = 1025. We did not consider the χ 2 test for model rejection, because its sensitivity increases with sample size: In large samples—as is the case in this study—even small model–data discrepancies produce significant results (Bollen & Long, 1993; Cheung & Rensvold, 2002)

    CFI comparative fit index, RMSEA Root Mean Square Error of Approximation, SRMR Standardized Root Mean-square Residual

    *p < 0.001

     

    Item parameters

    The German P-PSE scale resulted in a mean score of M = 2.78 (SD = 0.88). Item–total correlations varied between r = 0.49 (item 1) and r = 0.79 (items 6 and 7), with a mean correlation of r = 0.67 (SD = 0.11). Item difficulty ranged from 0.24 (item 9) to 0.62 (item 1), with a mean difficulty of 0.44 (SD = 0.11). As suggested by Vecchione et al. (2014), we used the items 3, 4, 8, and 10 as a short version of the P-PSE scale, which resulted in a mean score of M = 2.90 (SD = 1.01). Other item-level statistics and intercorrelations are reported in the Table 3.

    Table 3

    Item-level statistics and intercorrelations of the translated P-PSE scale

    Item

    M

    SD

    (1)

    (2)

    (3)

    (4)

    (5)

    (6)

    (7)

    (8)

    (9)

    (10)

    1.

    3.46

    1.13

    0.49

    0.38

    0.45

    0.39

    0.32

    0.36

    0.37

    0.45

    0.24

    0.40

    2.

    3.22

    1.11

     

    0.51

    0.42

    0.44

    0.39

    0.38

    0.37

    0.38

    0.25

    0.50

    3.

    2.91

    1.23

     

     

    0.72

    0.56

    0.46

    0.60

    0.68

    0.64

    0.47

    0.52

    4.

    2.72

    1.20

     

     

     

    0.74

    0.59

    0.66

    0.63

    0.54

    0.52

    0.56

    5.

    2.40

    1.12

     

     

     

     

    0.66

    0.66

    0.57

    0.47

    0.48

    0.49

    6.

    2.56

    1.20

     

     

     

     

     

    0.79

    0.75

    0.63

    0.60

    0.56

    7.

    2.60

    1.27

     

     

     

     

     

     

    0.79

    0.70

    0.60

    0.54

    8.

    3.01

    1.25

     

     

     

     

     

     

     

    0.74

    0.51

    0.60

    9.

    1.95

    1.07

     

     

     

     

     

     

     

     

    0.61

    0.42

    10.

    2.94

    1.20

     

     

     

     

     

     

     

     

     

    0.69

    Note. N = 1025. The upper triangle displays intercorrelations. Values in the diagonal are corrected item-total-correlations 

    Objectivity

    The translated P-PSE scale contains a written instruction, Likert scale response options, and a simple aggregation rule to obtain the scale-level score. For paper-and-pencil and computer-based questionnaires, these features sufficiently ensure objectivity regarding administration and scoring (Lösel, 1999).

     

    Reliability

    In order to assess reliability, we calculated the internal consistency estimator Omega (ω) described by McDonald (1999), which has been demonstrated to perform better than Cronbach’s α (Dunn et al., 2014). The internal consistency of the German P-PSE scale was ω = 0.91 (95% CI [0.91, 0.92]), which indicates high internal consistency. The four-item short scale resulted in a lower, yet acceptable internal consistency of ω = 0.84 (95% CI [0.83, 0.86]). The Italian P-PSE scale yielded an internal consistency of McDonald’s ω = 0.92.

     

    Validity

    We assessed content, construct, criterion and incremental validity of the translated P-PSE scale and its four-item short scale.

    For content validity we compared the measured skill of the P-PSE and the NES. The scale’s variety of specific tasks contrasts with the NES scale, which relies on a more narrow understanding of political competence - rather focusing on knowledge and comprehension of politics. The P-PSE scale can therefore be argued to cover the phenomenon of internal efficacy better than its preceding alternatives, and hence, to offer a more content valid measure of the concept (cf. Fontaine, 2005, p. 804).

    In order to assess construct validity of the translated scale, we postulated a nomological network of theoretically related (convergent validity) and unrelated (discriminant validity) constructs and tested the correlations between these constructs and the P-PSE score (Hartig et al., 2008, pp. 148–154). For validation, we included the German internal efficacy scales by Beierlein et al. (2014; Spearman Brown coefficient = 0.83) and Vetter (1997; McDonald’s ω = 0.78), a five-item political interest scale (Otto & Bacherle, 2011; McDonald’s ω = 0.94), a three-item external efficacy scale (Vetter, 1997; McDonald’s ω = 0.74), a three-item scale of general self-efficacy (Beierleinet al., 2013; McDonald’s ω = 0.88), a self-placement of left–right orientation (based on GESIS, 2015), and a list of eleven items asking about the respondents’ political participation behavior (e.g., “During the last two years, how often did you actively participate in a political party or movement?”; McDonald’s ω = 0.83).

    All results are displayed in Table 4. As expected, we found high correlations with both preexisting internal efficacy scales. There was also a high correlation with political interest, which is in line with previous studies (e.g., Craig et al., 1990, p. 305; Foschi & Lauriola, 2014, p. 350). We found medium-sized correlations with external efficacy and general self-efficacy. Again, this is very plausible, since both constructs are conceptually related to internal efficacy (Balch, 1974; Bandura, 1997). In line with previous findings and theoretical assumptions (e.g., Caprara et al., 2009), the P-PSE score was independent from left–right orientation, but revealed a small correlation with ideological extremity— operationalized as the squared z-standardized left–right score. Regarding sociodemographic variables, men and highly educated people scored higher on the P-PSE scale than women and people with lower levels of formal education, which is the typical pattern of internal efficacy (e.g., Arzheimer, 2005, p. 199). Additionally, we found a small positive correlation with age. Concluding, the correlations towards external criteria reveal the expected pattern for a measure of internal efficacy: high correlations with other internal efficacy measures and political interest, medium-sized correlations with related self-belief variables, and null-correlations with independent constructs. The same pattern emerged when using the four-item short scale (see Table 4).

    Table 4

    Pearson’s correlations of the P-PSE scales with theoretically related (convergent validity) and unrelated (discriminant validity) constructs

    Construct

    P-PSE full scale

    P-PSE short scale

    r

    p value

    r

    p value

    Sociodemographic variables

     

     

     

     

     

    Age

    0.07

    0.020

    0.12

    < 0.001

     

    Gender (0 = female, 1 = male)

    0.20

    < 0.001

    0.19

    < 0.001

     

    High level of formal education

    0.13

    < 0.001

    0.14

    < 0.001

    Vetter scale

    0.60

    < 0.001

    0.60

    < 0.001

    Beierlein scale

    0.61

    < 0.001

    0.60

    < 0.001

    Political interest

    0.59

    < 0.001

    0.61

    < 0.001

    External efficacy

    0.24

    < 0.001

    0.22

    < 0.001

    General self-efficacy

    0.27

    < 0.001

    0.25

    < 0.001

    Left–right orientation

    0.00

    0.988

    0.00

    0.942

    Extremism (based on left–right orientation)

    0.15

    < 0.001

    0.13

    < 0.001

    Note. Values of gender and level of formal education are point-biserial correlations

     

    One predominant aspect of internal efficacy is its predictive value regarding political participation behavior (Bandura, 1997; Krampen, 1990; Vecchione & Caprara, 2009). We therefore assessed the scale’s criterion validity (Hartig et al., 2008, p. 156) by measuring its relationship towards the propensity to participate in politics. Similar to other researchers in the field (e.g., Kaase, 1999; Peterson, et al., 2008), we asked about past involvement in eleven different activities of political participation behavior (e.g., stating one’s political opinion or signing a political online-petition) and used these items to build an index of political participation propensity (McDonald’s ω = 0.83). We used a hierarchical regression model including the control variables age, gender, and education to estimate the scale’s criterion validity. As expected, the P-PSE scale explained a substantial amount of variance in respondents’ participation propensity over and above the sociodemographic variables in its full ten-item version (β = 0.28, p < 0.001, ΔR2 = 0.26), and in its four-item short version (β = 0.25, p < 0.001, ΔR2 = 0.26).

    In their validation study, Caprara et al. (2009) showed that the original P-PSE scale accounted for unique variance in several indicators of political participation over and above the traditional NES scale. In order to corroborate the incremental value of the scale in the German context, we aimed at replicating this finding with regards to the Vetter and Beierlein scales. We estimated two more hierarchical regression models of political participation propensity. Each model included the before-mentioned control variables and one of the traditional measures before adding the translated PPSE scale in a second step. The P-PSE scale increased the explained variance by ΔR2 = 0.12 compared to the Vetter scale and by ΔR2 = 0.13 compared to the Beierlein scale. The four-item short scale revealed the same incremental value compared to the traditional scales (see Bromme et al., 2020).

     

    Descriptive statistics (scaling)

    As mentioned above, the full German P-PSE scale resulted in a mean score of M = 2.78 (SD = 0.88; skew = 0.18; kurtosis = -0.37), while the short version resulted in a mean score of M = 2.90 (SD = 1.01; skew = 0.03; kurtosis = -0.56).

     

    Further quality criteria

    We tested cross-cultural invariance of the scale by comparing our German sample (n1 = 1025) to the Italian sample (n2 = 1654). Using multigroup CFAs, we tested for configural invariance (same factor structure across samples), followed by metric invariance (same factor loadings across samples), scalar invariance (same item intercepts across samples), and residual invariance (same error variances across samples; for a similar procedure, see Baumert et al., 2014; for an overview of measurement invariance conventions, see Putnick & Bornstein, 2016). We assessed model fit of the configural invariance model using the same criteria as before (Hu & Bentler, 1999). All subsequent models are each nested within its preceding model (e.g., metric invariance within configural invariance) and were therefore assessed in comparison to the preceding model. To judge whether fit differences between nested models are substantial, we used the cutoff criteria by Chen (2007). For large sample sizes, she recommends the use of ΔCFI = 0.01 as main criterion, and ΔRMSEA = 0.015 and ΔSRMR = 0.01 (except for metric invariance, where ΔSRMR = 0.03) as additional criteria. Using the same specifications as before, we found full configural invariance between the two samples (CFI = 0.996; RMSEA = 0.035; SRMR = 0.036; see the lower part of Table 2). However, full metric invariance could not be established (e.g., ΔCFI = 0.015). We identified Item 2 to differ most strongly in its loading across samples, and - allowing this item’s loading to differ freely between samples, as suggested by Vandenberg and Lance (2000, p. 57) - established a model of partial metric invariance. This model met two of our fit criteria (ΔCFI = 0.009; ΔSRMR = 0.018), but exceeded the cut off value of the third one (ΔRMSEA = 0.023). Since ΔCFI is recommended as the main criterion (Chen, 2007, p. 501) and absolute model fit was still good in terms of the Hu and Bentler (1999) criteria, we decided to accept this model of partial metric invariance. Since the majority of item loadings was invariant (Vandenberg & Lance, 2000, p. 38; see also Putnick & Bornstein, 2016), we proceeded in testing scalar invariance. Again, the full scalar invariance model failed our criteria (e.g., ΔCFI = 0.033). Thus, we identified the three items which most strongly diverged in their intercepts (Items 1, 2, and 3) and - allowing these to differ between groups - established a model of partial scalar invariance (ΔCFI = 0.006; ΔRMSEA = 0.009; ΔSRMR = 0.007). Finally, we tested the residual invariance model against the preceding model and found it to fit relatively well (ΔCFI = 0.003; ΔRMSEA = 0.001; ΔSRMR = 0.006). Summarizing, we can say that the ten P-PSE items load on a single latent factor in both samples (Germany and Italy), and that nine out of ten items do so with equal factor loadings across samples, i.e., the scale shows partial metric invariance. The observed non-invariance of Item 2 (“Make certain that the political representatives you voted honor their commitments to the electorate”), indicates a cross-cultural difference in how much the described task (i.e., monitoring elected representatives) relates to the latent construct of internal efficacy, with slightly lower standardized loadings in Germany (λ2 = 0.53) than in Italy (λ2 = 0.78). Since only one item displays metric non-invariance, interpretation of the overall P-PSE mean score can be assumed to be unaffected (Steenkamp & Baumgartner, 1998; Steinmetz, 2013). Full scalar invariance was impeded by three out of ten items, with higher standardized intercepts in Germany (ν1 = 3.07, ν2 = 2.90, ν3 = 2.36) than in Italy (ν1 = 2.09, ν2 = 2.05, ν3 = 1.80). Since between-group differences in item intercepts can affect the comparability of observed mean scores (Steinmetz, 2013), we suggest to use latent modelling when group mean comparisons are of the essence, where partial scalar invariance is a sufficient prerequisite (Steenkamp & Baumgartner, 1998; see also Steinmetz, 2013).

    Table 5

    Measurement invariance (MI) model fit

    Model

    χ² (df)

    CFI

    RMSEA [90% CI]

    SRMR

    Model comparison

    Ref.

    Δχ 2 (Δdf)

    ΔCFI

    ΔRMSEA

    ΔSRMR

    MI 1: configural

    185.20* (70)

    0.996

    0.035 [0.029, 0.041]

    0.036

    -

    -

    -

    -

    -

    MI 2: metric

    603.73* (79)

    0.981

    0.070 [0.065, 0.076]

    0.064

    MI 1

    418.53* (9)

    0.015

    0.035

    0.028

    MI 2a: partial metric

    431.67* (78)

    0.987

    0.058 [0.053, 0.064]

    0.054

    MI 1

    246.47* (8)

    0.009

    0.023

    0.018

    MI 3: scalar

    1331.43* (87)

    0.954

    0.103 [0.099, 0.108]

    0.088

    MI 2a

    899.76* (9)

    0.033

    0.045

    0.034

    MI 3a: partial scalar

    593.78* (84)

    0.981

    0.067 [0.062, 0.072]

    0.061

    MI 2a

    162.11* (6)

    0.006

    0.009

    0.007

    MI 4: residual

    682.74* (94)

    0.978

    0.068 [0.064, 0.073]

    0.067

    MI 3a

    88.96* (10)

    0.003

    0.001

    0.006

    Note. n1 = 1025; n2 = 1654. We did not consider the χ 2 test for model rejection, because its sensitivity increases with sample size: In large samples—as is the case in this study—even small model–data discrepancies produce significant results (Bollen & Long, 1993; Cheung & Rensvold, 2002) CFI comparative fit index, RMSEA Root Mean Square Error of Approximation, SRMR Standardized Root Mean-square Residual, Ref. reference model *p < 0.001

     

    Acknowledgement

    We thank two anonymous reviewers for their thoughtful feedback on an earlier draft and Kristen Werling for her valuable help with the translation of the items.

     

    • Laurits Bromme, University of Koblenz-Landau, Fortstraße 7, 76829 Landau, Germany, E-Mail: bromme@uni-landau.de
    • Tobias Rothmund, Friedrich Schiller University of Jena, Jena, Thuringia, Germany
    • Gian Vittorio Caprara, Sapienza University of Rome, Rome, Italy.

    The datasets analyzed during the current study are available at https://osf.io/89j7h/ and can be accessed for scientific purposes.