Open Access Repository for Measurement Instruments

AkkordeonDetailedScaleViews

Epistemological attitudes in research scale (EARS-6)

  • Author: Schubert, K. & Fischer, M.
  • In ZIS since: 2023
  • DOI: https://doi.org/10.6102/zis336
  • Abstract: Epistemological questions – about how to achieve objective knowledge through research – have recently become a matter of political contestation, both from within and outside academia. P ... moreroponents of positivist epistemologies hold that objective knowledge can be produced by abstracting from one’s social position and accuse critical theories, such as gender studies, for moralizing and politicizing academia. Proponents of critical epistemologies hold that it is necessary to include socially situated knowledge into research. Therefore, they argue that a merely positivist epistemology stabilizes discrimination within academia and hinders broad knowledge by excluding minority standpoints. EARS-6 translates the epistemological and political debates to general attitudes towards research and the scale works independent of participants’ actual knowledge of epistemological theories. less
  • Language Documentation: English
  • Language Items: deutsch
  • Number of Items: 6
  • Survey Mode: CASI
  • Reliability: Raykov's omega = .62; Cronbach’s alpha = .63
  • Validity: evidence for predictive validity of prejudice and political orientation
  • Construct: Epistemological attitudes
  • Catchwords: positivism, critical epistemology, research, minority standpoints
  • Item(s) used in Representative Survey: no
  • Status of Development: tried
    • Instruction

      Nun geht es um Ihre Einstellung zur Wissenschaft. Inwiefern stimmen Sie den folgenden Aussagen zu?

       

      Items

      Table 1

      Items of the Scale Epistemological Attitudes in Research Scale (EARS)

      No.

      Item

      Polarity

      EARS_1

      Wissenschaft sollte politisch neutral sein.

      EARS_2

      Für wissenschaftliche Objektivität ist es wichtig, dass Forschende von ihren persönlichen Einstellungen absehen.

      EARS_3

      Rassismusforschung sollte von Forschenden durchgeführt werden, die selbst von Rassismus betroffen sind.

      +

      EARS_4

      Diversitätsfördernde Maßnahmen an der Universität sind schlecht für die Qualität der Forschung.

      EARS_5

      Gender Studies ist oft mehr Ideologie als Wissenschaft.

      EARS_6

      Gute Wissenschaft sollte kritisch reflektieren, welche politischen Auswirkungen sie hat.

      +

       

      Response specifications

      (1) Stimme überhaupt nicht zu

      (2) Stimme eher nicht zu

      (3) Teils/teils

      (4) Stimme eher zu

      (5) Stimme voll und ganz zu

       

      Scoring

      Items 1, 2, 4 and 5 are negatively keyed and should be coded reversely before combining all items into an equally weighted average score scale. In case of item non-response to one or more items in the scale, we would generally advise against constructing the scale with the remaining items. The item averages in our data vary between 1.84 and 4.2 (see table 4 below), which suggests that a missing value on one item best not be replaced with an average across the remaining items. The higher the value on EARS, the more critical the epistemological attitude. The lower the value on EARS, the less critical and more positivistic the epistemological attitude.

       

      Application field

      The scale can be used to measure whether people have a critical or positivistic understanding of research. It has been developed for a sample of university students in Germany. Adaptations for researchers are scheduled for testing in 2024. Due to the reliability of the scale, we recommend its use for research purposes only and not for individual diagnostics. So far, the scale was applied successfully in computer-assisted self-administered interviewing (CASI).

    Standpoint epistemology criticizes the conventional view of scientific ‘objectivity’, where objectivity is thought to be achieved through an abstraction from particular social positions and a commitment to political neutrality (Anderson, 2019; Harding, 2004b; Toole, 2021). By veiling researchers’ social position and (implicit) political leanings, and by “excluding politically engaged research on behalf of oppressed groups” (Harding, 2004a, p. 5), this positivist concept of objectivity may lead to stabilization of structures of discrimination in academia, like sexism, racism, or heteronormativity. These structures, in turn, limit the plurality of standpoints that is necessary for more complete accounts of the (social) world (Toole, 2022). What Harding calls “strong objectivity” (Harding, 2004a) can be reached by systematically including the multiplicity of “situated knowledges” (Haraway, 1988) of marginalized people into academia, connecting research with social critique. Disciplines that focus on the histories and persistent structures of social discrimination, such as gender studies, postcolonialism, critical race theory, or queer studies, often use methods that follow such standpoint epistemology. These epistemological questions have recently become a matter of cultural wars over identity politics (Schubert, 2023a), with conservatives and liberals charging such critical theories with relativism and partisanship (Boghossian, 2013; Pluckrose et al., 2018). The normativity of critical theories is portrayed as endangering academic freedom, excluding them from the realm of legitimate academic approaches, for example by the German Netzwerk Wissenschaftsfreiheit (engl. network academic freedom) (Schubert, 2023b). The statements of EARS-6 convey typical positions of these debates and allow to measure if a participant is leaning towards critical or positivist epistemological attitudes. EARS-6 thereby constitutes the first translation of this epistemological and political debate into a quantitative scale, which come forth from a transdisciplinary collaboration between a theorist and a quantitative empirical researcher.

    Item generation and selection

    The items were developed by leveraging the potential of transdisciplinary collaboration by a political theorist and a quantitative sociologist. We used an hourglass principle, whereby we began with broad and abstract ideas about epistemology and the role it plays in contemporary political struggles over identity politics led by Schubert’s expertise in political theory. This led to the identification of the following relevant aspects: 1) political neutrality or critical normativity of research, 2) objectivity as standpoint dependent or as abstraction from standpoints. This was followed by a reduction of these concepts into concrete quantitative measures under guidance of Fischer’s expertise in survey design and instrument development. EARS_1 and EARS_6 relate to aspect 1), EARS_2, _3, _4 relate to aspect 2), and EARS_5 relates to both aspects. Finally, we negotiated the wording of the empirical instruments in an iterative process to ensure their validity and to link them back to overarching political and epistemological discourses in contemporary research. 

     

    Samples

    The survey was conducted as part of an adversarial collaboration research project into academic freedom. The survey was conducted in CASI mode in 2023 among students at German universities. The recruitment was done through an access panel hosted by the commercial survey company Forsa. This allowed us to obtain a sizable sample of respondents at affordable cost. The survey was conducted in German. For the purpose of the EARS-6 scale construction, missing values on EARS_1 though EARS_6 were deleted listwise (N = 30). Table 2 shows the descriptive statistics of the analytical sample of N = 976.

     

    Table 2

    Description of the Sample

     

    N

    %

    Mean

    SD

    Cis-man

     

     

     

     

      Yes

    480

    50%

     

     

      No

    488

    50%

     

     

    Academic parent/s

     

     

     

     

      Yes, both

    251

    26%

     

     

      Yes, one

    279

    29%

     

     

      No

    445

    46%

     

     

    Heterosexual

     

     

     

     

      Yes

    785

    84%

     

     

      No

    153

    16%

     

     

    Born in Germany

     

     

     

     

      Yes

    939

    96%

     

     

      No

    36

    4%

     

     

    Both parents born in Germany

     

     

     

     

      Yes

    832

    85%

     

     

      No

    137

    14%

     

     

    Both grandparents born in Germany

     

     

     

     

      Yes

    647 

    68%

     

     

      No

    301

    32%

     

     

    Age

    970

     

    27.13

    6.97

    Study semester

    972

     

    7.71

    3.95

    Note. Some percentages do not combine to 100% within variables due to rounding error.

     

     

    Item analyses

    EARS-6 was constructed as single scale without subdimensions. We conducted a principal component analysis using Stata 18. Results lend support to a single factor solution as best suited to describe the data, when using a minimum eigenvalue of 1 as cut-off (see Fig 1.). The first factor has an eigenvalue of 1.46 and describes the average position on epistemological attitudes because it affects all items negatively in absolute values (considering the reversely worded items). EARS_4 and EARS_5 show the highest rotated factor loadings at about .70, whereas the remaining items load between .30 and .40 (see table 3).

     


    Figure 1
    . Screeplot after Principal Component Analysis EARS_1 through EARS_6, N = 976.

     

     

    Table 3

    Rotated Factor Loadings for EARS-6 After Factor Analysis

     

    Factor loadings

    EARS_1

    .42

    EARS_2

    .33

    EARS_3

    -.35

    EARS_4

    .67

    EARS_5

    .72

    EARS_6

    -.28

     

     

    Since these analyses assume multivariate normality, we also conducted a confirmatoryy factor analysis, which can account for non-normality. We used maximum likelihood estimation and fixed the variance of the factor to 1. Figure 1 shows the path diagram with standardized coefficients. All of the coefficients are statistically significant at p < .001.

     

    Figure 2. t-congeneric measurement model for the construct. Standardized path coefficients, RMSEA = .118, CFI = .856, SRMR = .069, c²(9) = 131.004, < .001, = 976.

     

     

    In terms of goodness of fit indices, we observe mixed results. The Chi-squared test is significant, which indicates poor fit of the model to the data (Wu & Thompson, 2020). However, the Chi-squared test assumes multivariate normality and it tends to indicate poor fit in large samples. The RMSEA is above .070 with a value of .118, also indicating poor fit of the model to our data. The CFI falls just short of the commonly used threshold of .950 for good model fit with a value of .856. The standardized root mean squared residual (SRMR), however, indicates an adequately fitting model with a score of .069 (Wu & Thompson, 2020).

     

    An examination of the modification indices shows that adding the covariance between the error term of EARS_1 and EARS_2 would improve model fit the most (MI = 84.30, p < .001). When allowing for the covariance between these two items, we observe a RMSEA below .070 (RMSEA = .068) and a somewhat improved Chi2-test. The second largest improvement to model fit is by adding the covariance between the error terms of EARS-4 and EARS-5. As a result, all measures of fit indicate that the updated model fits the data well. Specifically, we observe c²(7) = 28.042, < .001, RMSEA = .056, CFI = .976 and SRMR = .026 (= 976).

     

    Item parameters

     

    Table 4

    Mean, Standard Deviation, Skewness and Kurtosis of EARS individual items

     

    Mean

    Standard Deviation

    Skewness

    Kurtosis

    EARS_1

    3.99

    1.13

    -0.93

    2.97

    EARS_2

    4.23

    0.93

    -1.11

    3.73

    EARS_3

    2.59

    1.01

    -0.01

    2.52

    EARS_4

    1.84

    1.13

    1.37

    4.05

    EARS_5

    2.72

    1.29

    0.29

    2.02

    EARS_6

    3.76

    1.19

    -0.81

    2.78

    Note. Scale ranging from 1 (stimme überhaupt nicht zu) to 5 (stimme voll und ganz zu), = 976.

    Objectivity

    The objectivity of application for EARS-6 is given, since the scale has a predefined introduction to be used in self-administered survey modes. If the introduction is used unchanged by trained interviewers in interviewer-assisted survey modes, the application objectivity is given. Moreover, the EARS-6 satisfies evaluation objectivity since the scale construction is standardized as described in this document. When it comes to the interpretation of the scale, there are no objective cut-offs to determine the substantive conclusions to be drawn. It is up to the researcher to contextualize the scale value within their specific research design (e.g., population, geo-graphical context, etc.).

     

     

    Reliability

    The EARS-6 demonstrates adequate reliability with a Cronbach’s alpha of α = .63. There is however room to improve reliability of the scale, as also suggested by Raykov's omega of ω = .62. This reflects the fact that the EARS-6 bundles various aspects of the overarching concept of epistemological attitudes. Any omission of individual items would leave Cronbach’s alpha unchanged or lower. 

     

    Validity

    For validity, we present correlations between the EARS-6 and neo-prejudice on the one, and political self-positioning, on the other hand. Neo-prejudice refers to negative attitudes toward minorities, which have been called modern or covert, as opposed to traditional and overt racism, sexism or homophobia (Lima et al., 2019; Morrison & Morrison, 2003; Tougas et al., 2004; Walls, 2008). The neo-prejudice index combines modern stigma toward sexual and ethnic minorities and women. A higher score indicates more prejudice. Political self-positioning is measured on a scale from 1 left to 10 right.

     

    Pairwise correlations show that there are significant and sizable correlations between these variables. Respondents with more critical epistemological attitudes hold less prejudice than respondents with positivist epistemological attitudes. People with critical epistemic attitudes identify more strongly as politically left, whereas people with more positivist epistemological attitudes identify more with the political right. This validates the EARS-6 as tapping into the discursive intricacies of the politization of research.

     

    Table 5

    Pairwise correlation of EARS-6 scale, neo-prejudice and political self-positioning

     

    (1)

    (2)

    (3)

    (1) EARS-6

    1.00

     

     

    (2) Neo-prejudice

    -.68***

    1.00

     

    (3) Political self-positioning

    -.56***

    .64***

    1.00

    Note. *** p < .001.

     

    Descriptive statistics

     

    Table 6

    Mean, Standard Deviation, Skewness and Kurtosis of EARS-6

     

    Mean

    Standard Deviation

    Skewness

    Kurtosis

    EARS-6

    2.93

    0.66

    -0.19

    2.98

    Note. Scale ranging from 1 (stimme überhaupt nicht zu) to 5 (stimme voll und ganz zu), = 976.

     

     

    Acknowledgement

     

    EARS-6 was developed by Schubert and Fischer in the scope of an adversarial collaboration that brings together critical and positivist researchers to develop a questionnaire to measure potential threats to academic freedom. Further participants of the collaboration are Richard Traunmüller, Matthias Revers, Claudia Diehl, Nils Weidmann, and Alexander Wuttke.

    Karsten Schubert, Humboldt-University Berlin, karsten.schubert@hu-berlin.de

    Mirjam Fischer, Humboldt-University Berlin and Goethe-University Frankfurt, mirjam.fischer@hu-berlin.de