The Short Form Health Survey SF-12 questionnaire (Ware et al., 1996) is a short version of the SF-36 (Morfeld & Bullinger, 2008; Ware et al., 1993) used to assess the impact of health on an individual’s everyday life. Thanks to its brevity, this quality of life measure is appropriate for use in survey research. Like its predecessor the SF-36, it provides composite scores for both PH and MH across eight domains of health: PH-related scales comprise 1) General Health; 2) Physical Functioning; 3) Role Physical and 4) Body Pain. MH-related scales include 5) Vitality; 6) Social Functioning; 7) Role Emotional and 8) Mental Health.

In recent decades, there has been a marked shift in prevalent health problems from acute to chronic conditions in industrialized nations, influenced by medical advances and demographic changes (Robert Koch Institute, 2015). Traditionally, evaluating the impact of diseases on a population has relied on mortality data. However, with increasing life expectancy and advances in medical treatments, the need to assess the severity of illness, both physical and mental, has become increasingly apparent. Such assessments need to encompass various aspects of health and their consequences for the well-being of individuals (Bullinger & Quitmann, 2014). Therefore, health-related quality of life measures have become increasingly established in both research and clinical practice, playing a crucial role in assessing the overall well-being of individuals and providing insights for informed decision-making in healthcare settings (Wirtz & Bengel, 2011). While a consistent definition of the concept is lacking, it commonly considers physical, psychological, social, family, and work-related aspects (Otto & Ravens-Sieberer, 2020).

The SF-12 is one of the most widely utilized tools for evaluating health-related quality of life, in both German-speaking contexts and internationally (Morfeld & Bullinger, 2008). The reliability of the SF-12 has been evaluated in the U.S., UK., and China for general populations (Fong et al., 2010; Ware et al., 1996) as well as for groups with mental illness (Huo et al., 2018). Test-retest reliability scores have been reported as consistently higher for PH (.61 (for clinical samples) to .89) than for MH (.57 (for clinical samples) to .77). With respect to validity, SF-12 form was found to strongly correspond to the longer SF-36 form, to correlate with other measures of PH and MH to the expected degree (Ware et al., 1996) and to differentiate between groups with and without a corresponding health diagnosis (Fong et al., 2010). Some previous studies have indicated potential factorial structure problems as it varied between different language and ethnic groups (Fleishman & Lawrence, 2003; Fong et al., 2010). A version in German and in languages of migrant groups in Germany has been used in the SOEP (Andersen et al., 2007). The factorial structure of the SF-12 is not supported in the SEOP refugee panel (Tibubos & Kröger, 2020).

Cognitive pretests

The German, Arabic, and Farsi versions of the SOEP (Andersen et al., 2007) were adapted. The Farsi version was adapted to Dari using the TRAPD approach (Translation, Review, Adjudication, Pretest, Documentation) with two professional translators. In the next step Cognitive Interviews (CI) were conducted with 18 refugees (Syrian N = 6, Iraqi N = 6, Afghan N = 6) of different gender, age, and education (Hadler et al., 2021). The pretest was conducted through remote interviews using either video or telephone conferences. In addition to the interviewees, interpreters also took part in the conferences in order to take cultural and linguistic particularities into account. In the case of several items, interviewees expressed confusion regarding whether certain items referred to PH, MH, or were intended as indicators of general health. Further challenges arose due to the lack of clarity regarding temporal references, as interviewees found it difficult to distinguish between their current feelings and those experienced over the previous four weeks. Refugees also tended to understand “all-day activities” as referring primarily to activities relating to their professional work. Respondents also interpreted the items regarding emotional feelings differently and discussed cultural and gender differences in their expression. Confusions were also associated with the varying number of response categories, with three categories used for some indicators and five for others. Respondents also had difficulties with double barreled questions (DBQs), i.e. questions that include enumerations or more than one stimulus to evaluate in one indicator (Menold, 2020).

Item generation and selection

Revisions of the instruments were provided to address these problems. Both the participants' feedback and the interpreters' assessments were taken into account. Additional indicators were developed to balance PH and MH. DBQs were split into different indicators. The number of response options was unified and five categories were used for all items. Redundancies were omitted and a clearer visual layout was used. Finally, the findings from the cognitive pretest were used to make adjustments to the introduction, to unify and highlight the temporal reference, and to provide alternatives to emotional terms.

In response to feedback from interviewees and upon splitting DBQs, various alternatives for item formulation were generated and incorporated into the tested form, resulting in a total of 18 items. Items were selected to find the most suitable six items per subscale (PH or MH), applying the principle of a clear separation between PH and MH. In addition, indicators that exhibited high differences in thresholds among the three groups were excluded. The general well-being item was excluded from the instrument, because it cannot be uniquely assigned to PH or MH.

Samples

Register sample

A probabilistic sample of three large cities in Saxony (Dresden, Leipzig, Chemnitz) was implemented. The target population included German-speaking residents and Arabic-speaking Syrian and Iraq residents (refugees). The addresses of sampled persons, together with information on gender, age, and country of origin, were provided by municipal registration offices. We only required the selection of addresses for refugees who had been resident in the city since 2014. In total, contact details for 1,000 German speakers, 2,000 Arabic speakers and 1,000 Dari speakers were provided. We used mail contact and re-contacts to recruit the participants to a web-based survey. Response rates RR6 (AAPOR, 2023) were 28.42 % for German-speaking residents, 19% for Arabic-speaking refuges and 16% for Dari-speaking refugees. The sample was randomized, with one group of respondents answering the SF-12 questionnaire version prior to the cognitive pretest, and the other group answering the questionnaire version after the cognitive pretest. For the SF-12 version administered after the cognitive pretest, relevant to this report, the sample size was N = 147 for Arabic speakers and N = 125 for German speakers

Facebook sample

The Facebook survey was hosted on the servers of TU Dresden and targeted people from Iraq, Syria and Afghanistan living in Germany. An external service provider was commissioned to advertise the web-based survey. In addition, Bielefeld University supported advertising, but was unable to collect socio-demographic information. To ensure that participants were refugees who spoke these languages, we screened participants to limit them to refugees living in Germany only. The questionnaire was available in Arabic and Dari only. A random part of them administered the adapted instrument as developed after the cognitive pretests. For the SF-12 version administered after the cognitive pretest, relevant to this report, the sample size was N = 779 for Arabic speakers and N = 503 for Dari speakers. See Tables 3 and 4 for a description of both samples.

Table 3

Descriptives Register sample

Note. N refers to the total sample size of respondents for German/Arabic with available demographic data for the specific question. Deviations in N due to Data Missing.

Table 4

Descriptives Facebook sample

Note. N refers to the total sample size of respondents for Arabic/Dari with available demographic data for the specific question. Deviations in N due to Data Missing.

Item analyses

Table  and 6 show the distribution parameters of the register and Facebook sample for each item of the instrument.

Table 5

Register sample: N, Means, Standard Deviations, Skewness, Excess Kurtosis, and Discriminatory Power of Items

Note. Note that polarity of item values was reversed for negatively poled items 3-12.

Table 6

Facebook sample: N, Means, Standard Deviations, Skewness, Excess Kurtosis, and Discriminatory Power of Items

Note. Note that polarity of item values was reversed for negatively poled items 3-12.

Reliability

To assess reliability, a composite general reliability coefficient (Raykov's Rho) based on factor analysis was calculated from the two-factor models reported in the “Factorial Validity” section. In addition, we calculated single general Rho based on the two factors (Raykov, 2012; see table 7). Correlated error terms were consistently included in the denominator as a part of the measurement error (cf. Raykov, 2012).

Table 7

Register and Facebook sample: Composite Reliability (Raykov´s Rho) with Standard Errors (SE)

Reliability analyses indicate high composite reliability scores for the PH scale in the probability sample and middle scores in the Facebook sample. Reliability of MH is high in both samples and all language groups. The two-factor based Rho (General Reliability) is reasonably good in the register and excellent in the Facebook sample. Overall, composite reliability of the adapted SF-12-ENSURE was found to be sufficient to excellent.

Validity

Factorial Validity

Factorial validity was evaluated by means of MG-CFAs for each sample separately (see Table 8). We used Mplus 8.7 (Muthén & Muthén,1998-2021) for these analyses. The model fit in the MG-CFAs was evaluated using the chi-square test, the Root-Mean-Square Error of Approximation (RMSEA), and the Comparative Fit Index (CFI) (Beauducel & Wittmann, 2005). The CFI of .95 or higher and RMSEA of.08 or less indicate an acceptable fit (Hu & Bentler, 1999). The Robust Maximum Likelihood estimator (MLR) was used due to the ordinal nature and non-normality of the data (Muthén & Muthén, 1998-2021). Due to five categories used we treated indicators as continuous (cf. e.g. Beauducel & Herzberg, 2006). The two factors could be supported as a just acceptable or acceptable model fit was reached in both samples. In both samples, two correlated error terms were included as suggested by high modification indices. The first correlated error was introduced between the items “restricted mentally” and “less accomplished mentally”. As the items are similar in content and wording and address limitations in activities due to the emotional problems, the error term can be justified. The second correlated error term was indicated between the items “stairs” and “hinder”, which seem to be more similar to each other with respect to physical limitations than other items of the PH factor. Both correlations are also reported for the SOEP samples (cf. Tibubos & Kröger, 2020) and are confirmed here, although the number of correlated error terms is reduced from four to two. The factor loadings were high for all items of the two factors in both samples. The factor correlation are of moderate size for German and Arabic languages, but high for Dari (Facebook sample).

Table 8

Register and Facebook sample: Results of MG-CFAs

Note. Standardized parameters reported., ***p < .001.

Convergent Validity

Convergent validity was assessed using Pearson correlations between the SF-12 scales and predefined concepts, which were assumed to be significantly (p < .001) correlated, with moderate to strong correlations of |.4| and above. There were positive correlations with similar concepts, both in the register and Facebook sample (see Table 9). There was a strong negative correlation with the general health status of refugees (Refugee Health Scale, RHS-SOEP, negatively polarized) both with regard to PH (r = -.58, N_Register = 110, r = -.66, N_Facebook = 359), as well as with regard to MH (r = -.80, N_Register = 110, r = -.82, N_Facebook = 359). This means that participants who described their physical and mental health as more pronounced in the SF-12-ENSURE also described themselves as healthier in the RHS. There were also correlations in the expected direction with regard to typical measures of MH. There were moderate negative correlations with depression and anxiety (Patient Health Questionnaire, PHQ-4) both with regard to PH (r_Depression = -.40, r_Anxiety = -.36, N_Register = 132; r_Depression = -.42, r_Anxiety = -.45, N_Facebook = 403 resp. 402), as well as with regard to MH (r_Depression = -.75, r_Anxiety = -.73, N_Register = 132; r_Depression = -.75, r_Anxiety = -.73, N_Facebook = 402 resp. 402). This means that participants who described their physical and mental health as more pronounced in the SF-12-ENSURE also described themselves as mentally healthier in the PHQ-4. Furthermore, in the register sample there is a weak positive correlation between PH and education (number of years of schooling, r = .21, N_Register = 226). Thus, health-promoting behaviour is usually associated with higher education (Adler & Newman, 2002; Zajacova & Lawrence, 2018), which corresponds to common findings and is also reflected in the results. With regard to MH, there is no correlation (r = .07, N_Register = 226, r = .00, N_Facebook = 559).

Divergent Validity

For divergent validity, correlations between the SF-12 scales and unrelated concepts were anticipated to be non-significant (p > .05), with correlation coefficients below |.3|, indicating weak associations. As expected, we observed very weak or no correlations between PH and MH in the SF-12-ENSURE and different concepts. However, there were weak to moderate correlations towards attitudes to democracy as well as weak correlations to authoritarianism and faith (see Table 9).

Table 9

Register and Facebook sample: Convergent and Divergent Validities of the SF-12

Note. PHQ-4 = Patient Health Questionnaire 4, RHS = Refugee Health Scale, pearson’s correlation, *p < .05, **p < .01, ***p < .001, Register Sample N = 104 – 241, Facebook Sample N = 329 – 655

Criterion Validity

The criterion validity has been established using Pearson correlations between the SF-12 scales and health outcomes, assumed to be significantly (p < .001) correlated, with scores of |.3|  and above. There were moderate to strong correlations with possible external criteria, both in the register sample and in the Facebook sample (see Table 10). There were moderate to strong negative correlations with reported comorbidities with regard to both PH (r = -.37, N_Register = 260, r = -.40, N_Facebook = 777) and MH (r = -.31, N_Register = 260, r = -.37, N_Facebook = 777). This means that people who described their physical and mental health as more pronounced in the SF-12-ENSURE also reported having fewer illnesses. There were moderate to strong positive correlations with reported general life satisfaction with regard to both PH ( r = .50, N_Register = 150, r = .42, N_Facebook = 536) and MH (r = .52, N_Register = 151, r = .52, N_Facebook = 534). This means that people who described their physical and mental health as more pronounced in the SF-12-ENSURE also described themselves as more satisfied overall. In the register sample here were weak negative correlations with alcohol misuse with regard to PH (r = -.19, N_Register = 251). This means that people who reported consuming alcohol on a regular basis also described their physical health as slightly worse. Finally, there were moderate to strong negative correlations with loneliness with regard to both PH (r = -.29, N_Facebook = 546) and MH (r = -.46, N_Register = 130, r = -.50, N_Facebook = 545). This means that people who described their physical and mental health as somewhat better in the SF-12-ENSURE also described themselves as being less lonely.

Table 10

Register and Facebook sample: Criterion Validities of the SF-12

Note. pearson’s correlations, *p < .05, **p < .01, ***p < .001, Register Sample N = 129 – 260, Facebook Sample N = 534 – 777

Measurement Equivalence

Measurement invariance analyses were conducted by means of MG-CFA when using the models obtained by the evaluation of factorial validity (Table 8). A difference was, however, that the introduced correlated error terms as described in the above section were kept equal between the respective groups to ensure equal configuration also for unexplained factors. The factor variance was set at 1 to allow comparisons among the loadings and the factor means were set at 0 to allow for comparison of intercepts and model identification (cf. e.g. Byrne, 2011). The analyses compare Arabic and German for the register sample and Arabic and Dari for the Facebook sample. A significant change of chi-square (Meredith, 1993) or a change of ΔCFI > .010 and ΔRMSEA > .010 indicated significant differences in model fit (Chen, 2007) and thus lack of exact measurement invariance. For the register sample, metric invariance is established according to all goodness of fit (GOF) statistics and scalar measurement invariance is given according to non-significant change of RMSEA and CFI. For Arabic and Dari, however, metric invariance is supported by RMSEA and CFI, whereas the scalar invariance is rejected by change of all GOF statistics. Partial scalar invariance is established in the Facebook sample when freeing the intercepts of three items (overall physical health, social functioning and less social physically). The adapted SF-12 can therefore be used to compare correlations between German and Arabic or between Arabic and Dari. Latent and summarized means are also fully comparable between Arabic and German. Mean comparisons between Arabic and Dari can be carried out based on partial measurement invariance models (cf. Pokropek et al., 2019). For comparisons among the three languages, further validation is required.

Table 11

Results of measurement invariance analysis in the register sample

Note. *p < .05, **p < .01, ***p < .001; Δχ² corrected by scale correction factor for MLR.

Table 12

Results of measurement invariance analysis in the Facebook sample

Note. *p < .05, **p < .01, ***p < .001; Δχ² corrected by scale correction factor for MLR.

Acknowledgement

This work was funded by German Science Foundation (DFG) in the scope of the ENSURE project as part of the PH-LENS Research Unit (FOR 2928/GZ: ME 3538/10-1).

Supplementary Material

S1: 13 Item SOEP-Version of the Short-Form Health Survey

Instruction

Gesundheit und Krankheit

Items

Response specifications SOEP

Luise Richter; Dresden University of Technology; Luise.Richter4@tu-dresden.de

Oliver Brust; Dresden University of Technology; Oliver.Brust@tu-dresden.de

Natalja Menold; Dresden University of Technology; Natalja.Menold@tu-dresden.de