Understanding the Method Effects of the Ethnic Identity Scale Items

Qothrun Nada; Sukaesi Marianti

doi:10.22146/jpsi.63670

Understanding the Method Effects of the Ethnic Identity Scale Items

Qothrun Nada, Sukaesi Marianti

(Submitted 27 January 2021)

(Published 24 December 2021)

Abstract

This study used the Correlated Traits-Correlated Method (CTCM) model to identify the method effects of the Ethnic Identity Scale (EIS) items, both in favorable and unfavorable items. The study involved 4 alternative models of the CTCM to test the suitability of the model. 440 secondary-school students were involved in this study. Confirmatory Factor Analysis was used employing R software Version 4.0.2. Results indicated alignments between the data with three CTCM alternative models that provided evidence to understand the effect of the method on the use of positive and negative items that could affect the validity of a measuring instrument.

Keywords

confirmatory factor analysis; correlated traits-correlated methods; method effect; positive and negative items

Full Text: PDF

DOI: 10.22146/jpsi.63670

References

Azzahrah, F. (2017). Karakteristik Psikometrik Skala Identitas Etnis Dilihat Melalui Pendekatan Delayed Alternate Form pada Remaja Awal dan Tengah. Universitas Brawijaya Malang.

Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological Bulletin, 56(2), 81–105. https://doi.org/10.1037/h0046016

de Lima, T. J. S., & de Souza, L. E. C. (2019). Rosenberg self-esteem scale: Method effect and gender invariance. Psico-USF, 24(3), 517–528. https://doi.org/10.1590/1413-82712019240309

DeVellis, R. F. (2016). Scale Development Theory and Applications (Fourth Edition). SAGE Publication, 4, 256.

DiStefano, C., & Motl, R. W. (2006a). Further investigating method effects associated with negatively worded items on self-report surveys. Structural Equation Modeling, 13(3), 440–464. https://doi.org/10.1207/s15328007sem1303_6

DiStefano, C., & Motl, R. W. (2006b). Further investigating method effects associated with negatively worded items on self-report surveys. Structural Equation Modeling, 13(3), 440–464. https://doi.org/10.1207/s15328007sem1303_6

DiStefano, C., & Motl, R. W. (2009a). Personality correlates of method effects due to negatively worded items on the Rosenberg Self-Esteem scale. Personality and Individual Differences, 46(3), 309–313. https://doi.org/10.1016/j.paid.2008.10.020

DiStefano, C., & Motl, R. W. (2009b). Self-esteem and method effects associated with negatively worded items: Investigating factorial invariance by sex. Structural Equation Modeling, 16(1), 134–146. https://doi.org/10.1080/10705510802565403

Fiske, D. W. (1982). Convergent-discriminant validation in measurements and research strategies. New Directions for Methodology of Social & Behavioral Science, 12, 77–92.

Greenberger, E., Chen, C., Dmitrieva, J., & Farruggia, S. P. (2003). Item-wording and the dimensionality of the Rosenberg Self-Esteem Scale: Do they matter? Personality and Individual Differences, 35(6), 1241–1254. https://doi.org/10.1016/S0191-8869(02)00331-8

Lindwall, M., Barkoukis, V., Grano, C., Lucidi, F., Raudsepp, L., Liukkonen, J., & Thgersen-Ntoumani, C. (2012). Method effects: The problem with negatively versus positively keyed items. Journal of Personality Assessment, 94(2), 196–204. https://doi.org/10.1080/00223891.2011.645936

Mangundjaya, W. L. H. (2013). Is There Cultural Change in the National Cultures of Indonesia? Steering the Cultural Dynamics, 59–68.

Marsh, H. W. (1996). Positive and Negative Global Self-Esteem: A Substantively Meaningful Distinction or Artifactors? Journal of Personality and Social Psychology, 70(4), 810–819. https://doi.org/10.1037/0022-3514.70.4.810

Marsh, H. W., Scalas, L. F., & Nagengast, B. (2010). Longitudinal tests of competing factor structures for the Rosenberg self-esteem scale: Traits, ephemeral artifacts, and stable response styles. Psychological Assessment, 22(2), 366–381. https://doi.org/10.1037/a0019225

Molt, R. W., & Conroy, D. E. (2000). Social Physique Anxiety Scale (SPAS). Psychobiology and Social Sciences, 32(5), 1007–1017.

Motl, R. W., & DiStefano, C. (2002). Longitudinal invariance of Self-Esteem and method effects associated with negatively worded items. Structural Equation Modeling, 9(4), 562–578. https://doi.org/10.1207/S15328007SEM0904_6

Paramita, W. M., Permatasari, D. P., Rahajeng, U. W., Psikologi, P. S., & Malang, U. B. (2014). Perbedaan identitas etnis pada remaja etnis jawa dan etnis tionghoa. Universitas Brawijaya Malang.

Podsakoff, P. M., MacKenzie, S. B., & Podsakoff, N. P. (2012). Sources of method bias in social science research and recommendations on how to control it. Annual Review of Psychology, 63, 539–569. https://doi.org/10.1146/annurev-psych-120710-100452

Pohl, S., Steyer, R., & Kraus, K. (2008). Modelling method effects as individual causal effects. Journal of the Royal Statistical Society. Series A: Statistics in Society, 171(1), 41–63. https://doi.org/10.1111/j.1467-985X.2007.00517.x

Rodrigo, M. F., Molina, J. G., Losilla, J. M., Vives, J., & Tomás, J. M. (2019). Method effects associated with negatively and positively worded items on the 12-item General Health Questionnaire (GHQ-12): Results from a cross-sectional survey with a representative sample of Catalonian workers. BMJ Open, 9(11). https://doi.org/10.1136/bmjopen-2019-031859

Tomás, J. M., & Oliver, A. (1999). Rosenberg’s self-esteem scale: Two factors or method effects. In Structural Equation Modeling (Vol. 6, Issue 1, pp. 84–98). https://doi.org/10.1080/10705519909540120

Tomás, J. M., Oliver, A., Galiana, L., Sancho, P., & Lila, M. (2013). Explaining Method Effects Associated With Negatively Worded Items in Trait and State Global and Domain-Specific Self-Esteem Scales. Structural Equation Modeling, 20(2), 299–313. https://doi.org/10.1080/10705511.2013.769394

Umaña-Taylor, A. J. (2004). Ethnic identity and self-esteem: Examining the role of social context. Journal of Adolescence, 27(2), 139–146. https://doi.org/10.1016/j.adolescence.2003.11.006

Umaña-Taylor, A. J., & Shin, N. (2007). An examination of ethnic identity and self-esteem with diverse populations: Exploring variation by ethnicity and geography. Cultural Diversity and Ethnic Minority Psychology, 13(2), 178–186. https://doi.org/10.1037/1099-9809.13.2.178

Vinet, L., & Zhedanov, A. (2011). A “missing” family of classical orthogonal polynomials. In Journal of Physics A: Mathematical and Theoretical (Vol. 44, Issue 8). https://doi.org/10.1088/1751-8113/44/8/085201

Widhiarso, W. (2016). Peranan Butir Unfavorable Dalam Menghasilkan Dimensi Baru Dalam Pengukuran Psikologi. Jurnal Psikologi Perseptual, 1(1), 1–11. https://doi.org/10.24176/perseptual.v1i1.1078

Widhiarso, W. (2017). Mengakomodasi Efek Metode dalam Pengujian Validitas Konstruk Melalui Analisis Faktor Konfirmatori. Psikologia : Jurnal Psikologi, 1(1), 37. https://doi.org/10.21070/psikologia.v1i1.478

Wu, C. H. (2008). An examination of the wording effect in the Rosenberg self-esteem scale among culturally Chinese people. Journal of Social Psychology, 148(5), 535–552. https://doi.org/10.3200/SOCP.148.5.535-552

Wu, Y., Zuo, B., Wen, F., & Yan, L. (2017). Rosenberg Self-Esteem Scale: Method Effects, Factorial Structure and Scale Invariance Across Migrant Child and Urban Child Populations in China. Journal of Personality Assessment, 99(1), 83–93. https://doi.org/10.1080/00223891.2016.1217420

Zeng, B., Wen, H., & Zhang, J. (2020). How Does the Valence of Wording Affect Features of a Scale? The Method Effects in the Undergraduate Learning Burnout Scale. Frontiers in Psychology, 11(September), 1–12. https://doi.org/10.3389/fpsyg.2020.585179

Refbacks

There are currently no refbacks.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.