Psychometric Properties of Self-Report Questionnaire-20 (SRQ-20) Indonesian Version

. The prevalence of mental disorders experienced by the global community, including in Indonesia, is increasing steadily. It is thus necessary to have a time-efﬁcient and valid screening measuring tool as a basis to provide appropriate treatment to mental disorders. The Self-Report Questionnaire-20 (SRQ-20) is a screening tool that is often used in Indonesia to identify mental disorder symptoms. However there is limited study that has attempted to discover the SRQ-20 factors and conducted a construct validity test on them. The purpose of this study was to investigate factors and test the construct validity of SRQ-20 developed by the World Health Organization (WHO). There were 674 students involved in the study. Exploratory factor analysis was used to discover the factors. Evidence based on internal structure using a conﬁrmatory factor analysis was also collected. Findings revealed ﬁve factors for the SRQ-20, these are energy, cognitive, depression, physiological, and anxiety. The ﬁve factor model is an alternative for classifying symptoms on the SRQ-20. This way, symptoms on the SRQ-20 can be grouped into ﬁve factors, of which can assist clinicians take more appropriate and faster action. Findings also revealed, however, that the construct validity of the ﬁve factor model needs is less than the one factor model. Therefore, for further research, using a wider population to check the stability of structure across developmental age. SRQ-20 could also be compared with the gold standard measurement to get a predictive validity. validity of the SRQ-20 was collected based on relations to other variables and as a screening In China, the SRQ-20 results were compared with the results of psychiatric interviews using the Composite International Diagnostic Interview (CIDI) guidelines and ran ROC analysis, showing good results with an AUC value of 0.83. Besides that, the study used Principal Factor Analysis to ﬁnd out SRQ-20’s factors. The study found three factors, consisting of depressive symptoms, feeling of anxiety and somatic symptoms, and anxiety symptoms. In Eritrea, the SRQ-20 was also compared with a valid diagnostic measuring tool, namely the Brief Psychiatric Rating Scale. The results of the two measuring instruments were correlated and represent a moderate level of correlation. The AUC value from the ROC analysis was 0.87. This study found that there are two factors in the SRQ-20 (Netsereab 2018).

The several mental disorders experienced by individuals can cause various negative impacts, such as having difficulty making the right decisions because of the possible disruption in cognitive-emotional functions (Oltmanns & Emery, 2019). These individuals tend to withdraw from their social surroundings, leading to risk of suicide due to lack of social affiliation support which causes their affiliation needs to be unfulfilled and could even promote suicide (Kearney & Trull, 2018).
Individuals with mental disorders are also more susceptible to physical illness, so these individuals cannot function optimally on their work/activity (Oltmanns & Emery, 2019;Taylor, 2015).
Mental disorders are a common phenomenon among university students. A survey by American College Health Association (2018) in the United States showed that 18.1% of University students in America had depression, 22.1% had anxiety disorders, and 11% experienced panic attacks.
A study by Auerbach et al. (2018). Research at 19 Universities in 8 Countries also showed that 31.4% of students experienced at least one mental disorder). Studies of Indonesian students resulted in a similar trend. Research in Banda Aceh, Indonesia found that 47.7% of university students experienced depression and 27.4% experienced anxiety (Marthoenis et al., 2018). Research at Universitas Udayana, Bali also found that 29.31% of new students experienced clinical symptoms that lead to major depression (Karin, 2017).
Taking into account the high prevalence of mental disorders especially in university students, it is crucial to have a valid and effective measuring tool to identify symptoms of mental disorders among university students (Ali et al., 2016;Siceloff et al., 2017). Such tools can enhance preventive and/or curative actions that can be taken immediately to these individuals so that they can develop more optimally.
This study focuses on investigating Self-Report Questionnaire 20 (SRQ-20) developed by the WHO to understand the validity and its measurement factors. To date, there have been several researches in this topic area, for example Ganihartono (1996) used criterion-related validity (i.e. Present State Examination) as a standard to test the validity of the SRQ-20. Sartono (2010) study has also collected evidence of reliability using the test-retest method. Nevertheless, there remains a need to explore and further validate SRQ-20 in Indonesia context. According to the American Educational Research Association, American Psychological Association, & National Council on Measurement Education (2014), validity is a process to accumulate relevant evidence that can support the process of interpreting a score. To the best of our knowledge, the Indonesian version of the SRQ-20 has one evidence of validity in the form of criterion-related validity by Ganihartono (1996). This study will, thus, fill this gap by collecting evidence of internal structure (construct validity) using the confirmatory factor analysis method. We will further investigate the factor structure of SRQ-20 using exploratory factor analysis.

Literature Review
Self-Report Questionnaire 20 (SRQ-20) is a mental disorder measuring tool that has been widely used to assess individuals (WHO, 1994). It is often used to identify mental disorders symptoms in individuals.

70
JURNAL PSIKOLOGI Prasetio et al Psychometric Properties of Self-Report This measuring tool cannot be used as a substitute for a diagnosis from a psychologist or psychiatrist, however, it can be taken into consideration by mental health medical personnel (WHO, 1994).
SRQ-20 was originally developed by WHO and several collaborating countries (WHO, 1994) to assist developing countries in overcoming mental health care issues. At the time, the number of mental health services in many developing countries remained very low, which caused knowledge on mental disorders and mental health services was limited. It was deemed necessary to have a measuring tool to quickly identify individuals in need of mental health services (WHO, 1994).
The development of SRQ-20 was not based on a particular theory (or atheoretical), but instead by comparing items on four screening instruments to measure mental disorders. The four instruments The validity evidence collection for SRQ-20 has been carried out in several countries and has even attempted to discover the probable factors that underlying SRQ-20 (Chen et al., 2009;Kootbodien et al., 2015;Netsereab et al., 2018;Scholte et al., 2011;van der Westhuizen et al., 2015). The
In Indonesia, many attempts have been made to carry out the translation process and the use of this measure. The earliest research found was a study in Semarang (Ganihartono, 1996)  There are only two studies to date investigating the validity of the SRQ-20 in the Indonesian version, research in Semarang (Ganihartono, 1996) and Jakarta (Sartono, 2010). In Ganihartono (1996) research, the validity evidence obtained was based on relations to other variables, then analyzed using the ROC curve. Sartono (2010) research used a test-retest as evidence of collecting reliability and validity. These studies indicated that SRQ-20 is a good measure for identifying mental disorders.
However, collecting the validity evidence of SRQ-20 in the Indonesian version is still required, especially in collecting evidence of validity based on internal structure and psychometric properties.
In addition, there is no published research so far that has tried to examine or investigate the factor structure of SRQ-20 in the Indonesian population. These two requirements need to be met considering that mental disorders are sensitive to cultural differences (Kearney & Trull, 2018), and can increase diagnostic efficacy, and also provide useful information on symptoms of mental disorders in the Indonesian population (Stratton et al., 2013). The study is thus aimed to fill the gap in the literature to collect validity evidence from the internal structure and psychometric properties and investigate the factor structure of the SRQ-20.

Participants
The study took place in Universitas Padjadjaran, West Java, Indonesia, of which undergraduate students enrolled in 2017 -2019 were the population of this study spread across 16 Faculties. We used stratified random sampling with 3 strata (i.e. academic year). From each strata, we collected

Procedure
The measuring instrument used is the Self-Report Questionnaire-20 (SRQ-20) constructed by WHO (1994). The SRQ-20 has been translated into Indonesian by Pusat Studi Psikometri, Asesmen, dan Evaluasi (Center of Psychometrics, Assessment, and Evaluation) Universitas Padjadjaran. SRQ-20 consists of 20 questions related to symptoms of neurotic disorders. Respondents were asked to remember whether these symptoms had been experienced during the last 30 days. Each item consists of two answer choices, "yes" and "no". "Yes" answer will be given a score of 1 and "No" answer will be given a score of 0. In a pilot study, the researchers administered the SRQ-20 to 16 undergraduate students. In this process, we checked all of the terms on the questions and made some adjustments to increase clarity of the questions.
There are two steps of procedure. The first step was to investigate the factor structure of the SRQ-20 using exploratory factor analysis (EFA). Below is the step-wise process of how EFA was carried out (Hair et al., 2019): 1. Examined some assumptions before EFA was carried out. In the EFA process, prerequisites must be met: The Bartlett test of sphericity must be significant (p <0.05), and the measure of sampling adequacy (MSA) value on the overall test and each individual variable must be less than 0.05.
2. Performed the EFA process using particular software. In this study, the software used was SPSS ver.23. This study involved the Maximum Likelihood approach, and the rotation factor used is varimax.
3. Determined the number of factors, and assessed overall fit. Two criteria were used to determine the number of factors: the eigenvalues > 1, and the inflection point on the scree test.
4. Interpreted factors. The factor loading of each item is at least 0.3. Each item can only have a factor loading greater than 0.3 on just one factor. Then, if each factor has gotten its items, each factor must be given a certain label.
5. Validated the EFA results through confirmatory factor analysis (CFA).
In the second step, the researchers collected validity evidence based on the internal structure. The stages were as followed (Hair et al., 2019): 1. Defined psychological constructs. Mental disorders are defined as signs or symptoms that occur in individuals, and these symptoms appear simultaneously and within a particular time.
2. Developed measurement models. This study examined two measurement models: one-factor of SRQ-20 (WHO, 1994) because it is assumed that general mental disorders will be measured without any distinct factors, and the other was factor structure that had been processed in the EFA process.
3. Designed a study to produce empirical results. The researcher had a population of students who enrolled at Universitas Padjadjaran from the academic year of 2017-2019. The sampling technique used was stratified random sampling with three strata (academic year of enrolment), then proportioned in 16 faculties. The total sample was 674 students.
4. Assessed measurement model validity. There are several parameters that can be used to determine goodness-of-fit, namely p-value > 0,05; X2/df ratio (Relative Chi-Squared) ≤ 3, GFI ≥ 0.95; RMSEA ≤ 0.07; NNFI ≥ 0.90; CFI ≥ 0.90; SRMR < 0.08 (Hair et al., 2019;Hopper et al., 2008;Kline, 1998Kline, , 2005. It is also necessary to look at the significance of factor loading, that is t-values. If t-values ≥ 1.96, then a factor loading can be claimed to be significant on the latent variable/construct. AVE (Average variance extracted) is also calculated to know about average communality of each latent variable (Hair et al., 2019). Acceptable value of AVE ≥ 0.5.
The internal structure validity was collected using Lisrel 8.8 software with Diagonally Weighted Least Squares (DWLS) estimation for ordinal data. The reliability of the SRQ-20 was measured using two approaches, namely Cronbach's alpha and composite reliability (CR). The limit of the reliability value can be said to be good when it is above 0,70 (Hair et al., 2019).

Results
The results of Exploratory Factor Analysis (EFA) using maximum likelihood estimation and varimax rotation with Kaizer normalization on the SRQ-20 measuring instrument resulted in five factors, which can be seen in table 1. In the EFA process, three items were not included in the factor, those were items 8 (Apakah Saudara mengalami kesulitan untuk berpikir jernih? Do you have trouble thinking clearly?), 15 (Apakah Saudara kehilangan minat untuk melakukan berbagai macam hal? Have you lost interest in things?), and 10 (Apakah Saudara lebih sering menangis dari biasanya? Do you cry more than usual?). Item 10 was not included in the factor because the factor loading value for each factor did not meet the specified minimum limit, which is below 0.30. Item 8 and 15 were decided not to be included in any factor because they had two factor loading values above 0,30 in two different factors. This is referred to as cross-loading. If this is the case, Hair et al. (2019) suggested calculating the difference in variance between these loadings. If the difference value is below or equal to 2, then the item does not need to be used. If above 2, then the smaller factor loading value can be ignored for interpretation purposes.
The next step was to validate the results obtained from Exploratory Factor Analysis using Confirmatory Factor Analysis. The estimation method used was Diagonally Weighted Least Square (DWLS). The one-factor model consisting of 20 items (WHO, 1994) had also been tested using CFA.
The goodness-of-fit results are shown in table 2.  Figure 1 presents the three models (one, one-modification, and five) of SRQ-20.
In addition to using the goodness-of-fit value as model determination, it is also necessary to know the significance value of the factor loadings of each observed variable or question item to the construct/latent variable being measured. Table 3 presents the factor loadings and t-values of each question item. Table 3 also presents Cronbach's alpha reliability and composite reliability of each factor.
Regarding standardized loading, t-values, Cronbach's alpha, and composite reliability, it was found that in the one-factor model and the modified one-factor model, all factor loadings were significant in the construct of general mental disorders. This can be interpreted that each item can significantly measure the construct of general mental disorders. In the five-factor model, it was found that in factor 1 all items had an insignificant factor loading. Then in factor 2, there was one item that had an insignificant factor loading (t-values ≤ 1.96). Meanwhile, other items had a significant factor loading. Cronbach's alpha in the one-factor model was classified as good, which was 0.84. However, in the five-factor model, each factor had a Cronbach's alpha below 0.7. The composite reliability in both models was classified as good because it had a value above 0.70.

Table 3
Standardized

Discussion
This research is one of the first studies in Indonesia that attempted to analyze the factor structure of the SRQ and test the existing model using CFA. The results of the study found that the Indonesian version of the SRQ-20 had five factors: Energy, Cognitive, Depression, Physiological, and Anxiety. The results of testing the one-factor model using CFA (WHO, 1994) and EFA results of the five-factor model suggested that the SRQ-20 is good enough to measure general mental disorders, without or with the factors.
The EFA process indicated five factors that suggest the structure of the SRQ-20 Indonesian version. This finding adds into an existing literature on the attempts to investigate the SRQ-20 factor structure that have been carried out in several countries (Iacoponi & de Jesus Mari, 1989;Kootbodien et al., 2015;Netsereab et al., 2018), which to date resulted in 2-7 factor structures, depending on the local culture context.
Findings also indicated that the EFA results are also supported by the CFA results. It was found that the goodness-of-fit indicators met acceptable values, i.e., the values of GFI, CFI, NNFI, SRMR, RMSEA, and relative chi-squared. However, there remained l goodness-of-fit indicators that did not meet criteria, i.e., chi-square. The significance of factor loadings (t-values) of each item on a certain factor also needs to be considered. In the five-factor model, two factors (Energy and Cognitive) had items with insignificant loading.
Energy (factor 1) represents some characteristics of people who experience a decreasing physical/psychological energy. It is characterized by frequent headaches, difficulty enjoying daily activities, and feeling tired. Cognitive (factor 2) describes some symptoms about cognitive problems, such as decreasing quality of work's result, loss of interest, and difficulty to make decisions. Depression (factor 3) describes some emotional and cognitive symptom which related to depression, such as feel unhappy and worthless, and think to suicide. Physiological (factor 4) characterized by poor digestion.
It represents the impact of psychological disorders on digestion. Anxiety (factor 5) represents some physiological and psychological symptoms which are related to anxiety, such as feeling tense or frightened, and hands shaking.
This study also continued to analyze the one-factor model of the SRQ-20. The CFA results of the one-factor model were also good, which can be observed from quite a lot of acceptable values of the goodness-of-fit criteria (NNFI, CFI, GFI, and RMSEA scores). There were still indicators that did not meet the criteria (chi-square, AVE, and SRMR values). In the one-factor model, modifications have been made. Modifications were made by correlating the covariance error value between items: 20 and 18 (Are you easily tired? and do you feel tired all the time), 6 and 4 (do you feel nervous, tense or worried? And are you easily frightened?), 16 and 9 (do you feel that you are a worthless person? And do you feel unhappy?). The modifications showed these items have a certain correlation. When compared with the five-factor model, the results of the modifications carried out show a similarity.
Items 20 and 18 were at factor 1, items 6 and 4 were at factor 5, and items 16 and 9 were at factor 3 in the five-factor model. This finding reaffirms that the presence of symptoms can cluster to form a 80 JURNAL PSIKOLOGI Prasetio et al Psychometric Properties of Self-Report specific factor.
One of them was the chi-square value. The chi-square test is used as a comparison of the observed covariance matrices with the expected covariance matrices. The chi-square value should not be significant, which means the two matrices do not have a significant difference (Hair et al., 2019). The disadvantage of using the chi-square test is that it is very sensitive to the number of samples (Hair et al., 2019). The SRMR (Standardized Root Mean Squared) value shows the standardized error value between the observed and expected covariance matrices. The greater the SRMR value, the greater the difference between the two matrices (Hair et al., 2019). In model 1, the SRMR value was still quite large, meaning that there is still a difference between the two matrices. The AVE (Average Variance Extracted) value shows how much the average variance value of all items in a construct is. Based on Hair et al. (2019), acceptable value of AVE is higher or same with 0,5. The greater the value of the variance, the smaller the error, and states that the item is better for measuring the construct (Hair et al., 2019).
The reliability of all tested models was also good. There were two types of reliability used, namely Cronbach's alpha and composite reliability (CR). The two types of reliability have different assumptions. Cronbach's alpha assumes that all items have the exact same contribution to each latent variable (Peterson & Kim, 2013). Meanwhile, the calculation of composite reliability takes into account the unique contribution of each item on the latent variable (Peterson & Kim, 2013). The use of composite reliability on the SRQ-20 is rarely done, as most studies commonly use Cronbach's alpha (Netsereab et al., 2018;Prasetio et al., 2019;Stratton et al., 2013). For CFA, composite reliability is suggested to use, because the construct loadings or weights are allowed to vary and it is used as the basis for estimating the reliability value, whereas the loadings for Cronbach's Alpha are constrained to be equal (Peterson & Kim, 2013). The results of both types of reliability indicated that the SRQ-20 as a measuring instrument can be trusted. This finding is also in line with several previous studies stated that the SRQ-20 has a good reliability (Kootbodien et al., 2015;Prasetio et al., 2019; van der Westhuizen et al., 2015) (Kootbodien et al., 2015;Prasetio et al., 2019;Triwahyuni & Prasetio, 2021;van der Westhuizen et al., 2015).
Theoretically, the existence of correlated symptoms is a very reasonable thing. Based on the statement of Oltmanns and Emery (2019), mental disorders are a collection of biopsychosocial symptoms that occur simultaneously within a certain time in an individual. This definition is one of the reasons for the correlations between items on the SRQ-20 -which represent symptoms of mental disorders -and support the possibility of a special grouping of clinical symptoms that represent certain mental disorders or special symptoms.
The one-factor model, the modified one-factor model, or the five-factor model are good enough to be used as the basis for the interpretation of the SRQ-20. The one-factor model states that all items on the SRQ-20 measure only general mental disorders. This is in line with the assumption from WHO (1994) that the SRQ-20 only measures general mental disorders. The general mental disorders referred to are neurotic disorders, such as depression and anxiety (World Health Organization, 2017). The five-factor model is also still good enough to be used as a reference to interpret symptoms more specifically.
In SRQ-20, it is obvious that there are contributing cultural factors. This has been observed from several studies in various countries. These studies tried to analyze the number of factors or dimensions on the SRQ-20. They produced different numbers of factors, or the same number but the questions that describe these factors are different (Chen et al., 2009;Kootbodien et al., 2015;Netsereab et al., 2018;Paraventi et al., 2015;van der Westhuizen et al., 2015). In the SRQ-20 manual (WHO, 1994), it is also confirmed that the SRQ-20 is very likely to have more than one factor or dimension, but it depends on the population and local culture. Thus, WHO did not make certain dimensions or factors for the SRQ-20 measuring instrument. This study also confirmed, one factor model (WHO, 1994) has a good internal structure validity and can be used to assess mental disorders.
This study has several limitations, such as the sample was only taken among students with 19-23 years age range. This could affect the level of generalization to other age groups. Then, the five-factor model that has been found seems to still be unfit, because there were factors in which loading items were not significant. This indicates that the five-factor model found is not precise enough and needs further study.

Conclusion
This research is one of the first studies to examine the factor structure of the SRQ-20 and to collect the psychometric properties of the SRQ-20. The one-factor SRQ-20 model states that the construct being measured is a common mental disorder, while the five-factor model states that there are specific factors that can help psychologists to take more targeted actions. Both models are good to use as a reference for interpretation. For the five-factor model, it is still necessary to carry out further analysis and use a wider population.

Recommendation
Further study can be carried out on a different and wider population, such as middle adulthood or late adolescence, to check the stability of structure across developmental age. Besides that, to make a comparison to other countries or cultures, a one-factor model can be used. Furthermore, to add validity evidence and to discover the predictive power of the SRQ-20 Indonesian version, SRQ-20 can be compared with other gold-standard measuring tools in diagnosing mental disorders, such as the MINI-International Neuropsychiatric Interview, Composite International Diagnostic Interview (CIDI), or Brief Psychiatric Rating Scales. SRQ-20 is a dichotomous scale. SRQ-20 is a dichotomous scale.
Conducting research about the possibility of a polytomous scale (5 Likert scale) for SRQ-20 is also recommended. The Five-Likert scale can be provide more visible differences in the severity of the mental disorder in individuals.