CORRELATION BETWEEN WEEKLY FORMATIVE AND SUMMATIVE ASSESSMENT OF MEDICAL STUDENTS IN MULTIDISCIPLINARY EXAMINATION AND ORAL EXAMINATION REPRODUCTIVE SYSTEM BLOCK

Background: The impact of weekly formative assessments on the summative performance of undergraduate medical students is not widely studied. The formative assessment began to be applied in the Undergraduate Medical Study Program, Faculty of Medicine, Padjadjaran University (Program Studi Kedokteran Fakultas Kedokteran Universitas Padjadjaran or PSK FK Unpad) in early 2020. This study aims to determine whether there is a correlation between weekly formative and summative assessment in Multidisciplinary Examination (MDE) and Structured Oral Examination (SOE). Methods: This study was conducted using a numerical correlative bivariate analysis with a cross-sectional design. The data used in this study was weekly formative assessment scores, MDE and SOE scores of Reproductive System (RPS) I and II of 138 medical student batch 2017. Results: Rank-Spearman test shows there was no significant correlation between the formative test scores and the MDE RPS I score (p>0.05) and the SOE score (p>0.05). Meanwhile, the MDE RPS II score with the formative test was found to be a significant correlation (p<0.05) and positively correlated (r=+0.211). Conclusion: This study shows that the use of weekly formative assessments might contribute to the students’ achievement of MDE scores. Weekly formative assessments can be integrated into the curriculum of preclinical medical programs to improve student summative performance.


INTRODUCTION
Medical education has made tremendous progress in teaching and learning activities. Curricula, educators, and assessment systems are constantly being developed and updated to face future challenges. 1,2 These changes are made to improve the competence of every medical doctor to be able to provide optimal health services. 1,3 The competencies according to Peraturan Menteri Riset, Teknologi, dan Pendidikan Tinggi Republik Indonesia Nomor 18 Tahun 2018 are minimum criteria for medical doctor graduates that include attitudes, knowledge, and skills. 4 These competencies encourage a more comprehensive assessment in assessing aspects of students' knowledge and skills. 1 Each institution must have an assessment method that can describe the achievement of competencies following the Indonesian Doctor Competency Standards (Standar Kompetensi Dokter Indonesia or SKDI) agreed by Indonesian Medical Council (Konsil Kedokteran Indonesia or KKI) in 2012. 5 Competencies are achieved by the process of teaching and learning, supported by curriculum development, learning methods, and competent educators. Whether or not competence is achieved can be measured through assessment. In general, there are two types of assessment in medical education, summative assessment, and formative assessment. Summative assessment is used to assess the outcome of learning and is usually included in the cumulative grade point average (GPA). 2,6 In contrast to summative assessment, formative assessment focus on assessing students' learning progress compared to assessing final results. 7 Formative assessment aims to build lifelong learning habits by providing feedback from teachers on student performance, and vice versa. Feedback is provided so that students can understand their strengths and weaknesses with expectations that students will more easily identify materials that have not been mastered and must be studied. 6 The longterm goal of formative assessment is that students have intrinsic motivation to constantly improve their knowledge, skills, and learning behaviors. 7,8 Because each assessment is specific and equally important, the right combination of assessments is needed to achieve the required competency. 2,5 In medical education, several studies have shown that the application of formative assessment in learning systems can improve student summative assessment performance. Such as a study from the British Medical Center (BMC) Medical Education and the Association of Medical Education in the Eastern Mediterranean Region (AMEEMR) shows that repeated spaced formative assessments conducted using web-based can improve student summative assessment performance. 8,9 Other studies also stated that formative assessment had several significant effects on the performance of students' summative assessments from various aspects. 10 In Indonesia, several studies also showed similar results. For example, a study conducted by the Faculty of Medicine at Universitas Tarumanegara showed that the use of quizzes as a formative assessment could increase the percentage of block passing. 11 Another study conducted by the Faculty of Medicine at Universitas Islam Bandung also showed a positive correlation between self-assessment and OSCE graduation. 12 Faculty of Medicine, Padjadjaran University, particularly the Medical Study Program (PSK FK Unpad), has begun to develop formative assessments in the learning evaluation system. Since January 2020, formative assessments have been implemented in batch 2018 which is in the Cardiovascular and Respiratory System (CVS-RS) and batch 2017 which is in the Reproductive System I and II (RPS I-II). Some of the methods applied in this system are weekly formative MCQ, feedback on skills and lab activities, and weekly assignments. The selection of RPS I and II blocks were adjusted because of the completeness of the research data and changes in the placement of the blocks implemented in third-year students. Some of the studies described above certainly have different characteristics such as the availability of resources, differences in curriculum and policies, as well as facilities and infrastructure, so that they cannot fully represent the effect of formative assessment on summative assessment in FK Unpad. 6 Currently in Indonesia no research specifically addresses the relationship between formative assessment and summative performance assessment, particularly on written and oral exams.
This study aims to determine whether there is a relationship between weekly formative MCQ and summative assessment, especially in written exams in the form of one of the best answers called Multidisciplinary Examination (MDE) and oral examinations in the form of structured oral exams (SOE) which are components of summative assessment in PSK FK Unpad. The results of the study are expected to be one of the considerations for policymakers at the Faculty of Medicine, Padjadjaran University especially in the process of developing learning and assessment systems.

METHODS
This research was conducted from August to September 2020 after getting approval from the Research Ethics Committee of Padjadjaran University Bandung No.74/UN6.KEP/EC/2020. This study was a bivariate analysis of numerical correlation with a cross-sectional study design that used secondary data.
There were 283 students of batch 2017 PSK FK Unpad. The data used were weekly formative assessment scores, MDE and SOE scores for RPS I and II blocks. MDE and SOE are exams included in summative assessment at PSK FK Unpad. SOE aims to assess students' analytical abilities on a case learned in teaching and learning activities. Students will present one case randomly selected in front of the examiner. MDE aims to assess students' detailed knowledge related to basic science material and clinical science in each case and usually in the form of multiple-choice with one best answer.
The inclusion criteria were scores of students who took the weekly formative exams with a percentage of more than 80%, namely at least ten times out of a total of twelve formative exams, MDE and SOE score of RPS I and II blocks on the first exam and not remedial. Data was issued if the student is a student who repeats the learning blocks of RPS I and II and/or takes a follow-up exam. After a selection based on inclusion and exclusion criteria, 140 student data were obtained, with information that two of them had incomplete test scores and were not included in the data processing. Therefore, the total included 138 in statistical analysis were data from students.
All research data were obtained from the Academic Evaluation Unit, Faculty of Medicine, Padjadjaran University. This research used total sampling techniques. The statistical test was the Rank-Spearman test and processed with IBM ® SPSS ® version 20.

RESULTS AND DISCUSSION
The implementation of formative assessment was carried out voluntarily every week by students. The formative questions adjusted to the learning indicators every week. There were twelve formative examinations according to the number of cases in RPS I and block II, each block had six cases. The form of the formative exam questions was multiplechoice with one best answer and multiple answers. There was no feedback given by educators on this weekly formative exam. The characteristics of the formative exam in RPS I and II blocks such as student participation, average test scores, and descriptions of the feedback shown in Table 1.  Table 3. Weekly formative and SOE scores RPS I+II block were tested with the Kolmogorov-Smirnov normality test and showed an abnormal distribution because the significance value of one of the data was 0.003 (p<0.05). Due to the abnormal distribution of the data, the statistical analysis used was Rank-Spearman. The p value=0.329 obtained from the results of the analysis using the SPSS instrument used to accept the null hypothesis. Thus, the results of the analysis show that the mean weekly formative scores of RPS I+II do not correlate with the mean SOE scores of RPS I+II. The results of processing data on oral examination scores structured in RPS I+II blocks can be seen in Table 4. In this study, it was found that the application of weekly formative assessments in teaching and learning activities of RPS I and II had various outcomes. A significant correlation was found in the MDE RPS II, while the MDE RPS I and the SOE RPS I+II did not have a significant correlation. Although several previous studies have shown that the application of formative assessment can improve the achievement of students' summative assessments, several factors influence the success of formative assessment in each institution. 11 Formative assessment aims to build intrinsic motivation of students needing feedback as the key. This feedback is given by educators, peers, or themselves. The purpose of feedback is for the students to be able to conduct self-evaluation to make quality improvements. The quality of the feedback should be considered. Feedback will be effective if given immediately, topic-based, obvious, and constructive. Feedback is usually qualitative and given face-to-face or online. Effective feedback can motivate students to identify their strengths and weaknesses to make improvements in achieving competency in learning indicators. 9,11 If there is no feedback, students will find it difficult to identify learning topics that they have not mastered. The absence of providing feedback on formative assessments in this study might be one reason why this weekly formative did not give significant results on students' summative scores, as in the MDE block RPS I which did not show a significant relationship with the weekly formative exam.
In addition to the lack of feedback, the form of the question tested in the weekly formative might also affect. Each form of question has its purpose. The choice of multiple-choice instruments is aimed at testing knowledge and problem-solving skills, while the short form and essays are aimed at assessing clinical reasoning, interpretation of the diagnosis, theoretical basis and material principles, and problem-solving abilities. 7,10 Meanwhile, observerbased exam such as the oral exam aims to test the analytical, communication, and presentation skills of students. 7,13 In previous research, it was found that the form of formative assessment that can improve student summative assessment performance is varied but still adjusted to the summative exam form. For example, questions in the form of MCQ, short fills, and structured essays can hone learning material in more detail and increase long-term retention abilities, this form tends to influence students' abilities in written examinations. Also, formative exams in the form of self-assessment and peer-assessment are useful for increasing student confidence and communication skills so that they tend to influence the OSCE exam. 8,11,12 The questions tested in the weekly formative assessment block RPS I+II are designed to resemble MDE written exam questions, such as multiple choice with a one-best or multiple answers. Seeing the objectives of each different type of exam and the form of weekly formative assessment questions that are more similar to the MDE, making the application of weekly formative assessments has little effect on the outcome of the structured oral exam. It is proven that the SOE blocks RPS I and II do not show a significant relationship between SOE scores and weekly formative exam scores.
A study shows that the success of summative assessments is more determined by the level of student participation in formative assessments than the results of student formative assessments. 11 In the characteristics of the formative assessment in table 1, the mean level of student participation in the formative assessment of RPS II is decreased compared to that of RPS I but a significant relationship is found in MDE block RPS II. So the decrease in the level of participation cannot underlie the assumption that if the level of student participation is lower in the RPS II block, their summative assessment performance will be lower. In a previous study, it is known that the relationship or effect of formative assessment will be more visible in the comparison of groups of students who have never participated in formative assessments with diligent students. 8 The significant relationship in MDE RPS II can be caused by several things. First, there is continuity between RPS I and II block learning materials. Students have obtained the basics of the theory that can support understanding at RPS II. This understanding is strengthened by the repetition of material when students work on weekly formative exam questions. This is following several studies that show that formative assessments that are carried out repeatedly with a time interval can improve student summative assessment performance because it helps to understand the material in-depth and in more detail. 6,8 Long-term retention of students will increase due to the process of recalling and retrieving learning materials. Second, formative assessments that are carried out regularly every week can increase students' intrinsic motivation in learning to get better scores on the next formative exam. Students also have experience in working on formative RPS I questions, so that students will be more accustomed to working on questions in the RPS II block. 14 Finally, the emergence of the COVID-19 pandemic has also contributed to teaching and learning activities in Indonesia. One of the steps taken in the world of education to reduce the rate of spread of COVID-19 is by changing the learning system to online learning. Several factors affect the readiness of this new system, some of which are the readiness of educators in developing learning and evaluation methods, provision of learning devices, internet quotas, and mental readiness. High adaptability is needed to harmonize the rhythm of teaching and learning activities during the pandemic. [15][16][17] Since March 2020, the learning system at PSK Unpad has also shifted to online learning. Students study independently at home, and an evaluation system is carried out remotely. This condition requires the ability to adapt quickly under pressure, either because they are worried about the transmission of COVID-19 or because they are not used to studying independently without friends. 17 Besides, the pandemic caused the delay in the MDE RPS I exam. The delay in this exam time resulted in students taking exams in the week of RPS II block teaching and learning activities taking place. So even though they had a long time to prepare for exams, students found it difficult to focus on preparing for the MDE RPS I exam because they were probably distracted by the material and assignments that must be studied during the RPS II block. Finally, the pandemic may have an indirect effect on student performance in the MDE RPS I exam. Because several factors that affect student academic achievement is study habits, student stress and anxiety levels, and sufficient time to study exam material. 11 In block RPS II, student adaptation to COVID-19 tends to improve. Weekly participatory assessments help motivate students to learn by practicing questions independently with their respective awareness because the formative examination is voluntary. This voluntary work indicates that students who take the formative assessment of RPS II are students who have high intrinsic motivation. 8,9 Student preparation would have been better in studying the RPS II block material because of the sufficient time lag between the end of teaching and learning activities and the MDE RPS II exam. The repetition of weekly formative questions might help students prepare to understand the material. This could be another possible explanation why weekly formative assessments can have a positive impact on the achievement of the MDE RPS II.
The limitation of this study is that it does not consider whether the MDE summative assessment items are questions that have been tested in previous years. The repetition of this question will have an impact on students' ability to do MDE questions. This study also cannot represent the effect of weekly formative assessments on other learning blocks.

CONCLUSION
Weekly formative assessments can have a positive effect on the summative assessment performance of medical students in the RPS block, namely the MDE written exam. The results of this research can be used as consideration for policymakers at the Faculty of Medicine, Padjadjaran University in making academic policies regarding learning and assessment systems.

RECOMMENDATION
Based on this research, if the faculty conducts formative examinations it may also include giving immediate feedback. In addition, this research could be further developed to consider the level of student participation in weekly formative assessments. Further studies are also needed to investigate which factors can influence the summative assessment performance of medical students in addition to the existence of formative assessments to provide more comprehensive research results.