Instrument Development for Study Programs Excellence Evaluation Based on Kirkpatrick Evaluation Model

https://doi.org/10.22146/jpki.91960

Surya Akbar(1*), Tezar Samekto Darungan(2), Dinda Saufia Rahma(3)

(1) Medical Education Unit, Universitas Islam Sumatera Utara, Medan
(2) Medical Education Unit, Universitas Islam Sumatera Utara, Medan
(3) Faculty of Medicine, Universitas Islam Sumatera Utara, Medan
(*) Corresponding Author

Abstract


Background: The study programs excellence stated in the institution's vision and mission acts as a guide for the institution in carrying out its activities. That excellence must be reflected in all aspects of academic activities. Evaluation of the implementation and achievement of study program’s excellence is important to fulfill the vision, and mission of the institution. This research aims to develop an evaluation instrument that is useful for monitoring the achievement of study program’s excellence. 

Methods: Two part of instrument development were carried out in this study, namely: instrument development, and instrument validation process. Instrument development part consist of two stage (item and scale development, and expert judgement). Meanwhile, instrument validation process part consist of 5 stages of comprehensive validation process by Sireci. This validation process used argument based validation concept. 

Results: This instrument uses the Kirkpatrick model which consists of 4 evaluation levels. Each level consists of several dimensions. A total of 55 instrument items were proven valid (r>0.217) and reliable (Cronbach’s Alpha>0.7). The instrument uses 2 types of data in the assessment process, namely primary data (measuring responses from the academic community using a questionnaire), and secondary data (assessment of document evidence). 

Conclusion: The instrument evaluation is prepared for the purpose of providing an overview of the implementation and achievement of institutional’s excellence. This is useful in providing valuable information to the institution to set strategies and policies to fulfill the institution's excellence.The instrument proved to be valid and reliable by an argument-based validation analysis.

Keywords


instruments, evaluation, focus science, kirkpatrick model

Full Text:

PDF


References

  1. Presiden Republik Indonesia. Undang-Undang RI No.12 Tahun 2012 tentang Pendidikan Tinggi. Jakarta: Presiden Republik Indonesia; 2012.

  2. Menteri Pendidikan dan Kebudayaan Republik Indonesia. PerMen Ristek Dikti No.3 Tahun 2020: Standar Nasional Pendidikan Tinggi. Jakarta, Indonesia: Mendikbud Ristekdikti; 2020.

  3. Konsil Kedokteran Indonesia. Standar Kompetensi Dokter Indonesia. Kedua. Jakarta, Indonesia: Konsil Kedokteran Indonesia; 2012.

  4. Menteri Pendidikan dan Kebudayaan Republik Indonesia. Lampiran Peraturan Menteri Pendidikan dan Kebudayaan No. 139 Tahun. 2014 Tentang Pedoman Statuta dan Organisasi Perguruan Tinggi. 2014.

  5. LAM-PTKes. Buku II-Kriteria dan Prosedur Akreditasi Program Studi Pendidikan Profesi Dokter. Jakarta: LAM-PTKes; 2016.

  6. Kirkpatrick S. Understanding The Role of Vision, Mission, and Values in The HPT Model. Perform Improv. 2017;56(3):6–14.

  7. Dewanti SS. Standard Assessment Model for Integration-Interconnection As Material for Evaluation of Vision and Mission Achievement. J Kependidikan J Has Penelit dan Kaji Kepustakaan di Bid Pendidikan, Pengajaran dan Pembelajaran. 2023;9(3):771–81.

  8. Shaw S, Crisp V. An approach to validation: Developing and applying an approach for the validation of general qualifications. Res Matters A Cambridge Assess Publ. 2020;(3):3–44.

  9. Lauwaert P. On Validity. Vol. 23, Studies in Applied Linguistics and TESOL. 2023.

  10. Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: A practical guide to Kane’s framework. Med Educ. 2015;49(6):560–75.

  11. Kalkbrenner MT. A Practical Guide to Instrument Development and Score Validation in the Social Sciences: The Measure Approach. Pract Assessment, Res Eval. 2021;26(January):1–18.

  12. Sireci S, Benítez I. Evidence for Test Validation: A Guide for Practitioners. Psicothema. 2023;35(3):217–26.

  13. Kirkpatrick J, Kirkpatrick WK. An Introduction to The New World Kirkpatrick Model [Internet]. Newnan, USA: Kirkpatrick Patners; 2021. Available from: http://www.kirkpatrickpartners.com/Portals/0/Resources/White Papers/Introduction to the Kirkpatrick New World Model.pdf

  14. Haski-Leventhal D. On Purpose, Impact, Vision and Mission. In: The Purpose-Driven University. United Kingdom: Emerald Publishing; 2020. p. 7–22.

  15. Shakman K, Wogan D, Rodriguez S, Boyce J, Shaver D. Continuous Improvement in Education: A Toolkit for Schools and Districts. US Departement of Education; 2020.

  16. Alfagih M, Kariri I. A Proposal to Develop the Performance of School Leaders in Saudi Arabia in the Light of Human Performance Technology (HPT). Inf Sci Lett. 2022;11(6):2053–92.

  17. Von Soest C. Why Do We Speak to Experts? Reviving the Strength of the Expert Interview Method. Perspect Polit. 2023;21(1):277–87.

  18. Aiken LR. Content validity and reliability of single items or questionnaires. Educ Psychol Meas. 1980;40(4):955–9.

  19. Roebianto A, Savitri SI, Aulia I, Suciyana A, Mubarokah L. Content Validity: Definition and Procedure of Content Validation in Psychological Research. TPM - Testing, Psychom Methodol Appl Psychol. 2023;30(1):5–18.

  20. Aiken LR. Three coefficients for analyzing the reliability and validity of ratings. Educ Psychol Meas. 1985;45(1):131–42.

  21. Sabaruddin, Nur M, Fadli M, Mazlan. Content Validity of KIP-K Scholarship Instruments at Higher Education Using Aiken’s Coefficient Validity Scale. AL-TANZIM J Manaj Pendidik Islam. 2022;6(3):934–47.

  22. Anggraini D, Khumaedi M, Widowati T. Validity and Reliability Contents of Independence Assessment Instruments of Basic Beauty Students for Class X SMK. J Educ Res Eval. 2020;9(1):40–6.

  23. Mansyur FA, Arsyad, Manaf A. Analysis of The Aiken Index to Measure The Content Validity of The Wolio Language and Culture Preservation Model Instrument. ELS-Journal Interdiscip Stud Humanit [Internet]. 2023;6(3):567–73. Available from: http://journal.unhas.ac.id/index.php/jish/article/view/29667%0Ahttp://journal.unhas.ac.id/index.php/jish/article/download/29667/10337

  24. Hidayati R, Kusmanto AS, Kiswantoro A. Development and Construct Validation of Indonesian Students Self-Confidance Scale Using Pearson Product Moment. Pegem J Educ Instr. 2023;13(3):94–103.

  25. Jain S, Angural V. Use of Cronbach’s Alpha in dental research. Med Res Chronicles. 2017;4(3):285–91.

  26. Suhartini R, Ekohariadi, Nurlaela L, Wahyuningsih U, Yulistiana, Prihatina YI. Validity, Reliability, Intra-rater Instrument Parameter Teaching Factory and Learning Outcomes of Industrial Clothing. Proc Int Jt Conf Arts Humanit 2021 (IJCAH 2021). 2022;618:1230–9.

  27. Gundumogula M. Importance of Focus Groups in Qualitative Research. Int J Humanit Soc Sci [Internet]. 2020;8(11):299–302. Available from: https://hal.univ-lorraine.fr/hal-03126126/document

  28. Smith JD, Li DH, Rafferty MR. The implementation research logic model: a method for planning, executing, reporting, and synthesizing implementation projects. BMC Implement Sci. 2020;15(84):1–12.

  29. Robertson KN, Becho LW, Wingate LA. Logic Model Guide for ATE Projects. EvaluATE; 2023.

  30. Departement of Finance. Guidelines to Developing Outcomes: A Program Logic Model Approach. Government of Western Autralia; 2023. p. 2–5.



DOI: https://doi.org/10.22146/jpki.91960

Article Metrics

Abstract views : 927 | views : 620

Refbacks

  • There are currently no refbacks.


Copyright (c) 2024 Surya Akbar

Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.

Jurnal Pendidikan Kedokteran Indonesia (The Indonesian Journal of Medical Education) indexed by:

  

 JPKI Stats