Pemahaman Peneliti Psikologi mengenai Besaran Sampel:  Data dan Simulasi

Wisnu Wiradhany; Krisna Adiasto; Jony Eko Yulianto; Indra Yohanes Kiling

doi:10.22146/jpsi.24260

Pemahaman Peneliti Psikologi mengenai Besaran Sampel: Data dan Simulasi

Wisnu Wiradhany, Krisna Adiasto, Jony Eko Yulianto, Indra Yohanes Kiling

(Submitted 21 April 2017)

(Published 6 August 2019)

Abstract

The lack of knowledge on how to determine sample sizes in experiments is arguably one of the main reasons underlying the replication crisis in psychological science. A survey distributed among Indonesian students and researchers concerning 1) familiarity and understanding of statistical concepts related to sampling size determination, 2) current sample size determination practices in experiments, and 3) ideal sample sizes for experiments. Subsequently, we simulated expected statistical power given the sample sizes reported in the survey. Results demonstrated that 1) while a majority of participants were somewhat familiar with statistical concepts related to sampling size determination, they did not always endorse the correct and/or complete definition of each concept. Furthermore, 2) participants relied on practical considerations in determining sample sizes. Consequently, 3) the reported sample sizes did not have sufficient power to detect small to medium effect sizes, which are commonly present in psychological science.

Keywords

effect size; replication crisis; sample size; statistical power

Full Text: PDF

DOI: 10.22146/jpsi.24260

References

Attali, Y., & Bar-Hillel, M. (2003). Guess where: The position of correct answers in multiple-choice test items as a psychometric variable. Journal of Educational Measurement, 40(2), 109–128. doi: 10.1111/j.1745-3984.2003.tb01099.x

Badenes-Ribera, L., Frias-Navarro, D., Iotti, B., Bonilla-Campos, A., & Longobardi, C. (2016). Misconceptions of the p-value among Chilean and Italian academic psychologists. Frontiers in Psychology, 7(August), 1247. doi: 10.3389/fpsyg.2016.01247

Badenes-Ribera, L., Frías-Navarro, D., Monterde-I-Bort, H., & Pascual-Soler, M. (2015). Interpretation of the p-value: A national survey study in academic psychologists from Spain. Psicothema, 27(3), 290–295. doi: 10.7334/psicothema2014.283

Bakker, M., Hartgerink, C. H. J., Wicherts, J. M., & van der Maas, H. L. J. (2016). Researchers intuitions about power in psychological research. Psychological Science. doi: 10.1177/0956797616647519

Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7(6), 543–554. doi: 10.1177/1745691612459060

Button, K. S., Ioannidis, J. P. A., Mokrysz, C., Nosek, B. A., Flint, J., Robinson, E. S. J., & Munafò, M. R. (2013). Power failure: Why small sample size undermines the reliability of neuroscience. Nature Reviews. Neuroscience, 14(5), 365–76. doi: 10.1038/nrn3475

Champely, S. (2009). Package “ pwr .” October, 1–21.

Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Statistical Power Analysis for the Behavioral Sciences, 2. doi: 10.1234/12345678

Cohen, J. (1990). Things I have learned (so far). American Psychologist. Retrieved from http://psycnet.apa.org/psycinfo/1991-11596-001

Cohen, J. (1992a). A power primer. Psychological Bulletin, 112(1), 155–159. doi: 10.1037/0033-2909.112.1.155

Cohen, J. (1992b). Statistical power analysis. Psychological Science. doi: 10.1111/1467-8721.ep10768783

Dufresne, R. J., Leonard, W. J., & Gerace, W. J. (2002). Making sense of students ’ answers to multiple-choice questions. The Physics Teacher, 40(March), 174–180.

Durlak, J. A. (2009). How to select, calculate, and interpret effect sizes. Journal of Pediatric Psychology, 34(9), 917–928. doi: 10.1093/jpepsy/jsp004

Gigerenzer, G. (2004). Mindless statistics. Journal of Socio-Economics, 33(5), 587–606. doi: 10.1016/j.socec.2004.09.033

Hoekstra, R., Morey, R. D., Rouder, J. N., & Wagenmakers, E.-J. (2014). Robust misinterpretation of confidence intervals. Psychonomic Bulletin & Review, 21(5), 1157–1164. doi: 10.3758/s13423-013-0572-3

Ioannidis, J. P. A. (2005). Why most published research findings are false. PLoS Medicine, 2(8), e124. doi: 10.1371/journal.pmed.0020124

Ioannidis, J. P. A. (2008). Why most discovered true associations are inflated. Epidemiology, 19(5), 640–648. doi: 10.1097/EDE.0b013e31818131e7

Ioannidis, J. P. A., Munafò, M. R., Fusar-Poli, P., Nosek, B. A., & David, S. P. (2014). Publication and other reporting biases in cognitive sciences: Detection, prevalence, and prevention. Trends in Cognitive Sciences, 18(5), 235–241. doi: 10.1016/j.tics.2014.02.010

Ioannidis, J. P. A., Ntzani, E. E., Trikalinos, T. A., & Contopoulos-Ioannidis, D. G. (2001). Replication validity of genetic association studies. Nature Genetics, 29, 306–309. doi: 10.1038/ng749

Lakens, D. (2013). Calculating and reporting effect sizes to facilitate cumulative science: A practical primer for t-tests and ANOVAs. Frontiers in Psychology, 4(NOV), 1–12. doi: 10.3389/fpsyg.2013.00863

Lindsay, S. (2015). Replication in Psychological Science. Psychological Science, 26(12), 1827–1832. doi: 10.1177/0956797615616374

Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716-aac4716. doi: 10.1126/science.aac4716

Pluye, P., & Hong, Q. N. (2014). Combining the power of stories and the power of numbers: Mixed methods research and mixed studies reviews. Annual Review of Public Health, 35(1), 29–45. doi: 10.1146/annurev-publhealth-032013-182440

R Core team. (2015). A language and environment for statistical computing. R Foundation for Statistical Computing , Vienna, Austria. ISBN 3-900051-07-0, URL http://www.R-Project.org/. Retrieved from http://www.mendeley.com/research/r-language-environment-statistical-computing-96/%5Cnpapers2://publication/uuid/A1207DAB-22D3-4A04-82FB-D4DD5AD57C28

Rouder, J. N. (2014). Optional stopping: No problem for Bayesians. Psychonomic Bulletin & Review, 21(2), 301–8. doi: 10.3758/s13423-014-0595-4

Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2011). False-positive psychology: Undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychological Science, 22(11), 1359–1366. doi: 10.1177/0956797611417632

Simonsohn, U., Nelson, L. D., & Simmons, J. P. (2014). P-curve: A key to the file-drawer. Journal of Experimental Psychology: General, 143(2), 534–547. doi: 10.1037/a0033242

Wagenmakers, E.-J. (2007). A practical solution to the pervasive problems of p values. Psychonomic Bulletin & Review, 14(5), 779–804. doi: 10.3758/BF03194105

Wasserstein, R. L., & Lazar, N. A. (2016). The ASA’s statement on p -values: Context, process, and purpose. The American Statistician, 70(2), 129–133. doi: 10.1080/00031305.2016.1154108

Wickham, H. (2010). A layered grammar of graphics. Journal of Computational and Graphical Statistics, 19(1), 3–28. doi: 10.1198/jcgs.2009.07098

Refbacks

There are currently no refbacks.

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.