Learner Outcomes in Calculus-Based Physics Using Physics Education Technology Simulations: A Meta-Analysis

Main Article Content

Jorge Victor M. Sales https://orcid.org/0000-0002-6490-3626
Jhoanne E. Rafon-Paghubasan https://orcid.org/0009-0000-4855-190X
Maricar S. Prudente https://orcid.org/0000-0003-1156-0380

Keywords

Blended learning, calculus-based, inquiry, physics, PHET, simulation, teaching

Abstract

PhET became a tool for learning Calculus-Based Physics (CBP) alongside inquiry-based teaching. A meta-analysis using a random-effects model was conducted following PRISMA guidelines to synthesize outcomes from eligible studies published from January 2006 to December 2025 in selected research databases. Of the 649 studies that were culled, 27 studies (N=10,062) met the criteria, with a moderate inter-rater agreement (κ=0.674,p<.001). A significant positive moderate random effect size (ES) (g=0.580,p<.001) indicated positive improvement in CBP learning. Moderator analysis further detected a positive effect on the learning outcomes among high school learners (g=1.144,p< .001), and when inquiry through exploration (g=0.898,p< .001) was employed. The implications of its effectiveness depend on the sustainability, digital nativeness, mathematical preparation, curriculum integration of inquiry-teaching frameworks, and the appropriateness of the digital tools used.

Abstract 0 | PDF Downloads 0 Video Abstract Downloads 0

References

Agustina, R. D., & Putra, R. P. (2022). Sophisticated thinking blended laboratory (STB-LAB) learning model: Implications on virtual and real laboratory for increasing undergraduate student’s argumentation skills. Jurnal Pendidikan IPA Indonesia, 11(4), 657–671. https://doi.org/10.15294/jpii. v11i4.38772.

Almadrones, R. D., & Tadifa, F. G. (2024). Physics educational technology (PHET) simulations in teaching general physics 1. International Journal of Instruction, 17(3), 635-650. https://doi. org/10.29333/iji.2024.17335a

Ashraf, M. A., Yang, M., Zhang, Y., Denden, M., Tlili, A., Liu, J., Huang, R., & Burgos, D. (2021). A systematic review of systematic reviews on blended learning: Trends, gaps and future directions. Psychology Research and Behavior Management, 14, 1525-1541. https://doi.org/10.2147/PRBM. S331741

Astutik, S., Mahdiannur, M. A., Suliyanah, L., & Prahani, B. K. (2019). Improving science process skills of junior high school students through the implementation of collaborative creativity learning (CCL) model in physics learning. Journal of Physics: Conference Series, 1171(1), 012006. https://doi.org/10.1088/17426596/1171/1/012006

Bohrnstedt, G. W., Ogut, B., Yee, D., & Bai, Y. (2023). AP Calculus and Science coursetaking: Their relationships with choosing a STEM major and expecting to be in a STEM occupation. AERA Open, 9, 23328584231184426.

Borenstein, M. (2022). Comprehensive meta-analysis software. In M. Egger, J. P. T. Higgins, & G. D. Smith (Eds.), Systematic reviews in health research: Meta-analysis in context (3rd ed., pp. 535–548). John Wiley & Sons. https:// doi.org/10.1002/9781119099369.ch27

Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2009). Fixed-effect versus random-effects models. Introduction to Meta-analysis, 77, 85.

Borenstein, M., Hedges, L. V., Higgins, J. P., & Rothstein, H. R. (2021). Introduction to meta-analysis. John Wiley & Sons. Borenstein, M., Higgins, J. P. T., Hedges, L. V., & Rothstein, H. R. (2017). Basics of is not an absolute measure of heterogeneity. Research Synthesis Methods, 8(1), 5-18. https://doi.org/10.1002/jrsm.1230

Brahmia, S. W., Olsho, A., Smith, T. I., & Boudreaux, A. (2020). Framework for the natures of negativity in introductory physics. Physical Review. Physics Education Research, 16(1). https://doi.org/http://dx.doi.org/10.1103/PhysRevPhysEducRes.16.010120

Bruun, M., Willoughby, S., & Smith, J. L. (2018). Identifying the stereotypical who, what, and why of physics and biology. Physical Review Physics Education Research, 14(2), Article 020125. https://doi.org/10.1103/ PhysRevPhysEducRes.14.020125

Brydges, C. R. (2019). Effect size guidelines, sample size calculations, and statistical power in gerontology. Innovation in Aging, 3(4), Article igz036. https://doi.org/10.1093/geroni/igz036

Budnyk, O. (2019). Innovative competence of a teacher: Best European practices. Journal of Vasyl Stefanyk Precarpathian National University, 6(1), 76-89. https://doi.org/10.15330/jpnu.6.1.76-89

Burghardt, M., Ferdinand, P., Pfeiffer, A., & Romagnoli, G. (2020). Integration of new technologies and alternative methods in laboratory-based scenarios. International Conference on Remote Engineering and Virtual Instrumentation. https://doi.org/10.1007/978-3-03052575-0_40

Couch, A. S. (2014). The impact of virtual simulations on student comprehension of mechanics (Publication No. 151) [Master's thesis, Louisiana State University]. LSU Scholarly Repository. https://repository.lsu.edu/gradschool_ theses/151

Doherty, M. (2010). The effect of daily graded reflective journaling on gains in conceptual understanding and the scientific attitude toward conceptual understanding for high school students studying Newtonian mechanics (Publication No. 3424167) [Doctoral dissertation, University of Massachusetts Lowell]. ProQuest Dissertations and Theses Global.

Do, T. D., & Zsolnai, A. (2022). Teachers' social and emotional competence: A new approach of teacher education in Vietnam. Hungarian Educational Research Journal, 12(2), 131-144. https://doi. org/10.1556/063.2021.00050

Forero, D. A., Lopez-Leon, S., Gonzalez-Giraldo, Y., & Bagos, P. G. (2019). Ten simple rules for carrying out and writing meta-analyses. PLoS Comput Biol, 15(5), e1006922. https://doi.org/10.1371/journal.pcbi.1006922

Helm, F., & Beaven, A. (Eds.). (2020). Designing and implementing virtual exchange – A collection of case studies. Research-publishing.net. https://doi.org/10.14705/rpnet.2020.45.9782490057726

Hurtado-Bermúdez, S., & Romero-Abrio, A. (2020). The effects of combining virtual laboratory and advanced technology research laboratory on university learners' conceptual understanding of electron microscopy. Interactive Learning Environments, 1-16. https://doi.org/10.1080/104948 20.2020.1821716

Kaldaras, L., & Wieman, C. (2023). Introducing an instructional model for teaching blended math-science sensemaking in undergraduate STEM courses using computer simulations. arXiv. https://doi.org/10.48550/arXiv.2305.13451

Klassen, M., Jesiek, B. K., Zheng, L., & Case, J. M. (2022). Institutionalizing engineering education research: Comparing Australia, China, and the United States. In S. Christensen, C. Baillie, & G. Östlund (Eds.), Philosophy of engineering and technology: Engineering, social sciences, and the humanities: Have their conversations come of age? (pp. 33-63). Springer. https://doi.org/10.1007/978-3-03111601-8_3

Kohl, P., Kuo, V., & Ruskell, T. (2008). Documenting the conversion from traditional to studio physics formats at the Colorado School of Mines: Process and early results. AIP Conference Proceedings, 1064, 135-138. https://doi.org/10.1063/1.3021247

Kohn, P. K., Underwood, S. M., & Cooper, M. M. (2018). Energy connections and misconnections across chemistry and biology. CBE—Life Sciences Education, 17(1), ar3. https://doi.org/10.1187/cbe.17-08-0169

Madsen, A., McKagan, S. B., & Sayre, E. C. (2015). How physics instruction impacts learners' beliefs about learning physics: A meta-analysis of 24 studies. Physical Review Special Topics-Physics Education Research, 11(1), Article 010115. https://doi.org/10.1103/ PhysRevSTPER.11.010115

Massey-Allard, J. (2021). Learning physics with interactive simulations: Inductive inquiry learning activities for an introductory electromagnetism course [Doctoral dissertation, University of British Columbia]. UBC Theses and Dissertations. https://open.library.ubc. ca/cIRcle/collections/ubctheses/24/items/1.0402206

Molina, V. C., Guzman, D. S., Montalvo, T. R., & Garcia-Salcedo, R. (2025). Teaching competencies in physics for engineering education: A qualitative analysis from teaching practice. arXiv. https://doi.org/10.48550/arXiv.2510.27674

Moore, E., & Perkins, K. (2014a). Assessing the implicit scaffolding design framework: Effectiveness of the build a molecule simulation. In Proceedings of the National Association for Research in Science Teaching (NARST) Annual International Conference. ComPADRE. https://www.per-central.org/items/detail.cfm?ID=16318

Moore, E. B., Chamberlain, J. M., Parson, R., & Perkins, K. K. (2014b). PhET interactive simulations: Transformative tools for teaching chemistry. Journal of Chemical Education, 91(8), 1191-1197. https://doi.org/10.1021/ed4006497

Neidorf, T., Arora, A., Erberber, E., Tsokodayi, Y., & Mai, T. (2019). Review of research into misconceptions and misunderstandings in physics and mathematics. In T. Neidorf, A. Arora, E. Erberber, Y. Tsokodayi, & T. Mai (Eds.), Learner misconceptions and errors in physics and mathematics: Exploring data from TIMSS and TIMSS Advanced (pp. 11–20). Springer. https://doi.org/10.1007/9783-030-30188-0_2

Pedaste, M., Lazonder, A., Raes, A., Wajeman, C., Moore, E., & Girault, I. (2016). Grand Challenge Problem 3: Empowering science teachers using technology-enhanced scaffolding to improve inquiry learning. In T. de Jong, M. Specht, & F. Wild (Eds.), Grand challenge problems in technology-enhanced learning II: MOOCs and beyond (pp. 17-20). Springer. https://doi.org/10.1007/9783-319-12562-6_5

Peters, M., von Doetinchem, P., & von Doetinchem de Rande, S. (2023). Virtual physics laboratory courses: An evaluation of students' self-efficacy and intelligence mindset. arXiv. https://doi.org/10.48550/arXiv.2301.02699

Pospiech, G., Michelini, M., & Eylon, B.-S. (Eds.). (2019). Mathematics in physics education. Springer. https://doi. org/10.1007/978-3-030-04627-9

Redish, E. F. (2021). Using math in physics: Overview. The Physics Teacher, 59(5), 314-318. https://doi.org/10.1119/10.0004880

Rothstein, H. R., Sutton, A. J., & Borenstein, M. (2006). Publication bias in meta-analysis: Prevention, assessment and adjustments. John Wiley & Sons. https://doi.org/10.1002/0470870161

Shi, L., & Lin, L. (2019). The trim-and-fill method for publication bias: Practical guidelines and recommendations based on a large database of meta-analyses. Medicine, 98(23), e15987-e15987. https://doi.org/10.1097/MD.0000000000015987

Soderstrom, N. C., & Bjork, R. A. (2015). Learning versus performance: An integrative review. Perspectives on Psychological Science, 10(2), 176-199. https://doi. org/10.1177/1745691615569582

Sukarelawan, R. M., Pranata, R., & Putri, A. (2024). The effectiveness of PhET simulations in physics education: A comprehensive meta-analysis of secondary vs. tertiary outcomes (2018-2023). Indonesian Science Education Journal, 17(3), 632-648.

Sterne, J. A., Hernán, M. A., Reeves, B. C., Savović, J., Berkman, N. D., Viswanathan, M., Henry, D., Altman, D. G., Ansari, M. T., & Boutron, I. (2016). ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions. BMJ, 355, i4919. https://doi.org/10.1136/bmj.i4919

Taylor, J. M., & Alanazi, S. (2023). Cohen's and Hedges'g. Journal of Nursing Education, 62(5), 316-317. Thomas, G. (2016). Metacognition in science education: considering cultural and contextual orientations. Alberta Science Education Journal, 44(2), 4-15.

Tokarz, K., Czekalski, P., Drabik, G., Paduch, J., Distefano, S., Di Pietro, R., Merlino, G., Scaffidi, C., Sell, R., Kuaban, G. S., Gansch, R., Grocholewicz, M., Harasymiv, S., Kocur, D., Orłowski, M., Pławiak-Mowna, A., Podaza, J., Robert, J., Schillinger, A., . . . Žagar, M. (2020). Internet of things network infrastructure for the educational purpose. 2020 IEEE Frontiers in Education Conference (FIE), 1–8. https://doi.org/10.1103/ FIE44824.2020.9274040

Tunggyshbay, M., Balta, N., & Admiraal, W. (2023). Flipped classroom strategies and innovative teaching approaches in physics education: A systematic review. Eurasia Journal of Mathematics, Science and Technology Education, 19(6), Article em2283. https://doi.org/10.29333/ejmste/13258

UNICEF. (2023). SDG 4 Quality Education: Ensure inclusive and equitable quality education and promote lifelong learning opportunities for all. UNICEF. Retrieved June 12, 2023 from https://www.unicef.org/unicef-andsustainable-development-goals#sdg4

Yadak, P. (2020). A meta-analysis of gender gap on the FCI in high school and college introductory physics courses in the US and international countries (Publication No. 552) [Doctoral dissertation, University of San Francisco]. USF Scholarship: a digital repository @ Gleeson Library | Geschke Center. https://repository. usfca.edu/diss/552

Yunzal, A. N., Jr., & Casinillo, L. F. (2020). Effect of Physics Education Technology (PhET) simulations: Evidence from STEM students' performance. Journal of Education Research and Evaluation, 4(3), 221-226. https://doi.org/10.23887/jere.v4i3.27450