Much Ado About Immersion: Power, Reported Results, and the Validity of Research on the Psychology of Virtual Reality and Immersive Simulations

TR Number
Date
2018-06-26
Journal Title
Journal ISSN
Volume Title
Publisher
Virginia Tech
Abstract

Virtual reality (VR) technology has permeated consumer culture in recent years, consequentially inspiring a hotbed of interdisciplinary academic VR research to better understand its effects as a medium. It has become a popular subject of study in fields as varied as engineering, computer science, communication, and psychology. The present study evaluates methodological trends in behavioral research on VR in terms of best practices regarding data collection, reporting, and availability. A meta-scientific content analysis of 61 articles focused on power, p-values, reporting errors, and transparency of data, all of which respectively represent four stages of research: data collection, analysis, reporting, and sharing. The findings from 1,122 statistical tests show that there is room for improvement in much behavioral research on VR in terms of methodological trends regarding number of participants, reporting of results, and data availability. Although no firm conclusions can be drawn about the presence of p-hacking or other questionable research practices (QRPs), the present study demonstrates that chronically small sample sizes, instances of errors in reporting, and a lack of transparent supplemental data are evident. The trends observed are broad, yet informative, and further research in this area is crucial. Methodological recommendations are made for future research dealing with VR applications, particularly given the potential social and cultural impact of the technology.

Description
Keywords
Virtual reality, meta-science, open science, psychology, communication
Citation
Collections