Developing economical yet effective methods of incorporating usability engineering as an integral part of software engineering is a primary focus of human-computer interaction (HCI) research. However, much HCI research has focused primarily on inspecting and evaluating applications supporting command-line or graphical user interface (GUI) interaction styles. With the dramatic increase in virtual environment (VE) research in recent years, the HCI community is beginning to place an added emphasis on creating methodologies to ensure usability in VE development. While the demand for VE-specific usability engineering methods and criteria is dire as the amount of money invested by military, government, commercial, and industrial organizations continues to grow, widely accepted methodologies for assessing VE usability are, at this point in time, minimal. There has been a recent increase in research discussing the need of VE-specific usability engineering methodologies, but few research projects have concentrated their efforts on providing such methodologies. Therefore, application developers attempting to apply a user-centered design approach in constructing VEs must often perform largely ad-hoc assessments or in-house evaluations using existing non-VE-specific usability engineering methodologies.

The primary focus of this research was to develop a method to guide usability engineering of VEs. The strategy used to develop this usability evaluation method was to modify existing usability evaluation methodologies to support VE development by leveraging the results of previous VE usability research performed at Virginia Tech and elsewhere. The result was a VE-specific usability evaluation methodology that encompasses two existing usability assessment techniques: usability inspections and formative evaluations. We applied this methodology to Crumbs, an immersive visualization VE developed at the National Center for Supercomputing Applications (NCSA).

A multi-dimensional framework of VE usability characteristics was a topic of research at Virginia Tech. This framework provided the backbone for VE-specific modifications to the existing usability evaluation techniques proposed in this research. Framework design guidelines permitted usability specialists to perform guidelines-based usability inspections of Crumbs. Results gathered from the guidelines-based usability inspections were used not only to redesign the Crumbs user interface but also drive creation of a formative evaluation plan. Application of the methodology not only uncovered usability issues with Crumbs, but also provided invaluable information regarding the effectiveness of the methodology itself. We conclude this thesis by describing a usability evaluation methodology, called the Modified Concept Book Usability Evaluation Methodology, proposed to improve the usability evaluation methodology performed on Crumbs and other VEs. Our methodology was adapted from an established methodology for performing content analysis on a large volume of qualitative data.

Because the realm of VEs is so vast and diverse in application domains and devices, we do not claim that our methodology supports an exhaustive usability evaluation of all VEs. However, the proposed technique is a first attempt at modifying existing usability evaluation methods, and therefore can be used as a launching pad for methodologies to evaluate other aspects of specific VE applications.