Evaluating the Usability and User Experience of a Virtual Reality Painting Application


As part of my master’s degree, I designed and conducted a usability study on Tilt Brush, a VR painting app. I wrote a 21k word paper about the experiment and my findings, comparing users with and without formal artistic training. I designed an experiment and collected interview, observation, and survey data from participants. I coded the interview responses into themes, and highlighted key factors that contributed to and detracted from the usability and overall experience of the app.

This project gave me experience working on all aspects of a usability study from planning, design, recruiting, moderating, data collection, data analysis, drawing conclusions, and writing an academic research paper.

Some highlights of my findings were:

Find the paper here!

Abstract

The purpose of this study is to identify the factors that contribute to and detract from the usability and experience of Tilt Brush. It compares usability issues between users with formal training in art, and users with no formal training. Twenty participants were observed creating artwork in Tilt Brush and interviewed about their experience. They also answered a questionnaire based on the System Usability Scale.

The data collected identified that conventions from 2D apps and a sense of presence in the 3D virtual environment positively impacted usability. A lack of control, issues navigating virtual reality, and a lack of familiarity with Tilt Brush negatively impacted usability. The group with formal training overall wanted more tools that increased control over the artwork than the group without formal training. Finally, the user experience of Tilt Brush was positively impacted by immersion, novelty, tools, possibilities, and potential as a creative outlet.