Auditory cues can facilitate the visually-induced self-motion illusion (circular vection) in Virtual Reality - Riecke, B. E.; Schulte-Pelkum, J.; Caniard, F.; Bülthoff, Heinrich H. (8. Tübingen Perception Conference (TWK), 2005)
INTRODUCTION: There is a long tradition of investigating the self-motion illusion induced by rotating visual stimuli ("circular vection"). Recently, Larsson et al. (2004) showed that up to 50% of participants could also get some vection from rotating sound sources while blindfolded, replicating findings from Lackner (1977). Compared to the compelling visual illusion, though, auditory vection is rather weak and much less convincing. METHODS: Here, we tested whether adding an acoustic landmark to a rotating visual photorealistic stimulus of a natural scene can improve vection. Twenty observers viewed rotating stimuli that were projected onto a curved projection screen (FOV: 54$^°$x40.5$^°$). The visual scene rotated around the earth-vertical axis at 30$^°$/s. Three conditions were randomized in a repeated measures within-subject design: No-sound, mono-sound, and 3D-sound using a generic head-related transfer function (HRTF). RESULTS: Adding mono-sound showed only minimal tendencies towards increased vection and did not affect presence-ratings at all, as assessed using the Schubert et al. (2001) presence questionnaire. Vection was, however, slightly but significantly improved by adding a rotating 3D-sound source that moved in accordance with the visual scene: Convincingness ratings increased from 60.2% (mono-sound) to 69.6% (3D-sound) (t(19)=-2.84, p=.01), and vection buildup-times decreased from 12.5s (mono-sound) to 11.1s (3D-sound) (t(19)=2.69, p=.015). Furthermore, overall presence ratings were increased slightly but significantly. Note that vection onset times were not significantly affected (9.6s vs. 9.9s, p>.05). CONCLUSIONS: We conclude that adding spatialized 3D-sound that moves concordantly with a visual self-motion simulation does not only increase overall presence, but also improves the self-motion sensation itself. The effect size for the vection measures was, however, rather small (about 15%), which might be explained by a ceiling effect, as visually induced vection was already quite strong without the 3D-sound (9.9s vection onset time). Merely adding non-spatialized (mono) sound did not show any clear effects. These results have important implications for the understanding or multi-modal cue integration in general and self-motion simulations in Virtual Reality in particular. SUPPORT: EU grant POEMS-IST-2001-39223 (see www.poems-project.info) and Max Planck Society.
Keywords: Riecke; Riecke_2005; bernhard-riecke; iSpaceWeb; multi-modal vection; talkWithoutPaper; vection