publications

Ultrasonic mid-air haptic feedback enables the tactile exploration of virtual objects in digital environments. However, an object’s shape and texture is perceived multimodally, commencing before tactile contact is made. Visual cues, such as the spatial distribution of surface elements, play a critical first step in forming an expectation of how a texture should feel. When rendering surface texture virtually, its verisimilitude is dependent on whether these visually inferred prior expectations are experienced during tactile exploration. To that end, our work proposes a method where the visual perception of roughness is integrated into the rendering algorithm of mid-air haptic texture feedback. We develop a machine learning model trained on crowd-sourced visual roughness ratings of texture images from the Penn Haptic Texture Toolkit (HaTT). We establish tactile roughness ratings for different mid-air haptic stimuli and match these ratings to our model’s output, creating an end-to-end automated visuo-haptic rendering algorithm. We validate our approach by conducting a user study to examine the utility of the mid-air haptic feedback. This work can be used to automatically create tactile virtual surfaces where the visual perception of texture roughness guides the design of mid-air haptic feedback.

Interactive digital signage is increasingly deployed in urban environments, from airports and train stations, to cinemas and shopping malls, whilst their integration into public spaces introduces new possibilities in multimedia presentation. In this paper, we explore the impact of mid-air haptic feedback on user engagement during gesture-based interactions with digital posters. To that end, a user study with seventeen participants was undertaken with two independent variables: interactivity (high/low), and mid-air haptic cues (on/off), whilst user engagement and emotional affect was measured with respect to various metrics. In this first attempt to understand the significance of mid-air haptic interactions for digital signage, we found increased user engagement levels, comparable to, if not greater than those achieved by content gamification. In particular, our analysis suggests that mid-air haptic feedback significantly improved usability and aesthetic appeal in comparison to digital signage without haptic feedback. Similarly, a higher level of gamification was also found to boost user engagement and helped to offer more compelling experiences with digital signage.

Automated vehicles seek to relieve the human driver from primary driving tasks, but this substantially diminishes the connection between driver and vehicle compared to manual operation. At present, automated vehicles lack any form of continual, appropriate feedback to re-establish this connection and offer a feeling of control. We suggest that auditory feedback can be used to support the driver in this context. A preliminary field study that explored how drivers respond to existing auditory feedback in manual vehicles was first undertaken. We then designed a set of abstract, synthesised sounds presented spatially around the driver, known as Spatial Earcons, that represented different primary driving sounds e.g. acceleration. To evaluate their effectiveness, we undertook a driving simulator study in an outdoor setting using a real vehicle. Spatial Earcons performed as well as Existing Vehicle Sounds during automated and manual driving scenarios. Subjective responses suggested Spatial Earcons produced an engaging driving experience. This paper argues that entirely new synthesised primary driving sounds, such as Spatial Earcons, can be designed for automated vehicles to replace Existing Vehicle Sounds. This creates new possibilities for presenting primary driving information in automated vehicles using auditory feedback, in order to re-establish a connection between driver and vehicle.

As automated vehicles currently do not provide sufficient feedback relating to the primary driving task, drivers have no assurance that an automated vehicle has understood and can cope with upcoming traffic situations [16]. To address this we conducted two user evaluations to investigate auditory displays in automated vehicles using different types of sound cues related to the primary driving sounds: acceleration, deceleration/braking, gear changing and indicating. Our first study compared earcons, speech and auditory icons with existing vehicle sounds. Our findings suggested that earcons were an effective alternative to existing vehicle sounds for presenting information related to the primary driving task. Based on these findings a second study was conducted to further investigate earcons modulated by different sonic parameters to present primary driving sounds. We discovered that earcons containing naturally mapped sonic parameters such as pitch and timbre were as effective as existing sounds in a simulated automated vehicle.

There is currently a distinct lack of design consideration associated with autonomous vehicles and their impact on human factors. Research has yet to consider fully the impact felt by the driver when he/she is no longer in control of the vehicle [12]. We propose that spatialised auditory feedback could be used to enhance driver awareness to the intended actions of autonomous vehicles. We hypothesise that this feedback will provide drivers with an enhanced sense of control. This paper presents a driving simulator study where 5 separate auditory feedback methods are compared during both autonomous and manual driving scenarios. We found that our spatialised auditory presentation method alerted drivers to the intended actions of autonomous vehicles much more than all other methods and they felt significantly more in control during scenarios containing sound vs. no sound. Finally, that overall workload in autonomous vehicle scenarios was lower compared to manual vehicle scenarios.

We present a novel method of interaction for users to preview an audio track haptically. The preview enables users to "feel" the track they want to select, thus saving them from having to look at the screen or listen to the track before actually playing it. Our results show that users enjoyed the combination of audio and haptic feedback and that users would very much like to see this type of sensory collaboration being incorporated into their own mobile device.

The paper describes the creation of a mobile music player accompanied with synchronised haptic feedback to create a novel method of audio playback on a mobile device. There has been extensive research into the development of audio haptic systems to enhance user interaction on mobile devices as well as providing useful information about users’ surroundings while way-finding. We investigated an alternative collaboration of haptics and audio feedback which we present here. We believe that this collaboration provides a novel and fun method of interaction with a music playback application.Our results show that users enjoyed the combination of audio and haptic feedback and that users would very much like to see this type of sensory collaboration being incorporated into their own mobile device.