Jumbli

Breath detection is the low hanging fruit of VR input. Head tracking already provides the location and rotation of your mouth and the mic provides the ability to crudely detect and measure exhalations. Feedback on my old DK2 experience, Breath Tech Demo, confirmed that seeing and using your breath helps to increase immersion. Try the remastered Breath Tech experience for free and see what you think.

While your HMD's built-in microphone allows us to prototype the possibilities of breath detection, it could be so much better. I believe that HMDs should include a low cost breath sensor to more reliably detect when you exhale and ideally even inhale.

Experiencing the world without your voice and breath leaves you somewhat detached, as though you're looking out from a bubble or spacesuit. Seeing your breath extend into a cold environment makes you feel more connected. Simple actions such as blowing away dustmotes, blowing out a candle or misting up a mirror allow natural interactions that you don't give a second thought to in real life. Subtle responses to automatic or natural interactions can help achieve a sense of presence or at the very least increase immersion.

The more that we represent a user's physicality in the virtual world the more natural it feels. You’ve experienced it with your hands or seeing your shadow / reflection move with you. It's the same with your breath, but breath is less visual than the former examples so you are less likely to question the accuracy of the representation.

I don't think so.

Hardware costs: We can already prototype and create some reasonable first attempts using microphones and I don't expect dedicated sensors to be high cost if included in HMDs together with next gen features.

Resource usage: The resource usage is tiny. I only use 8000 samples per second which means we need to process less than 100 samples per frame at 90 fps.

Implementation effort: I've documented my experience so far to help with this: Implementing Breath Detection In VR

Meditation and pain control
VR is already being used for meditation and to help people control pain. Adding breath detection can make these experiences aware of the users breathing, giving valuable feedback that can be used to increase the effectiveness of the experience.

Sports and fitness
Monitoring the player's breath allows us to better understand the players state. Games could automatically step up the work rate or give rest periods as required to keep the player at optimum levels.

An accessible interaction method

A VR user interface can be driven with very little movement and sensitive breath detection, allowing us to be more inclusive towards people with disabilities. When coupled with eye tracking there is so much potential for flexible interactions.

Character awareness
Characters in the virtual world should be able to hear you breathing heavily or talking. Even if they don't understand you, your presence should at least be recognised. This can already be largely achieved just using the microphone, but it's very rarely implemented. A breath sensor would help to differentiate between the player talking and background noise. If you're chatting away while stealthily sneaking up on an enemy, I have to doubt how immersed you feel :)

Avatars and user understanding
Together with facial recognition and audio capture, breath could provide data to make more accurate avatars. Your chest could expand as you take a deep breath, expressions could change or you could become flushed when out of breath. The more data we have about the user the more we can reflect and accommodate their state in the experience.

Try my Breath Tech mini game to experience seeing and using your breath in VR. Do you think breath detection is a gimmick or something that will be commonplace in the future?

If, like me, you want a dedicated breath sensor as standard in the next generation of VR devices please be vocal about it so that manufacturers can recognise the level of customer interest, #weWantToBreatheInVR.

Connect with me for more news and insights