How to keep the sense of being physical/tangible in the VR experience?

"The window" proposes an alternative visual experience to confront the immersive effects of virtual reality, bridging the separated physical and digital world and constructing transitions to avoid being lost in the mediated vision.

The word "window" has been signified as the interface to the digital world for a long time thanks to the well-known operating system. The project plays with the "window" as a medium, hoping to pull the metaphor back from the digital to the physical, vision back to the body, and the technical mania to the anthropocentric emotion.

The project also critiques the boosterish promotion of virtual reality and speculates on its approaching visual hegemony in the near future. Using the concept and abstract significance of the window as a central critical image, my position focuses on the implicit impacts of this mediated reality on human perception, especially on the sense of being physical/tangible.

Introduction

"We put more energy into imagining our way out of places than imagining our way into them" (Yi-Fu Tuan, Escapism)

It's human nature to always keep expectations for the outside of window which could be out of curiosity to the unknown or the fear to the nature. So it is witnessed that the windows in the modern house get larger and larger until it takes up entire wall. People are pursuing better view to exchange sunlight and get close to nature.

People are building a window to get access to the nature in the physical world. In parallel, we also build up the well-known operating system - the windows which provides us a chance to glance at the digital world which we can escape ourselves in the virtual. The same trend happens again with the metaphor "window" that we are pursuing larger interfaces to get immersed into the digital world, from simple screens to VR goggles .

The reality is that people could lose their feeling at home and be rootless when they go outside and pursuing a nomad life. But they could rehabilitate the feeling at home, the comfort of connecting to base through locating and settle down. My concern is how people could restore the sense of being physical when they step out of in the virtual world in the long term.

Why window?

We are constructing a house to create binary - the inside and the outside, the home and the nature, the comfort and the adventure. People are building a plural relation through window which gives a chances to interface and mix the binary. You could glimpse into the nature meanwhile leave your body in the house. Reversely, you could enjoy immersing into a strange world meanwhile still keep the sense of staying at home. The window as a medium provides a third status in the tension and makes interactions possible.

How I got here?

AR/VR exploration (7 weeks)

I spent 7 weeks learning how to create AR and VR prototypes through unity and vuforia. I acquired lots of practices in making but got lost in exploring my point of view when engaging too deep in the medium. So I decided to get away from coding and content creating, and turned to create a speculative fiction imagining how people will adapt back to the physical world when they are getting accustomed to living with VR headsets in the future.

Speculative Fiction (7 weeks)

From my first part of exploration, I found that it is easy to be attracted by the contents in the digital world and be lost in thinking how our feeling and sensation is impacted by the medium. So using a cardboard box to simulate the VR headset, I'm committed to think from medium level instead of content in the speculative fiction. In my experiments I asked users to keep wearing the box on their head and blocking their vision letting them more focus on the body feeling.

I cut out holes on the headset and covered the holes with different layers like a mosaic filter, a microscope and a phone screen projecting the camera etc. I assumed that in that future human's vision are not supposed to only limited to naked eye vision or one hundred percent digital. There could be other forms of vision in the continuum like distorted vision or enlarged vision.

The speculative fiction provokes me to reconstruct the reality-virtuality continuum by Milgram & Kishino. Milgram and Kishino proposed the reality-virtuality continuum and in that spectrum the augmented reality and augmented virtuality bridges the two extremes. However, from the vision perspective, I believe what we perceived between the extreme of reality though eyes and virtuality is optical reality through lens and mediated reality through camera.

Reality - Optical Reality - Mediated Reality - Virtuality

We are already familiar with living and perceiving the optical reality for people who wear glasses all the time. The perceived vision actually is already intervened by people through the lenses. As for the mediated reality, the example is that human gain access to control and manipulate the perceived reality after mediation and digitalization like people could edit the photos in the photoshop and we are watching through screens. And virtuality, from the vision perspective, just the extreme of mediated reality instead of a parallel world.

VR Experience Design (10 weeks)

Based on my point of view from the speculative fiction, I get back to the current context and endeavor to think how to keep the sense of being physical before we stepped into the ubiquitous VR future. So I use the window to be the critical medium in the VR experience to connect the both sides, in other word, immersing in the virtual at the same time keeping the connection to the physical world.

Features

The whole system is composed of two parts including physical and digital window.

The physical window to connect the inside & the outside.

For the physical prosthetic part, I collaborated with product designer Luka Shengchen Fang. It is noticed that the VR headset blocking the user from the outside world and it is really hard to communicate with people in the physical world when the headset on. So we are designing a VR headset which could facilitate user smoothly transfer from reality to mediated reality (VR). Besides, the lighting on the glass indicates the player status which aims to help people outside to get more information before interacting with the player.

The digital window to connect the physical & the digital.

For the digital VR experience, I designed a toolkit including three different types of windows to connect the physical and digital world.

Physical-to-Digital Controller

P2D Controller provides a one stop button for user to jump completely from physical world back to the the digital world, or vise versa. Besides, I also experimented with mixed mood to experience the augmented reality showing parts of digital assets mixing in the physical world.

Interface Window

User could build up interface windows to merge the physical views into the digital world. By tapping on the window, the window is foldable depending on how immersive the user want the VR experience to be. User will also get informed when there are some visitors from physical world coming by.

Back Window

Human only use the forehead to explore directions. We overlook how we might make use of our back direction. So I am designing a back window as an exit in the VR experience for people to step back and quit the digital world.

(You could find more details in the attached website link.)