Project HERMITS explores a way to open up a new level of versatility for table-top robots in tangible interaction design. Inspired by hermit crabs, we designed a modular system for table-top wheeled robots to dock to passive attachment modules, defined as 'mechanical shells'. Different types of mechanical shells can uniquely extend and convert the motion of robots with embedded mechanisms, so that, as a whole architecture, the system can offer a variety of interactive functionality by self-reconfiguration.

As for our proof-of-concept system, we have implemented an interactive system based on an off-the-shelf robotic toy, toio, with several modifications to enable the active docking capability. We have developed custom-built control architecture to handle the I/O for multiple robots using a single computer. A variety of mechanical shells were designed based on four primitive designs and fabricated with 3D printers. With this system, we introduce a rich application space including physical space organization, digital data physicalization, and entertainment and storytelling systems.

The general approach in HERMITS expands how physical interfaces and computers in our daily life can adapt and reconfigure for user interactions with passive attachments. We are in the age where robotic systems are emerging in our living space (e.g. robotic vacuum cleaner, drones). The idea presented in HERMITS has greater implications beyond our prototype, where everyday robotic systems gain a numerous amount of functionality, expressivity, and interactivity by switching mechanical shells.

HERMITS - Concept and System Design

HERMITS is a research concept and design for an interactive modular system for self-propelled tangible user interfaces (TUIs). The components of the system include the (1) self-propelled TUIs (robots), (2) mechanical shells, (3) an interaction stage, and (4) a computer to control the system. In the system, robots can selectively attach/detach to the mechanical shells by locomoting across the stage. While the robots themselves support generic interactions to users, a variety of mechanical shells allows the system to provide specific and reconfigurable interactivity to the users dynamically and on-demand. Regarding the number of robots, some of the mechanical shells in HERMITS can be docked and controlled with only a single robot, while others need to be activated by multiple robots. Hence, it is not necessary that HERMITS have to have a swarm of robots. For example, one robot activating a variety of mechanical shells one after another is an architecture option in the design of HERMITS. As a design space, we also consider having a swarm of robots to be an exciting option. This opens the door to interactivity with collective systems to push their versatility.

Mechanical Shell: Definition and Interaction Design Roles

The mechanical shell is a critical component in the design of HERMITS. We define the mechanical shell as "an external passive attachment to self-propelled TUIs that can be automatically attached or detached." The shell is an attachment that can modify, convert or extend the interactive functionalities of the actuated TUI.

For example, mechanical shells can modify the interactive property of 'shape', convert the locomotive 'motion' via transmission mechanisms, or extend other I/O capabilities by taking advantage of installed sensors and actuators in the device. The mechanical shells can be designed with combinations of these tangible augmentation capabilities.

Mechanical shells can serve multiple purposes. By reconfiguring its shape and converting mechanical I/O motion, the mechanical shells can enhance the dynamic physical affordances of the system and enrich its interaction space. For example, this enables the system to be used for tangible and haptic controllers as well as for concrete shape representation. Reconfiguring the shell with expressive and iconic shapes and motion would enhance its communication capability to users which can be a great tool for storytelling. Furthermore, the motion conversion capability allows the self-propelled TUIs to extend their robotic functionalities to affect the physical environment, such as with the manipulator and locomotor units.

The benefit of the mechanical shell concept is not only that they can provide each capability described above, but that they allow for the system to dynamically adapt and be reconfigured to flexibly support interactions.

APPLICATIONS

In order to demonstrate the strengths of HERMITS, we present its applications to fulfill three different use-cases.

Interactive Mobility Simulation

We designed an application for HERMITS to support a traffic intersection simulation. The actuated TUI enables vehicle motion to be simulated and for the location of different entities (e.g. buildings, traffic lights, as well as different vehicles, such as cars and busses) can be automatically adjusted to match the simulated data and view. They can simultaneously be controlled by users' for real-time tangible intervention with the simulation model. The mechanical shells in this application feature optic transmission of light and automated docking. The light is used to show the break-lamp of cars, as well as to control the color of the traffic signals at the intersection. This feature is critical to allow users to understand the intricate behavior of the traffic at the intersection. The importance of automated docking to shells is evident in this application, as the types of vehicles in the simulation can be consistently changing and updating. Finally, the interaction stage helps to reveal and conceal different entities as the robots transform their identities by docking different shells. The interaction stage permits modules to switch between on-stage and back-stage locations. This allows for robots and shells that are not needed to be full of out-of-view from the user, to match the simulated data.

Adaptive Environment/Desktop

HERMITS can also be used for enriching physical environments in our everyday lives with their high level of reconfigurability and adaptability. HERMITS can sequentially switch the interaction mode by reconfiguring their mechanical shells. On this desktop, there are several shells including a fan shell for temperature adjustment, the dozer shell to provide strong haptic feedback for 'push' notifications, and two vertical motion shells to adapt the height of a display. Beyond the desktop environment, we believe that the broad design space for mechanical shells can also be applied to dynamic physical environments in general. In a daily living environment, robotic vacuums are now commonly deployed. As this type of hardware share similar technical components with the wheeled system used in HERMITS, the idea of mechanical shells may give such robotic devices a way to handle extensively versatile tasks at home and in the office (e.g. reconfiguring furniture, climbing stairs, and moving obstacles away with grippers).

Storytelling

The strength of HERMITS is also highlighted through a storytelling application. Taking inspiration from Alice in Wonderland, we use modules to represent the scene where Alice chases after the rabbit that holds an iconic spinning watch. The units are able to play out the chasing motion between the characters across the stage. The mechanical shells bring these characters to life by giving them high-fidelity forms that closely resemble the characters. Similar to the Interactive Mobility Simulation, the stage enables the robots to change their shells out-of-view from the user and return to the visible stage with their correct appearance to match the story. This mimics how actors move on and off-stage to change costumes and play different characters to support the story. As the characters can be controlled by users as well, this application highlights HERMITS' ability to be used as a tangible educational tool for creating animated objects.

CONCLUSION

We introduced HERMITS, a novel approach for dynamic reconfigurable TUIs to augment the interactivity of self-propelled TUIs with passive attachments, named mechanical shells. We demonstrated this concept with a proof-of-concept implementation based on off-the-shelf wheeled robots. Our applications demonstrated the use of reconfigurability and adaptability in our approach to dynamically provide different interaction opportunities to users. With this project, we speculate the future of computational systems in physical environments to open up a new way to fuse actuated computational devices with passive mechanical modules to enrich our digital and physical interactions with greater reconfigurability.