"The best way to remember the dead is to honor the lives they lived."

- Yaffa Eliach

Tessellate created an Augmented Reality (AR) experience in the iconic Tower of Faces (TOF) gallery at the United States Holocaust Memorial Museum in Washington, DC, as a celebration of life. Challenged by the Museum in an open call for proposals, we wanted to find both a human and technical solution to tell the authentic story of the thriving life and culture of Eishishok, a Jewish community destroyed by Nazi German perpetrators and their local helpers during the Holocaust.

The USHMM is a living memorial to the Holocaust, inspiring citizens and leaders worldwide to confront hatred, prevent genocide, and promote human dignity. Spanning multiple floors within the Museum's Permanent Exhibition, the Tower of Faces displays over 1,000 photographs collected by Yaffa Eliach. The photographs portray ordinary individuals and everyday life in Eishishok. The Museum sought an AR experience to enhance and facilitate deeper engagement with the photographs, without disturbing the existing visitor experience.

As visitors walk through the TOF, an Augmented Reality experience brings a selection of 30 portraits to life through text, spoken narration, and imagery. An initial moment of color overlaid on the original black-and-white photography conveys the vibrant spirit of the community and the individuals who lost their lives. The experience is self-guided: visitors select AR-activated photos, hear the stories behind the photos, and even see relationships among people through animated connections visualized by AR, which lead visitors to nearby related portraits. An open-ended exploration approach allows visitors to meander through photographs and dive deeper into personal stories. Our goal is for people to leave the Museum with a profound understanding of life and humanity, encouraging them to navigate daily life with a new perspective.

All technology and physical exhibit elements were designed to be accessible to all and follow a credo to "Do No Harm" to the existing architecture and tone of the display. The non-intrusive design blends seamlessly with the historical exhibit and building style, allowing the photographs' narratives to be the central focus. The devices are positioned along the walls of the gallery for optimal visitor circulation, supporting the 1.6M visitors the Museum welcomes annually. A visual prompt indicates to visitors that the tablets can be picked up for exploration. Once picked up, the tablet detaches from its magnetic charging stand, and visitors start their journey with an overview of Eishishok and the bonds among all the individuals in the exhibit.

The experience utilizes custom, site-specific AR software tightly mapping the real space with its digital twin and a back-end software platform that the Museum can administer, serving as a scalable system that evolves over time when new stories and features are added. Hardware includes an 11" tablet for visitor interaction and media presentation, a magnetic inductive charging mount, and a robust mounting solution including custom-designed handles. The exhibit launched publicly in May 2022, with over 70,000 visitors viewing 107,687 AR-activated photos to date.

The Tower of Faces (TOF) is a key gallery within the main exhibition at the United States Holocaust Memorial Museum (USHMM). It is a space that brings visitors face-to-face with portraits of hundreds of ordinary citizens from a small town called Eishishok, the majority of whom were murdered by Nazi German perpetrators and their local helpers. However, many visitors passed through the space without lingering, misunderstood the context, or were left wondering who the people depicted in the photographs were, and what happened to them. The Tower of Faces Augmented Reality (AR) experience was designed to achieve critical comprehension goals, support the gallery's primary message, and operate within two important constraints: 1) no signage specific to the AR experience or iPads, and 2) no Museum staff for visitor guidance, or for device management and maintenance. Finally, the experience had to grapple with sensitive subject matter within a space memorializing the lives of real people.

AR content conveys the primary message & achieves comprehension goals by:

• Briefly providing overall context about the Eishishok community. An introductory video underscores the gallery's primary message: One flourishing community with many years of history was destroyed in just a few days in September 1941.

• Telling the stories of individuals' lives and fates, with an emphasis on personal details that illuminate "sparks of life." As one visitor observed, "Everyone has a story. It means more when you know the story behind the photos."

• Placing individuals in the context of a family and community, which encourages an understanding of what was lost with the destruction of longstanding Jewish communities during the Holocaust.

DESCRIPTION OF EXPERIENCE

A visitor walks into the Tower of Faces gallery. They see a selection of iPads and pick one up. The software detects that the device has been picked up; immediately a short contextual video begins and tells the story of Eishishok with text, photo, and audio narration.

The video ends with a prompt that directs the visitor to explore the stories of the people in the Tower by choosing a photo that interests them from a curated selection of 30 black-and-white images at eye level. As the visitor holds the iPad in front of the gallery walls, AR-activated photos glow and signal interactivity. The visitor points the iPad at a photo of interest to select it. The photo briefly appears on the iPad screen in full color, marking a moment of wonder before transitioning back to black-and-white. The story of the person or people in the photo plays with text and audio narration. The story ends and the iPad directs the visitor to explore a related photo, or to select a new photo that interests them.

The visitor selects a new photo. This time, a different voice narrates the story. The visitor turns to share and discuss the story with a friend who is also exploring the gallery. When this story concludes, they choose to explore a related photo. A virtual ribbon of light marks a path from the current photo to the related one, visualizing the ties that connected the close-knit town community, while guiding the visitors through a spatial narrative of their own choosing.

After exploring several photo stories over the course of several minutes, the visitor and friend return the iPad to its place and leave the gallery. They leave the gallery with the understanding that a flourishing community with many years of history was destroyed in just a few days. They feel a sense of the "spark of life" that quickly ended and perhaps remember a specific person from the gallery. They have new questions and new insights. And, for a brief moment, the community of Eishishok appears to have come back to life.

TECHNICAL OVERVIEW

The interactive system can be broken down into three main components: the AR Software running on tablet devices, the Experience Management System, and Physical Design.

AR Software

• 30 photos with narrative sequencing, and specific links to related photos.

• Application is built in Unity/C# and supported by the Vuforia AR framework.

• iOS devices running custom AR software, locally installed.

The AR experience leverages the latest spatial recognition technology powered by PTC's Vuforia. It is enabled by a custom software specifically designed and capable of precisely recognizing and tracking the Tower of Faces gallery, while tightly mapping interactive content both to the room, and individual photos. All content lives on the device, avoiding loading times and the need for network usage and/or network overloads. Content can be updated by the Experience Management System during the Museum's off hours.

EXPERIENCE MANAGEMENT SYSTEM

• Customized build running on Ruby.

• Analytics and usage logs available on demand for all devices.

• Staff-friendly website for photo curation and visitor behavior analysis

We could have called it a Content Management System (CMS) but it turned out to be so much more.

The experience backbone is a custom-designed management system that allowed us to accomplish the following: creation and exploration of a content repository system; processing, management, and distribution of this content; and managing the devices remotely. Early on, our custom content management and research tools shone, quickly becoming a center of collaborative research between the Museum and Tessellate teams. Especially during a time of remote work, this central "map" of the gallery's photographs and their digitized information was indispensable for content creation and testing. The backend system allowed our teams to iterate on user interface labels and prompts, which was (and continues to be) critical for ensuring visitor success with the devices. Furthermore, the EMS allows the Museum to have quantitative tools to measure and analyze participants' engagement for each interaction in the TOF gallery.

PHYSICAL DESIGN

• Tablet devices are iPad Pro 11" running the latest operating system

• Induction charging stations (wireless) using iPort charger and iPort case

• Custom 3D-printed handles

The experience was designed and built to run on large and light modern tablet devices. A custom-designed protective case with two handles provides comfortable holding surfaces for single or bi-manual handling. Installed along the gallery's main walls are induction charging stations featuring wireless charging and a specially-designed magnetic latch to easily place and pick up the devices. The placement of the charging stations maximizes circulation: USHMM receives 1.6 million visitors annually. Integrating the charging stations directly into the wall of photographs required zero impact on the existing mounting structure. The discreet appearance of the charging stations and devices signals availability without detracting from the memorial atmosphere.

RESULT

The AR experience has achieved project goals and is widely used by Museum visitors. Thousands of visitors—nearly 20,000 over a 6-week testing period in the summer of 2022—are spending more time in the Tower of Faces gallery, engaging deeply with AR-activated photos, and demonstrating a stronger, more accurate awareness of the gallery's core meaning. Among the most gratifying observations is the significant number of visitors who share the devices with friends, family members, or classmates. The social aspect strengthens learning and sparks meaningful conversation.

Designing a solution that would not rely on day-to-day intervention from staff was a significant consideration. Furthermore, AR interventions in museums are still rare, and visitors have not yet developed an ease and familiarity with picking up gallery devices on their own. No one is on hand to explain how to pick up, use, and replace the devices. Through clear design affordances, visitors are empowered to use the devices and experience something beyond what the physical space provides, even if they have never used AR before.

The combination of backend software, seamless user experience design, and accessible physical design powers a visitor-centric, visitor-led discovery of stories of ordinary people and their experiences before and during the Holocaust.