We believe that warehouse workers deserve more than just physical safety - they deserve to feel valued and supported in their workplace. We've developed a suite of cutting-edge technologies to make the warehouse a safer, happier place to work, including our three key innovations.

First, our collision alert and avoidance system is a game-changer for warehouse safety. Drawing inspiration from smart cars, we've developed a five-level alert system that helps workers respond quickly and appropriately to potential dangers. Our system tracks a worker's state and adjusts the collision boundary as well as the response to the system accordingly. This means that if a worker is carrying heavy boxes or moving quickly, the system will create a larger safety zone around them and note that if a collision occurs, it will need a higher level response as the chance of serious injury is higher. We've also integrated multiple communication modalities into the system, including visual indicators, voice user interface, navigation route changes in the chance of collision, and haptic feedback from the safety vest about potential collisions and a mobile app. By providing workers with real-time information and feedback, we're able to significantly reduce the risk of collisions and injuries on the job.

Our second innovation is the use of an artificial intelligence humanoid guide for route navigation. We designed a language and behavior database that includes reminders, greetings, guidance, cheers, and encouragement. By providing workers with a humanoid companion as the interface, Sensewear creates a supportive environment that enhances their mental health and makes them feel less isolated. Our AI humanoid guide helps workers navigate the warehouse safely and efficiently, reducing the risk of fatigue stress and injury. It also provides emotional support throughout the workday, offering positive reinforcement and motivation to help workers feel valued, accompanied and supported.

Finally, our AR dashboard provides real-time information about the work environment, including identity authentication, clock-out time, task lists, bird's eye mini-map, and obstacle avoidance indicators. This dashboard enhances productivity and ensures worker safety by providing workers with all the information they need to do their jobs safely and efficiently. Our obstacle avoidance indicators are particularly important, as they help workers avoid dangerous obstacles in real-time. By providing workers with this level of situational awareness, we're able to significantly reduce the risk of accidents and injuries in the workplace.

Overall, we believe that Sensewear has the potential to make a huge impact on the lives of warehouse workers. By providing cutting-edge safety technology and emotional support, we're able to create a safer, happier workplace for everyone. We're proud of what we've accomplished so far, and we're excited to continue pushing the boundaries of what's possible in workplace safety and wellness.

Introducing Sensewear, a groundbreaking solution designed to revolutionize warehouse safety and efficiency. Our project presents a unique system that combines mixed reality technologies with personal protective equipment, catering to the needs of warehouse workers and ensuring a safer and more supportive work environment. In the following description, we will delve into the key aspects of Sensewear's innovative design.

By exploring these critical aspects of Sensewear, we aim to demonstrate the impact our safety system can have on the warehouse environment, paving the way for a safer and more productive future in the industry.

Haptic PPE Vest

We started by rethinking what a Personal Protective Equipment (PPE) does and what it can do if it were to be coupled with mixed reality technologies. Our research of PPE led us to discover the various types that exist and how they were being used. We chose the vest because it was an equipment that people were irregardless of the warehouse's conditions and weather. PPE has traditionally been used so that people have more visibility in hazardous environments. We wanted to improve this visibility literally by allowing users to see others were the vest not only when they are in the field of view but also from the birds eye view. Other ways that we've improved visibility is also by not relying just on the eyes as senses to see but also using haptics to get a sense of the direction and proximity that a hazardous obstacle or person is coming from. The PPE also includes a singular button that allowed users to control their headset's state which takes away people's fear of using gestures / new controllers to engage with the emerging technology of AR. From our user testing, we've learned that constantly providing haptic feedback can make a person numb to it, hence we built a system logic that decides when haptics should be used and when AR overlays can be used instead. Moreover, the vest is designed in two distinct parts. The reflective outer layer houses the sensors and circuit board, while the inner layer serves primarily as a barrier between the person and the vest. This construction enables the vest to be washed without damaging the electronic components.

The Backend System's Logic

To develop a warehouse safety system, we began by categorizing the main hazard factors in warehouse environments into two groups: stationary and moving. Stationary objects include boxes and conveyor belts, while moving objects consist of people and forklifts. We then identified the states of these objects, with each state representing factors that could change the level of hazard in a given situation. For example, a box's state may depend on its weight or position, while a person's state may be based on their activities, such as resting, idle, moving, lifting, or carrying. Similarly, forklifts have different states like stationary, moving empty-handed, loading, moving with a load, and reversing.

Drawing inspiration from the concept of four distance zones in proxemics, we assigned different collider sizes to each state of a person. For instance, the resting mode would not have a collider, whereas the idle state would have a sphere collider similar to the proxemics of intimate space. A moving person would have a capsule collider with a larger buffer space to account for the increased potential danger compared to an idle person. The collider size changes when a person is lifting or carrying and walking, as this state can lead to the most severe injuries in case of a collision. The same logic applies to the other states. Our team successfully implemented the system to detect a person's various states and adjust colliders accordingly in Unity.

In addition to obstacles and states, we researched warehouse zoning, which is a method of dividing hazardous materials in warehouses. Our future steps will involve incorporating these zones into the system's calculation of danger levels.

After conducting thorough research and diagramming, we established five warning levels. These levels are activated based on the states of objects involved in potential collisions. For example, a person walking towards someone who is idle would trigger the lowest warning level, while two people carrying boxes and passing each other would activate a significantly higher level. Each level has different responses, ranging from lighting up the user on the minimap, avatar guides providing collision avoidance instructions, warning UI overlays, haptic feedback, and finally, alerting the general system in case of a collision to call for help and notify managers or supervisors for immediate medical attention. This system ensures that users are not constantly overstimulated by warning signals, thus preventing desensitization, and provides appropriate attention based on the relevant danger levels.

The Avatar Guide

The avatar guide plays a crucial role in the overall experience, as it not only assists with navigation but also offers a future projection trajectory for users to follow. Designed to predict potential collisions, the avatar guide is equipped with colliders, enabling users to see and avoid potential collisions before they occur. By following the guides, warehouse workers can efficiently navigate between tasks while reducing the risk of accidents.

In addition to serving as a helpful companion for often-isolated warehouse workers, the humanoid avatar guide can demonstrate proper techniques for lifting heavy equipment or interacting with appliances. We carefully adjusted the avatar's material style and translucency based on user feedback to prevent it from obstructing vision. We developed the latest shader using Unity's shader editor, creating a gradient design with increased transparency towards the top. We also experimented with 8 different voices before choosing what we felt fit our communication goals.

Our most recent project can be found in a separate GitHub repository (https://github.com/LucasThin/SenseWearDemo), while the full functionality prototype is available here (https://github.com/LucasThin/Warehouse) for inspection.

Visual Design

Our UI design underwent multiple iterations, initially starting with flat 2D interfaces such as arrows for navigation. Eventually, we identified three guiding keywords for our visual design style: Protective, Vigilant, and Composed. To embody these principles, we selected the Bai Jamjuree font for its rectangular yet subtly curved edges, reflecting a sense of protection and composure. We used Semi-bold weight to convey sturdiness. For a vigilant appearance, we incorporated colors commonly associated with warning signals, such as red, yellow, blue, and orange, while also emphasizing the composure aspect with blue.

In our second draft, we experimented with a black background and a neumorphic look to capitalize on the 3D medium. However, user testing revealed that this design obstructed vision and could potentially compromise safety. Consequently, we redesigned the UI with gradient shadows for increased translucency. To further declutter the user's field of view, we opted to display only the necessary information at a given moment, allowing them to remain more alert and aware of their surroundings.

Mobile App

Lastly, we developed a mobile application to address the potential discomfort from extended use of mixed reality headsets. By replicating the AR view's UI content on the mobile app, we offer users the flexibility to choose between a headset or their phone. While mobile users may not experience the same level of warning signs due to the absence of a HUD, we incorporated haptic and auditory signals to provide alternative alerts. Regardless of whether users wear a headset, the vest functions consistently. Even if a user without a headset cannot see their guide, others wearing headsets will still be able to see the guide in front of them.

In conclusion, our project presents a comprehensive solution that revolutionizes warehouse safety and efficiency by integrating cutting-edge mixed reality technologies with personal protective equipment. Our system prioritizes user experience through a series of iterative improvements and user testing, resulting in a dynamic and adaptable design that addresses various warehouse hazards and promotes a safer, more supportive work environment.

As we continue to develop and refine our project, we are currently in the process of transitioning our user interface from version 1 to version 2, incorporating valuable feedback and insights from our testing phase. This upgrade will further enhance the functionality and user experience of our safety system. We firmly believe that our innovative approach has the potential to significantly impact the lives of warehouse workers, paving the way for a safer and more productive future in the industry.