See What I Mean (SWiM) is an app which translates words into pictures. The instant translation of word to image brings the idea into a moment of conversation and has been found to be effective at stimulating memories, improving communication, mood and levels of engagement amongst people with dementia.

For people living with dementia words can become increasingly hard to understand, making communication and social interaction difficult. Images, on the other hand, can retain their meaning long after the sense of a word has been lost. This means that for most people living with dementia an image communicates an idea or emotion more clearly than a word.

See What I Mean was originally created as a speech to image communication tool by Ilyanna Kerr (Co-Founder) in the final year of her design degree at Goldsmiths University. During the early stages of its development she found the app to have many potential uses through experimentation and testing with a wide range of people. This design process led her to discover it to be a powerful tool in helping to stimulate memory and interaction for people affected by dementia. Since then See What I Mean have continued to work alongside people living with dementia and their friends, family and carers to develop an app that meets their needs. The SWiM app is now being used by care providers in the UK to support the care of people living with dementia and will launch to the wider public in the Summer of 2015.

(The project video documents feedback from a dementia care day centre we piloted our app with. Production quality is low due to the difficulties involved in filming within this setting)

See What I Mean was not originally created with a specific end user in mind or as something that was developed to solve a specific problem. It emerged from an experimental design process during my design degree that investigated language, meaning making and visual experience and the possibilities for design and technology to harness the power images have to aid communication.

During this time, I designed and developed many experiments and prototypes that I tested with a wide range of people. This led to the identification of many potential users and use cases, One of the groups I tested with included people living with dementia, revealing its potential as a powerful communication tool in helping to stimulate memory and interaction.

Dementia is a term used to describe the symptoms that occur when the brain is affected by specific diseases and conditions such as Alzheimer's, there are over 100 different types of dementia. A 3rd third of us of us who live over the age of 65 will develop dementia and right now there's over 800,000 people with this condition in the UK and 44 million worldwide. Dementia creates distinct challenges in how people express themselves and understand what is being communicated to them. Losing this ability to communicate is one of the most frustrating and difficult problems for people with dementia, their families and carers. If people are not regularly engaged it can cause their health to rapidly deteriorate through a 'social death', despite this being a growing problem there is a real lack of solutions that meet this increasing need.

Towards the end of my final year project Peter (CTO) my computing tutor and I having identified our prototype and concept to be a valuable tool for people living with dementia wanted to develop it further to make it into something people living with dementia could access and benefit from. It was clear that we would only be able to reach a large number of people and create impact at scale by turning the project into a sustainable social venture. Just before I graduated we received investment, mentoring and support from Bethnal Green Ventures.

Starting up a social venture brings with it many challenges however the most important design challenge we faced from the beginning was ensuring that we created something that could meet the needs of people with dementia and their carers whilst making sure it could fit into current modes of care within the wider social care sector. To achieve this we took a co-design approach, involving all key stakeholders in the design, testing and development from its early stages. This way of working has proved invaluable to our development in terms of design and when considering the constant change within the social care sector as well as the way people use and interact with technology in general.

Our Point of View

Design within this context is not just about the way things look, its about how and what we choose to design and how well it caters for our 'wants' and 'needs' as we age. When designing for older adults it's important not to look at this group as a set of people who have reached a specific age or to categorise them as simply having a set of cognitive or physical impairments. Aging is far more complex so it's crucial to look at the wider context in which people live as they age; the shifts in economic and social circumstances and the effect this has on our engagement with technology. This is even more relevant when designing for dementia, as the disease will affect each person in a unique way creating a diverse set of needs that change as the disease progresses.

The Power of Pictures

It's long been recognized that Reminiscence Therapy – which uses prompts like photos or music to engage with and trigger memories for people living with dementia - has a significant and positive effect on mood, self-care, the ability to communicate and wellbeing. In some studies the therapy has even been shown to improve intellectual functioning [1]. Other reported benefits include enhancing the opportunity to provide personal and individualised care and assisting moves between different care environments such as home to care home, or between care homes [2] [3].

Speech to Image

Whilst designing and developing the See What I Mean app, it quickly became apparent that the app had the potential to amplify and build upon the positive impacts evidenced in Reminiscence Therapy (RT). When we used the app during RT sessions the instant translation of word to image, plus the fact that discussion was not limited just to hard copy images which might be to hand, encouraged a natural flow to conversation and greater levels of engagement.

The instant connection between word and image also has benefits beyond creating a natural flow to conversation. We have observed many times that seeing the word and the corresponding image together can be really useful in reassuring people of what the image is and means, giving that individual more confidence to talk about what they are seeing.

Music and Memory

Music has also long been known to be incredibly powerful in engaging and triggering memories in people living with dementia. At SWiM we found the combination of music and our app to be particularly powerful. In 2014 we piloted the app at a dementia care day centre. The carers and facilitators at the centre used the app as part of the music reminiscence sessions and brought up images relating to the music and the discussion in the group. The facilitators and carers found this combination of stimuli to work incredibly well in engaging people, especially in groups where music tastes may vary or if hearing is impaired.

How it Works

It's simple. You say a word and the SWiM app will find an image which corresponds to that word. For example, in a conversation about holidays, a trip to Paris is mentioned. The app brings up a selection of images of Paris, which in turn helps stimulate memory and encourages interaction.

The SWiM app has access to a huge range of images from Google and Getty Images. The app also allows you to save your favourites, so you can build a collection of images which hold particular meaning or interest.

We're currently developing a 'Personal' version of the app which will allow users to upload their own photographs, creating a library of meaningful images, unique to them.

The App

SWiM Care is designed for use within care environments such as day centres, home-care and residential settings. It makes a real difference to reminiscence and activity sessions - improving communication, engagement and mood for people with dementia. We offer training - including advice on best practice, suggestions for activities and on going customer support as part of the package.

SWiM Personal is currently in development. You can upload your own photos as well as access a huge range of images from Google and Getty Images allowing you to build collections meaningful to you. Available on the app store 2015.

The Benefits

The benefits of using images with people living with dementia have long been recognised, e.g reminiscence therapy. Images help stimulate memory and engagement, which in turn can improve wellbeing and quality of life.

With the SWiM app the conversation is not limited to the hard copy images you have to hand, and it's not slowed down by searching for images on the internet. The ease of use and instant translation of word to image encourages a natural flow to conversation. Images can be saved, creating a familiar and meaningful collection. This ability to refer back to these images can play an important role in supporting a person with dementia as their condition progresses.

Our Impact

In 2014 we tested the effects SWiM could have on communication, engagement and mood on people living with dementia at a dementia care centre.

It produced some remarkable results:

In 90% of sessions the app improved communication between participants and facilitators

In 70% of sessions the app led to a significant improvement in mood of the participants

The app clearly encouraged participant engagement and in 87% of sessions significantly increased the level of participant engagement

100% of care staff who used the app would use it on a regular basis and recommend it to friends and colleagues

See What I Mean was made available to care providers in 2014 and we are now working to develop a version we can release to the public this year. We will continue to work alongside people living with dementia to improve upon what we have already built as well as begin to validate other potential use cases.

We think now is a really important time to think of more solutions that can better support human interaction and emotional connection; it is a time for us to look to the future and think about our changing relationship with technology and its potential to support us in more holistic ways as we age.

References

-

1. Michael Craig Miller, M.D., Harvard Health Publications, abstract:

2. Murphy, 2000 – taken from Iriss http://www.iriss.org.uk/resources/supporting-those-dementia-reminiscence-therapy-and-life-story-work

3. http://memoryboxnetwork.org/reminiscence-2/