See What I
Mean (SWiM) is an app which translates words into pictures. The instant
translation of word to image brings the idea into a moment of conversation and
has been found to be effective at stimulating memories, improving
communication, mood and levels of engagement amongst people with dementia.
living with dementia words can become increasingly hard to understand, making
communication and social interaction difficult. Images, on the other hand, can retain their
meaning long after the sense of a word has been lost. This means that for most
people living with dementia an image communicates an idea or emotion more
clearly than a word.
See What I Mean was originally created as a speech
to image communication tool by Ilyanna Kerr (Co-Founder) in the final year of
her design degree at Goldsmiths University. During the early stages of its
development she found the app to have many potential uses through
experimentation and testing with a wide range of people. This design process
led her to discover it to be a powerful tool in helping to stimulate memory and
interaction for people affected by dementia. Since then See What I Mean have continued
to work alongside people living with dementia and their friends, family and
carers to develop an app that meets their needs. The SWiM app is now being used
by care providers in the UK to support the care of people living with dementia
and will launch to the wider public in the Summer of 2015.
(The project video documents feedback from a dementia care day centre we piloted our app with. Production quality is low due to the difficulties involved in filming within this setting)
Ilyanna Kerr - Designer
Peter Rogers - Developer
Simon Jeal - Designer
See What I Mean was not originally created with a
specific end user in mind or as something that was developed to solve a
specific problem. It emerged from an experimental design process during my design degree
that investigated language, meaning making and visual experience and the possibilities for design and
technology to harness the power images have to aid communication.
time, I designed and developed many experiments and prototypes
that I tested with a wide range of people. This led to the identification of
many potential users and use cases, One of the groups I tested with included people living with
dementia, revealing its potential as a powerful communication tool in helping
to stimulate memory and interaction.
Dementia is a term used to describe the symptoms that occur when the brain is affected
by specific diseases and conditions such as Alzheimer's, there are over 100 different types of
dementia. A 3rd third of us of us who live over the age of 65 will develop dementia and
right now there's over 800,000
people with this condition in the UK and 44 million worldwide. Dementia
creates distinct challenges in how people express themselves and understand
what is being communicated to them. Losing this
ability to communicate is one of the most frustrating and difficult problems
for people with dementia, their families and carers. If people are not
regularly engaged it can cause their health to rapidly deteriorate through a
'social death', despite this being a growing problem there is a real lack of solutions
that meet this increasing need.
end of my final year project Peter (CTO) my computing tutor and I having identified
our prototype and concept to be a valuable tool for people living with dementia
wanted to develop it further to make it into something people living with
dementia could access and benefit from. It was clear that we would only be able to
reach a large number of people and create impact at scale by turning the
project into a sustainable social venture. Just before I graduated we received
investment, mentoring and support from Bethnal Green Ventures.
Starting up a social venture brings with it many
challenges however the most important design challenge we faced from the
beginning was ensuring that we created something that could meet the needs of
people with dementia and their carers whilst making sure it could fit into
current modes of care within the wider social care sector. To achieve this we
took a co-design approach, involving all key stakeholders in the design,
testing and development from its early stages. This way of working has proved
invaluable to our development in terms of design and when considering the constant
change within the social care sector as well as the way people use and interact
with technology in general.
Our Point of View
Design within this context is not just about the way things look, its about how and what we
choose to design and how well it caters for our 'wants' and 'needs' as we age.
When designing for older adults it's important not to look at this group as a
set of people who have reached a specific age or to categorise them as simply
having a set of cognitive or physical impairments. Aging is far more complex so
it's crucial to look at the wider context in which people live as they age; the
shifts in economic and social circumstances and the effect this has on our
engagement with technology. This is even more relevant when designing for
dementia, as the disease will affect each person in a unique way creating a
diverse set of needs that change as the disease progresses.
The Power of Pictures
been recognized that Reminiscence Therapy – which uses prompts like photos or
music to engage with and trigger memories for people living with dementia - has
a significant and positive effect on mood, self-care, the ability to
communicate and wellbeing. In some studies the therapy has even been shown to
improve intellectual functioning . Other reported benefits include
enhancing the opportunity to provide personal and individualised care and
assisting moves between different care environments such as home to care home,
or between care homes  .
Whilst designing and developing the See What
I Mean app, it quickly became apparent that the app had
the potential to amplify and build upon the positive impacts evidenced in
Reminiscence Therapy (RT). When we used the app during RT sessions the instant translation of word to image, plus
the fact that discussion was not limited just to hard copy images which might
be to hand, encouraged a natural flow to conversation and greater levels of
The instant connection between word and image also has benefits beyond creating a
natural flow to conversation. We have observed many times that seeing the word
and the corresponding image together can be really useful in reassuring people
of what the image is and means, giving that individual more confidence to talk
about what they are seeing.
Music and Memory
also long been known to be incredibly powerful in engaging and triggering
memories in people living with dementia. At SWiM we found the combination of
music and our app to be particularly powerful. In 2014 we piloted the app at a
dementia care day centre. The carers and facilitators at the centre used the
app as part of the music reminiscence sessions and brought up images relating
to the music and the discussion in the group. The facilitators and carers found
this combination of stimuli to work incredibly well in engaging people,
especially in groups where music tastes may vary or if hearing is impaired.
How it Works
It's simple. You say a word and the SWiM app will find an image which corresponds to
that word. For example, in a conversation about holidays, a trip to Paris
is mentioned. The app brings up a selection of images of Paris, which in
turn helps stimulate memory and encourages interaction.
app has access to a huge range of images from Google and Getty Images. The app
also allows you to save your favourites, so you can build a collection of
images which hold particular meaning or interest.
We're currently developing a 'Personal' version of
the app which will allow users to upload their own photographs, creating a
library of meaningful images, unique to them.
SWiM Care is designed for use within care environments such
as day centres, home-care and residential settings. It makes a real difference
to reminiscence and activity sessions - improving communication, engagement and
mood for people with dementia. We offer training - including advice on best
practice, suggestions for activities and on going customer support as part of
SWiM Personal is currently in development. You can upload
your own photos as well as access a huge range of images from Google and Getty
Images allowing you to build collections meaningful to you. Available on the
app store 2015.
The benefits of using images with people living with dementia have long been
recognised, e.g reminiscence therapy. Images help stimulate memory and
engagement, which in turn can improve wellbeing and quality of life.
With the SWiM app the conversation is not limited to
the hard copy images you have to hand, and it's not slowed down by searching
for images on the internet. The ease of use and instant translation of word to
image encourages a natural flow to conversation. Images can be saved, creating
a familiar and meaningful collection. This ability to refer back to these
images can play an important role in supporting a person with dementia as their
In 2014 we tested the effects SWiM could have
on communication, engagement and mood on people living with dementia at a
dementia care centre.
It produced some remarkable results:
In 90% of sessions the app improved communication between participants and facilitators
In 70% of sessions the app led to a
significant improvement in mood of the participants
The app clearly encouraged participant
engagement and in 87% of sessions significantly increased the level of
100% of care staff who used the app would use it on a
regular basis and recommend it to friends and colleagues
See What I Mean was made available to care providers in 2014 and we are now working
to develop a version we can release to the public this year. We will continue
to work alongside people living with dementia to improve upon what we have
already built as well as begin to validate other potential use cases.
We think now is a really important time to think of more solutions that can better
support human interaction and emotional connection; it is a time for us to look to the future and
think about our changing relationship with technology and its potential to
support us in more holistic ways as we age.
1. Michael Craig Miller, M.D., Harvard Health Publications, abstract: