twitter YouTube LinkedIn

Embracing evaluation for the right reasons



How can we make evaluation more meaningful? In this blog, Holly Donagh from the Paul Hamlyn Foundation reflects on some of the challenges associated with evaluation, as well as what can be done to re-frame the discussion.

The word ‘evaluation’ can carry a heavy load in the arts and cultural sector, as well as generating a certain amount of anxiety and resistance. I think this is because it is often seen as something academic or requiring specialist knowledge, which particularly for non-specialist senior leaders or trustees can be a barrier.

Evaluation can also be unhelpfully bound-up with funding discussions and a fear that anything less than a glowing evaluation might mean a difficult conversation (or in the case of outcomes-based funding, a real loss of income). And of course the whole idea of ‘value’ and therefore ‘evaluation’ in the context of culture is contentious, moving across the very personal to the highly political, and playing into a deeply ingrained sense of the need (often, but not always, justified) to fight for the value of art and culture as a public good.

Paul Hamlyn Foundation helped set up and fund the Centre for Cultural Value in part to break open this discussion. We think that talking about value is interesting and that developing our collective facility with understanding and demystifying evaluation, research and knowledge-creation is necessary for progress.

A key debate is whether it is possible to evaluate the outcome of activity in a cultural context. There is certainly a lot of challenge to the idea of outcome-based models in public policy. Toby Lowe’s is very interesting on the perversity created within systems which demand simple answers to complex situations. In this reading there is a form of complicity between funders asking for evidence without interrogating the basis on which knowledge could credibly be obtained and organisations supplying data to fit this model, despite not having confidence in its origin.

There are many people working on this subject who are better placed than me to comment on whether outcomes can ever be meaningful in arts and culture. Whether or not that is the case I think there is more we should do to understand and unpack what we can observe. For example, what it is about the way an organisation or an individual interacts that makes them original and different and how can that knowledge be shared amongst practitioners?

In turn this might help with testing the logic of how different forms of engagement could lead to certain outcomes – asking what mechanisms or ‘intermediary outcomes’ are in play – thereby providing a better knowledge base to build-on.

Aligned with this would be a cultural shift away from expecting ‘transformative’ results. This is not to doubt the extraordinary power that all kinds of engagement with arts and culture can have on people and communities, but it is to say that to calibrate around exceptional stories of growth or change sets an expectation that is not necessarily helpful for learning. The sector needs to have the confidence to be comfortable dealing in small but relevant changes, understanding what really were the factors that made the difference – even if these were more about the context of the work than the engagement itself.

The Centre’s work on evaluation principles will be a great opportunity for professionals in the sector to debate these ideas and hopefully bring-in voices from outside the sector who are grappling with similar challenges. I feel comfortable that in most situations a re-framing of evaluation towards ‘learning’ is effective and useful, but there may be circumstances where it is appropriate to look for more definitive objective assessments. Having the space to debate these approaches with colleagues would help the field as a whole.

Perhaps evaluation would generate less anxiety if it was brought into the centre of discussions about the purpose, value and intent of practice and not kept at arm’s length. This would help with meeting head-on the current debate about equity in the context of evaluation. Who decides on the framing? Who is in the room thinking about what good looks like? How are people and communities involved, not as research subjects but as active commissioners? This touches on the bigger debate within the social research sector around the power dynamics inherent in evaluation and how they are unpacked and ultimately challenged. The arts has a real opportunity to bring some interesting ideas into this space because of the strong tradition of imaginative and collaborative modes of working but it would need to do so with humility and openness.

As we wake-up from a period of change and uncertainty it feels like a good moment to re-set intentions towards embracing evaluation for the right reasons, part of how we learn and grow together.

Holly Donagh is Director of Strategic Learning, Insight and Influence at the Paul Hamlyn Foundation.

Take a look at the co-created Evaluation principles. You can also join the conversation on social media using the hashtag #TheEWord

Related news

Collaborate research project. A large black square structure in the centre of leeds Market. The structure has a range of screens on it, one screen has the word ‘Time Bar’ written on it, one is playing a video of a person, one has a pub quiz. A group of people are watching the screens.
Photo: Lizzie Coombes
News

Collaborate fund – first cohort announced

Image credit: The Yorkshire Square by Small Acts Presented at Compass Festival 2021. Photo: Lizzie Coombes. We are delighted to ...
Person wearing a mask stands in a shop window next to poster titled 'This Poem is for You'. They appear to be smiling and reaching out to two people walking past outside.
Photo: Andrew Moore
News

Insight into action: policy recommendations launched at UK Parliament

Image credit: “This Poem Is For You” by Andrea Mbarushimana, Shop Front Theatre, Coventry, commissioned by Theatre Absolute. Photo: Andrew ...
Adah Parris
Photo: Adah Parris
News

Welcome new Advisory group and team members

We are pleased to share news of new appointments to our Advisory group. The Advisory group acts as a link ...
Light Night on Campus
Photo: University of Leeds
News

Fully-funded PhD opportunity – Evaluating Cities of Culture

Earlier this year we announced that we are working with LEEDS 2023 as their research partner in collaboration with The ...
A large board has text that reads: "369. Is the world how you imagined?" In the foreground people are moving cables and working on the stage.
Quarantine, 12 Last Songs at Transform Festival, October 2021 Photo: David Lindsay
News

Collaborate – the matchmaking phase

Image credit: Quarantine, 12 Last Songs at Transform Festival, 2021. Photo: David Lindsay Liz Harrop and Lisa Baxter write about ...
A building is reflected on the window of another building. There is a poster saying "Building Closed until further notice" on the window.
The Lowry in lockdown. Photo: Nathan Chandler
News

Culture in crisis – new report from our major research project into the impacts of Covid-19

UK cultural sector at a crossroads, as new research reveals profound impact of Covid-19 on workforce, audiences and organisations The ...






Keep in touch,
sign up to our newsletter



You can unsubscribe at any time by clicking the link in the footer of our emails or by emailing ccv@leeds.ac.uk For information about our privacy practices, please visit our website www.culturalvalue.org.uk

We use Mailchimp as our marketing platform. By clicking below to subscribe, you acknowledge that your information will be transferred to Mailchimp for processing. Learn more about Mailchimp's privacy practices here.