twitter YouTube LinkedIn

The E word – why we’re talking about evaluation



Photo showing group of performers of mixed ages on stage, many holding their arms above their heads with a screen showing a photo of a hand in the background

Evaluation in the cultural sector isn’t easy. In this joint blog our director Ben Walmsley and co-director Anne Torregiani, along with Oliver Mantell from The Audience Agency, discuss the essence of the Centre’s new Evaluation principles.

Evaluation has been a core strand of the Centre for Cultural Value’s activity since its conception. When we were bidding to host the Centre and sketching out its delivery plan, we were inspired by the diverse and creative methods fostered by the Arts and Humanities Research Council’s Cultural Value Project.

We were also aware that there was a deep-set problem with evaluation in the cultural sector, which often privileged advocacy over rigour. The result was that it often wasn’t taken seriously, even by the few funders who actually engaged with it.

When we travelled around the UK in 2019/20 to scope out the Centre’s work, we also heard about the heavy toll that evaluation can take on organisations, especially smaller ones. Whilst we experienced nothing short of a passion for high quality evaluation amongst the sector, we also heard tales of skills and confidence gaps, of funders’ apparent disengagement, and of a disconnect between evaluation and positive change.

Our mission at the Centre for Cultural Value is to develop a shared understanding of the differences that arts, culture, heritage and screen make to people’s lives and to society. We won’t be able to realise this mission if we don’t broker honest conversations about evaluation and embed a more rigorous and sustainable approach to conducting it. So these new, co-created evaluation principles represent a starting point on this journey. Whilst we are confident that they reflect and promote good practice, they can only ever be the start of an ongoing dialogue that will ultimately foster a community of engaged practitioners, researchers, funders and policymakers bonded by a common passion for learning.

Our core principles

Over the past year our expert working group, comprising more than 40 academics, cultural leaders and consultants, has co-developed a set of principles to guide cultural evaluation. After a series of interactive discussions and workshops, the group cohered around four key principles: that evaluation should be beneficial, robust, people-centred and connected. A set of sub-principles lie beneath each of these core values to offer the nuance required of cultural evaluation. Collectively, these principles represent an attempt to provide a new way of thinking about evaluation that places audiences and participants at its heart. We hope they privilege meaningful learning, reflection and positive change over empty justification and advocacy.

During the process of developing the principles, a number of underlying challenges rose to the fore. For example, the group discussed how there is simultaneously too much and too little evaluation in the cultural sector. Lots of effort goes into producing low-value monitoring and reporting, but too little time is dedicated to the kind of deep reflection and learning required to develop critical understanding and enrich creative practice.

We hope we’ve lived up to our principles in the way in which we’ve produced them. We’ve worked with a wide range of people with significant experience in cultural evaluation and remain open to evolution of the principles based on other perspectives. Our ultimate goal is to support the development of a cultural sector that is confident to acknowledge failure and robust enough to enable evaluation to inform genuine change. A key part of this is the appropriate and rigorous application of mixed methods, helping cultural organisations to not just know ‘what happened’, but understand why and how.

Who is involved in evaluation matters: who decides what is worth evaluating and how it is carried out, whose voice is present in reporting and dissemination, who gets to hear and act on findings. We know that the cultural sector is often unrepresentative and exclusive. Evaluation offers a means to witness, challenge and address these problems, but is also capable of concealing, excusing and perpetuating them. Ignoring this bias is less, not more, objective. Increasing the range and number of viewpoints in evaluation enables better insight, not worse.

Evaluation is too often seen as serving the needs of funders, rather than organisations or most importantly – users and the wider public. We argue for a vision of evaluation which centres the experiences, wants, values and viewpoints of the people who are at the heart of the purpose of cultural activity in the first place. This is what funders tell us they want too.

Evaluation is an area of skilled practice, but part of that skill is about balancing expertise with openness. After all, evaluation of culture is necessarily involved with differing, evolving and even contested ideas of value, which evaluation should serve to articulate, rather than resolve. Evaluation is at its most useful when those involved are participating in a shared learning journey, and we should aspire to both learn from and contribute to the learning of each other. To do this requires humility, honesty, trust, curiosity and empathy. It is in this spirit that we share our own learning and launch the new Evaluation principles.

Take a look at the co-created Evaluation principles. You can also join the conversation on social media using the hashtag #TheEWord

Image: Orchestras Live, The Lost Letters with Britten Sinfonia at Saffron Hall. Photo: Paul Bellany. 

Related news

A group of people talking in a circle at Rising Arts Agency's 'This Is The Work' event, surrounded by shelves of books at Bristol's Bookhaus bookshop.
Rising Arts Agency - This Is The Work (Photo by Olu Osinoiki)
News

This Is The Work: reflecting on new research findings about power in partnerships

How do power imbalances manifest in creative partnerships - and how do we remove the impact of these imbalances from ...
Pam Johnson is dressed in black and sitting on a table against a white background. To the site is a plant in a terracotta pot.
 
News

Talking cultural policymaking with Pam Johnson, Leeds City Council

Policymaking can seem to many a mysterious process. Meanwhile, the term 'policymakers' can conjure up ideas of a homogeneous group ...
Performance of 12 last songs. Colourful confetti in the foreground obscures the image.
Quarantine's 12 Last Songs, at HOME Theatre Manchester. Photo by Chris Payne.
News

In Arts Professional: The Beauty Project

Artist Sarah Hunter, physicist Rox Middleton and philosopher Lucy Tomlinson spotlight what they learnt from spending a year exploring the ...
Two people using a digital touch screen together in a museum
 
News

Watch online: Evaluation for Arts, Culture and Heritage Workshop

Have you taken our new free-to-access online course, Evaluation for Arts, Culture and Heritage: Principles and Practice? Did you miss ...
Profile of a person in a bookshop. They are wearing a bright yellow hoodie and holding a camera on a tripod.
Photo by Olumide Osinoiki/Rising Arts Agency.
News

In Arts Professional: Power in partnerships

In their collaborative research project, Euella Jackson and Jess Bunyan of Rising Arts Agency have been exploring the unequal balance ...
A Beauty Project Lab book open on two pages. One page has yellow and pink circles and writing by a research participant. The other page is green with the word Beauty.
The Beauty Project. Photo by Lisa Mattocks.
News

Revealing the beauty of using lab books for audience research

Can returning to pen and paper provide more in-depth audience research data? The research team behind The Beauty Project, one ...






Keep in touch,

Sign up to our newsletter